On one of our remote systems mkdir -p $directory fails when the directory exists. which means it shows
mkdir: cannot create directory '$directory' : file exists
This is really puzzling, as I believed the contract of -p was that is always succeed when the directory already exists. And it works on the other systems I tried.
there is a user test on all of these systems, and directory=/home/test/tmp.
This could be caused if there is already a file by the same name located in the directory.
Note that a directory cannot contain both a file and folder by the same name on linux machines.
Check to see if there is a file (not a directory) with a name same as $directory.
mkdir -p won't create directory if there is a file with the same name is existing in the same directory. Otherwise it will work as expected.
Was your directory a FUSE-based network mount by any chance?
In addition to a file with that name already existing (other answer), this can happen when a FUSE process that once mounted something at this directory crashed (or was killed, e.g. with kill -9 or via the Linux OOM killer).
Check in mount if the FUSE mount is still listed there. If yes, you should be able to unmount it and fix the situation using fusermount -uz.
To see what is happening in detail, run strace -fy mkdir -p $directory, which shows all syscalls involved and their return values.
I consider the error messages emitted in this case a bug in mkdir -p (in particular the gnulib library):
When you run it on a dir that had a FUSE process mounted but that process crashed, it says
mkdir: cannot create directory ‘/mymount’: File exists
which is rather highly inaccurate, because the underlying stat() call returns ENOTCONN (Transport endpoint is not connected); but mkdir propagates up the less-specific error from the previous mkdir() sycall.
It's extra confusing because the man page says:
-p, --parents
no error if existing, make parent directories as needed
so it shouldn't error if the dir exists, yet ls -l / shows:
d????????? ? ? ? ? ? files
so according to this (d), it is a directory, but it isn't according to test -d.
I believe a better error message (which mkdir -p should emit in this case) would be:
mkdir: cannot create directory ‘/mymount’: Transport endpoint is not connected
Related
I have a program called HelloWorld belonging to user test
HelloWorld will create a file HelloWorld.pid in /var/run to keep single instance.
I using following command to try to make test can access /var/run
usermod -a -G root test
However, when I run it, falied
could someone help me?
What are the permissions on /var/run? On my system, /var/run is rwxr-xr-x, which means only the user root can write to it. The permissions do not allow write access by members of the root group.
The normal way of handling this is by creating a subdirectory of /var/run that is owned by the user under which you'll be running your service. E.g.,
sudo mkdir /var/run/helloworld
sudo chown myusername /var/run/helloworld
Note that /var/run is often an ephemeral filesystem that disappears when your system reboots. If you would like your target directory to be created automatically when the system boots you can do that using the systemd tmpfiles service.
Some linux systems store per-user runtime files in /var/run/user/UID/.
In this case you can create your pid file in /var/run/user/$(id -u test)/HelloWorld.pid.
Alternatively just use /tmp.
You may want to use the user's name as a prefix to the pid filename to avoid collision with other users, for instance /tmp/test-HelloWorld.pid.
I use rsync for automatic periodic syncing of the home folder (root user) in a linux server that is used by several people. A service that users need is the possibility of mounting remote directories through sshfs. However, when there is an sshfs mount, rsync fails giving the following messages
rsync: readlink_stat("/home/???/???") failed: Permission denied (13)
IO error encountered -- skipping file deletion
...
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1183) [sender=3.1.1]
Because of this error, the automated sync does not work as expected, in particular due to skipping the file deletion and a non-zero exit code. The sync is only necessary for the file system where home is mounted, so the wanted behavior is that the sshfs mounts be ignored. The -x / --one-file-system rsync option does not resolve it.
This problem is clearly explained in https://www.agwa.name/blog/post/how_fuse_can_break_rsync_backups . The follow-up article (https://www.agwa.name/blog/post/easily_running_fuse_in_an_isolated_mount_namespace) proposes a solution, though not an acceptable one because fuse mounts are only visible to the process created the mount.
I am looking for a solution that does not affect sshfs usability and is transparent for the users.
The problem is that FUSE denies stat access to other users, including root. Rsync requires stat access on all source files and directories specified. But when an rsync process owned by another user stats a FUSE mount-point, FUSE denies that process access to the mount-point's attributes, causing rsync to throw the said "permission denied" error. Mauricio Villega's solution works by telling rsync to skip FUSE mount-points listed by the mount command. Here is another version of Villega's solution that specifies a white-list of filesystem types using the findmnt command. I chose ext3 and ext4 but you may add other types as needed.
#!/bin/sh
# Which paths to rsync (note the lack of trailing slash tells rsync to preserve source path name at destination).
SOURCES=(
/home
)
# Which filesystem types are supported.
FSTYPES=(
ext3
ext4
)
# Rsync each source.
for SOURCE in ${SOURCES[#]}; do
# Build exclusion list (array of "--exclude=PATH").
excludedPaths=$(findmnt --invert --list --noheadings --output TARGET --types $(IFS=',';echo "${FSTYPES[*]}"))
printf -v exclusionList -- "--exclude=%s " ${excludedPaths[#]}
# Rsync.
rsync --archive ${exclusionList[#]} --hard-links --delete --inplace --one-file-system ${SOURCE} /backup
done
Note that it builds the exclusion list inside the loop to address a fundamental problem with this solution. That problem is due to rsync'ing from a live system where a user could create new FUSE mount-points while rsync is running. The exclusion list needs to be updated frequently enough to include new FUSE mount-points. You may divide the home directory further by each username by modifying the SOURCES array as shown.
SOURCES=(
/home/user1
/home/user2
)
If you are using LVM, an alternative solution is rsync from an LVM snapshot. An LVM snapshot provides a simple (e.g., no FUSE mount-points) and frozen view of the logical volume it is linked to. The downside is that you must reserve space for the LVM snapshot's copy-on-write (COW) activity. It is crucial that you discard the LVM snapshot after you are done with it; otherwise the LVM snapshot will continue to grow in size as modifications are made. Here is a sample script that uses LVM snapshots. Note that it does not need to build an exclusion list for rsync.
# Create and mount LVM snapshot.
lvcreate --extents 100%FREE --snapshot --name snapRoot /dev/vgSystem/lvRoot
mount -o ro /dev/mapper/snapRoot /root/mnt # Note that only root has access to this mount-point.
# Rsync each source.
for SOURCE in ${SOURCES[#]}; do
rsync --archive --hard-links --delete --inplace --one-file-system /root/mnt/${SOURCE} /backup
done
# Discard LVM snapshot.
umount /root/mnt
lvremove vgSystem/snapRoot
References:
"How FUSE Can Break Rsync Backups"
This error does not appear if the fuse mount points are excluded in the rsync command. Since it is an automated sync, the mount command can be used to obtain all fuse mount points. The output of the mount command may differ depending on the system, but in a debian jessie sshfs mounts appear as USER#HOST:MOUNTED_DIR on /path/to/mount/point type fuse.sshfs (rw,...). A simple way to automate the exclusion of fuse mounts in bash+sed is the following
SOURCE="/home/"
FUSEEXCLUDE=( $( mount |
sed -rn "
/ type fuse/ {
s|^[^ ]+ on ([^ ]+) type fuse.+|\1|;
/^${SOURCE//\//\\\/}.+/ {
s|^${SOURCE//\//\\\/}| --exclude |;
p;
}
}" ) )
rsync $OPTIONS "${FUSEEXCLUDE[#]}" "$SOURCE" "$TARGET"
I'm running zsh on a Raspberry Pi 2 (Raspbian Jessie). zsh compinit is complaining about the /tmp directory being insecure. So, I checked the permissions on the directory:
$ compaudit
There are insecure directories:
/tmp
$ ls -ld /tmp
drwxrwxrwt 13 root root 16384 Apr 10 11:17 /tmp
Apparently anyone can do anything in the /tmp directory. Which makes sense, given it's purpose. So I tried the suggestions on this stackoverflow question. I also tried similar suggestions on other sites. Specifiacally, it suggests turning off group write permissions on that directory. Because of how the permissions looked according to ls -ld, I had to turn off the 'all' write permissions as well. So:
$ sudo su
% chmod g-w /tmp
% chmod a-w /tmp
% exit
$ compaudit
# nothing shows up, zsh is happy
This shut zsh up. However, other programs started to break. For example, gnome-terminal would crash whenever I typed the letter 'l'. Because of this, I had to turn the write permissions back on, and just run compinit -u in my .zshrc.
What I want to know: is there any better way to fix this? I'm not sure that it's a great idea to let compinit use an insecure directory. My dotfiles repo is hosted here, and the file where I now run compinit -u is here.
First, the original permissions on /tmp were correct. Make sure you've restored them correctly: ls -ld /tmp must start with drwxrwxrwt. You can use sudo chmod 1777 /tmp to set the correct permissions. /tmp is supposed to be writable by everyone, and any other permissions is highly likely to break stuff.
compaudit complains about directories in fpath, so one of the directories in your fpath is of the form /tmp/… (not necessarily /tmp itself). Check how fpath is being set. Normally the directories in fpath should be only subdirectories of the zsh installation directory, and places in your home directory. A subdirectory of /tmp wouldn't get in there without something unusual on your part.
If you can't find out where the stray directory is added to fpath, run zsh -x 2>zsh-x.log, and look for fpath in the trace file zsh-x.log.
It can be safe to use a directory under /tmp, but only if you created it securely. The permissions on /tmp allow anybody to create files, but users can only remove or rename their own files (that's what the t at the end of the permissions means). So if a directory is created safely (e.g. with mktemp -d), it's safe to use it in fpath. compaudit isn't sophisticated enough to recognize this case, and in any case it wouldn't have enough information since whether the directory is safe depends on how it was created.
Produces the following command on Ubuntu 64bit on VMWare:
mount /dev/sda1 /newroot
cd /newroot
mkdir old-root
pivot_root . old-root
I get an error that I do not understand
pivot_root: device or resource busy
Any ideas?
I saw the same error when the new root directory is a plain directory. When the new root is a mount, it will be ok. A bind mount of a directory is ok too. Also need to make sure the root directory permission is 0755, and owned by the root user.
The related answer states that you need to umount /proc first. I do not see the same.
The host ubnutu is 16.04 and it pivots into 18.04. Used unshare -m -p -f /bin/bash, followed by pivot_root . old_root. The -f is necessary to avoid a memory allocation error.
My app uses log4j and writes the logs to directory A which is in root directory. I want to move the logs out to a mounted drive without making any change in the application.
Can I use soft symlink to do this? I have created a symlink like this -
ln -s A mounted_drive_directory
But I still see logs written to directory A.
ln [OPTION]... [-T] TARGET LINK_NAME, so your arguments order is wrong. You'll have to delete (or move) A first before creating the link, or filename conflict will occur.
You could also use mountpoint bindings for that, e.g. mount --rbind /mounted/drive/directory /full/path/to/A, but it have to be done on each system boot (or saved in /etc/fstab to be auto-executed on boot).
ln works a little bit different:
first argument is real folder\file, second - symlink.
mv /root/A /root/B;
ln -s mounted_drive_directory /root/A;