Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 1 year ago.
Improve this question
there are a lot of similar problem like "Device or resource busy". But I think my problem is different with them.
I use mount --bind to bind a directory
mount --bind /tmp/origin /tmp/mount
and then could umount successfully
umount /tmp/mount
And then if I call rm at once
rm -rf /tmp/mount
I could get a error Device or resource busy. If I wait 2~3 seconds, and then call rm, it could success.
So this behaviour is very strange here. I try use
lsof +D /tmp/mount
could not see anything.
I also use fuser -vm /tmp/mount, could not see any process hold this folder.
I compare the /proc/mounts before umount /tmp/mount and after umount /tmp/mount. /tmp/mount has already removed.
I compare the stat /proc/mounts before umount /tmp/mount and after umount /tmp/mount. The inode also different, this means /tmp/mount has already removed complete.
Even I call sync && echo 2 > /proc/sys/vm/drop_caches and try to drop file caches, it still not work.
I try this in both Ubuntu 14.04 and CentOS 6.6. They have same results.
I faced problem like this because I mount shared folder in VM and I want to remove directory after unmount and I just want to share my solution.
un-mount path
sudo umount /your_path
remove mout path in /etc/fstab
sudo nano /etc/fstab
reboot
sudo reboot
remove directory
sudo rm -rf /your_path
In my experience, the following operations are asynchronous on Linux:
Closing file. Immediately after close() returns, umount() may return EBUSY while it performs asynchronous release. See discussion here: page 1, page 2.
Umounting filesystem. Mounted device may be busy until all data is written to disk.
Even I call sync && echo 2 > /proc/sys/vm/drop_caches and try to drop file caches, it still not work.
See sync(8):
On Linux, sync is only guaranteed to schedule the dirty blocks for writing; it can actually take a short time before all the blocks are finally written. The reboot(8) and halt(8) commands take this into account by sleeping for a few seconds after calling sync(2).
As for /proc/sys/vm/drop_caches, see here:
This is a non-destructive operation and will not free any dirty objects.
Thus, immediately after your command, data may be still queued for write and umounting is not yet finished.
Update
However, when asynchronous umounting is in action, kernel will return EBUSY for operations on mounted device, but not for mount point.
So the cases above could not be the reason of your problem :P
PS.
Actually I don't understand why the man page states that sync(8) is not synchronous in Linux. It calls sync(2) which states:
According to the standard specification (e.g., POSIX.1-2001), sync() schedules the writes, but may return before the actual writing is done. However, since version 1.3.20 Linux does actually wait. (This still does not guarantee data integrity: modern disks have large caches.)
Thank you for #g-v answer. But I found the result is another problem. We use CLONE_NEWNS flag when fork a process. More details could be found in CLONE_NEWNS flag and MESOS-3349 Device busy bug
In a short word, we mount in parent process. And then umount in child process, because of CLONE_NEWNS, the mount point still exists which handled by parent process. So when call rmdir would got EBUSY error code.
To avoid above problems, we could use shared mount or slave mount. More details could be found in LWN 159092
check
df -h
then
sudo umount /path
Follow these steps:
open Resource Monitor
click on "associated handles" drop box
search the folder creating problem
Right click on each process and "End process".
Now you can delete the folder
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 5 years ago.
Improve this question
So I am trying to chroot all the users who are in group www to the directory /var/www. But I every time I try to do that, it comes back saying the group doesn't exist. (even though the group does exist)
[root#server var]# cat /etc/fedora-release
Fedora release 26 (Twenty Six)
[root#server var]# groupadd -r www
[root#server var]# groupmod -R /var/www www
groupmod: group 'www' does not exist
[root#server var]# ls -la
drwxrwxrwx. 5 root www 46 Jul 12 06:44 www
As you can see the error message is less than helpful. I have looked around on stackoverflow but haven't come across an answer to this specific question yet.
Can anyone shed some light on what I am doing wrong?
That is not what groupmod -R does. What it means is that the groupmod program will chroot into the directory, and then do everything. It’s intended for when you have one system mounted inside another, such as if you booted from a live USB drive to make changes to a broken system.
Once groupmod has run chroot, it looks in the /var/www/etc/group file to figure out what group ID www corresponds to, which of course fails because if your system is at all sanely set up you don’t have a var/www/etc/group file.
I do not know how to make sure all processes by a specific user run in a chroot, and I don’t think that’s the right way to achieve your goal. If a program is chrooted into /var/www, it doesn’t have access to any of the utilities it might expect, like the web server executable. Instead, I would look at the documentation of your web server and see if it supports this directly, or see if you can get a custom mount namespace using systemd.
I've a mounted iso image in the path:
/mnt/iso
Inside this iso I've an install script install.sh
I run the installation script from the iso and at the end the script ask to the user if he want to umount the iso itself.
If the user press "y" the script execute the following code:
cd /
umount /mnt/iso
echo "Installation completed!"
Unfortunately when the script tries to execute the umount there's an error
umount: /mnt/iso: device is busy
I suppose it's due to the fact that the virtual device is busy from the script itself.
How can make it work?
Tnx
Use the -l or --lazy switch to umount which will do a lazy umount, where it is only fully unmounted once it is no longer in use. The full description in the manual page (this is a linux specific option) is:
Lazy unmount. Detach the filesystem from the filesystem hierarchy
now, and cleanup all references to the filesystem as soon as it is not
busy anymore. (Requires kernel 2.4.11 or later.)
TomH's solution will resolve the issue if you are using the latest. Otherwise the comment by Simone Palazzo is your best bet. You are unmounting something through a script located in the area you are unmounting. If you run the script from the root directory it will be successful.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed last year.
The community reviewed whether to reopen this question last year and left it closed:
Original close reason(s) were not resolved
Improve this question
I've got some samba drives that are being accessed by multiple users daily. I already have code to recognize shared drives (from a SQL table) and mount them in a special directory where all users can access them.
I want to know, if I remove a drive from my SQL table (effectively taking it offline) how, or even is, there a way to unmount a busy device? So far I've found that any form of umount does not work.
Ignoring the possibility of destroying data - is it possible to unmount a device that is currently being read?
YES!! There is a way to detach a busy device immediately - even if it is busy and cannot be unmounted forcefully. You may cleanup all later:
umount -l /PATH/OF/BUSY-DEVICE
umount -f /PATH/OF/BUSY-NFS (NETWORK-FILE-SYSTEM)
NOTE/CAUTION
These commands can disrupt a running process, cause data loss OR corrupt open files. Programs accessing target DEVICE/NFS files may throw errors OR could not work properly after force unmount.
Do not execute above umount commands when inside mounted path (Folder/Drive/Device) itself. First, you may use pwd command to validate your current directory path (which should not be the mounted path), then use cd command to get out of the mounted path - to unmount it later using above commands.
If possible, let us locate/identify the busy process, kill that process and then unmount the samba share/ drive to minimize damage:
lsof | grep '<mountpoint of /dev/sda1>' (or whatever the mounted device is)
pkill target_process (kills busy proc. by name | kill PID | killall target_process)
umount /dev/sda1 (or whatever the mounted device is)
Make sure that you aren't still in the mounted device when you are trying to umount.
Avoid umount -l
At the time of writing, the top-voted answer recommends using umount -l.
umount -l is dangerous or at best unsafe. In summary:
It doesn't actually unmount the device, it just removes the filesystem from the namespace. Writes to open files can continue.
It can cause btrfs filesystem corruption
Work around / alternative
The useful behaviour of umount -l is hiding the filesystem from access by absolute pathnames, thereby minimising further moutpoint usage.
This same behaviour can be achieved by mounting an empty directory with permissions 000 over the directory to be unmounted.
Then any new accesses to filenames in the below the mountpoint will hit the newly overlaid directory with zero permissions - new blockers to the unmount are thereby prevented.
First try to remount,ro
The major unmount achievement to be unlocked is the read-only remount. When you gain the remount,ro badge, you know that:
All pending data has been written to disk
All future write attempts will fail
The data is in a consistent state, should you need to physcially disconnect the device.
mount -o remount,ro /dev/device is guaranteed to fail if there are files open for writing, so try that straight up. You may be feeling lucky, punk!
If you are unlucky, focus only on processes with files open for writing:
lsof +f -- /dev/<devicename> | awk 'NR==1 || $4~/[0-9]+[uw -]/'
You should then be able to remount the device read-only and ensure a consistent state.
If you can't remount read-only at this point, investigate some of the other possible causes listed here.
Read-only re-mount achievement unlocked 🔓☑
Congratulations, your data on the mountpoint is now consistent and protected from future writing.
Why fuser is inferior to lsof
Why not use use fuser earlier? Well, you could have, but fuser operates upon a directory, not a device, so if you wanted to remove the mountpoint from the file name space and still use fuser, you'd need to:
Temporarily duplicate the mountpoint with mount -o bind /media/hdd /mnt to another location
Hide the original mount point and block the namespace:
Here's how:
null_dir=$(sudo mktemp --directory --tmpdir empty.XXXXX")
sudo chmod 000 "$null_dir"
# A request to remount,ro will fail on a `-o bind,ro` duplicate if there are
# still files open for writing on the original as each mounted instance is
# checked. https://unix.stackexchange.com/a/386570/143394
# So, avoid remount, and bind mount instead:
sudo mount -o bind,ro "$original" "$original_duplicate"
# Don't propagate/mirror the empty directory just about hide the original
sudo mount --make-private "$original_duplicate"
# Hide the original mountpoint
sudo mount -o bind,ro "$null_dir" "$original"
You'd then have:
The original namespace hidden (no more files could be opened, the problem can't get worse)
A duplicate bind mounted directory (as opposed to a device) on which
to run fuser.
This is more convoluted[1], but allows you to use:
fuser -vmMkiw <mountpoint>
which will interactively ask to kill the processes with files open for writing. Of course, you could do this without hiding the mount point at all, but the above mimicks umount -l, without any of the dangers.
The -w switch restricts to writing processes, and the -i is interactive, so after a read-only remount, if you're it a hurry you could then use:
fuser -vmMk <mountpoint>
to kill all remaining processes with files open under the mountpoint.
Hopefully at this point, you can unmount the device. (You'll need to run umount on the mountpoint twice if you've bind mounted a mode 000 directory on top.)
Or use:
fuser -vmMki <mountpoint>
to interactively kill the remaining read-only processes blocking the unmount.
Dammit, I still get target is busy!
Open files aren't the only unmount blocker. See here and here for other causes and their remedies.
Even if you've got some lurking gremlin which is preventing you from fully unmounting the device, you have at least got your filesystem in a consistent state.
You can then use lsof +f -- /dev/device to list all processes with open files on the device containing the filesystem, and then kill them.
[1] It is less convoluted to use mount --move, but that requires mount --make-private /parent-mount-point which has implications. Basically, if the mountpoint is mounted under the / filesystem, you'd want to avoid this.
Try the following, but before running it note that the -k flag will kill any running processes keeping the device busy.
The -i flag makes fuser ask before killing.
fuser -kim /address # kill any processes accessing file
unmount /address
Before unmounted the filesysem. we need to check is any process holding or using the filesystem. That's why it show device is busy or filesystem is in use.
run below command to find out the processes using by a filesystem:
fuser -cu /local/mnt/
It will show how many processes holding/using the filesystem.
local/mnt: 1725e(root) 5645c(shasankarora)
ps -ef | grep 1725 <--> ps -ef | grep <pid>
kill -9 pid
Kill all the processes and then you will able to unmount the partition/busy device.
Check for exported NFS file systems with exportfs -v. If found, remove with exportfs -d share:/directory. These don't show up in the fuser/lsof listing, and can prevent umount from succeeding.
Just in case someone has the same pb. :
I couldn't unmount the mount point (here /mnt) of a chroot jail.
Here are the commands I typed to investigate :
$ umount /mnt
umount: /mnt: target is busy.
$ df -h | grep /mnt
/dev/mapper/VGTout-rootFS 4.8G 976M 3.6G 22% /mnt
$ fuser -vm /mnt/
USER PID ACCESS COMMAND
/mnt: root kernel mount /mnt
$ lsof +f -- /dev/mapper/VGTout-rootFS
$
As you can notice, even lsof returns nothing.
Then I had the idea to type this :
$ df -ah | grep /mnt
/dev/mapper/VGTout-rootFS 4.8G 976M 3.6G 22% /mnt
dev 2.9G 0 2.9G 0% /mnt/dev
$ umount /mnt/dev
$ umount /mnt
$ df -ah | grep /mnt
$
Here it was a /mnt/dev bind to /dev that I had created to be able to repair my system inside from the chroot jail.
After umounting it, my pb. is now solved.
Check out umount2:
Linux 2.1.116 added the umount2() system call, which, like umount(),
unmounts a target, but allows additional flags controlling the
behaviour of the operation:
MNT_FORCE (since Linux 2.1.116) Force unmount even if busy. (Only for
NFS mounts.)
MNT_DETACH (since Linux 2.4.11) Perform a lazy unmount:
make the mount point unavailable for new accesses, and actually
perform the unmount when the mount point ceases to be busy.
MNT_EXPIRE (since Linux 2.6.8) Mark the mount point as expired. If a mount point
is not currently in use, then an initial call to umount2() with this
flag fails with the error EAGAIN, but marks the mount point as
expired. The mount point remains expired as long as it isn't accessed
by any process. A second umount2() call specifying MNT_EXPIRE unmounts
an expired mount point. This flag cannot be specified with either
MNT_FORCE or MNT_DETACH.
I recently had a similar need to unmount in order to change it's label with gparted.
/dev/sda1 was being mounted via /etc/fstab as /media/myusername. When attempts to unmount failed, I researched the error. I had forgotten to unmount a dual partitioned thumb drive with a mountpoint on /dev/hda1 first.
I gave 'lsof' a go as recommended.
$ sudo lsof | grep /dev/sda1
The output of which was:
lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/1000/gvfs
Output information may be incomplete.
lsof: WARNING: can't stat() fuse file system /run/user/1000/doc
Output information may be incomplete.
Since lsof burped up two fuse warnings, I poked around in /run/user/1000/*, and took a guess that it could be open files or mount points (or both) interfering with things.
Since the mount points live in /media/, I tried again with:
$ sudo lsof | grep /media
The same two warnings, but this time it returned additional info:
bash 4350 myusername cwd DIR 8,21 4096 1048577 /media
sudo 36302 root cwd DIR 8,21 4096 1048577 /media
grep 36303 myusername cwd DIR 8,21 4096 1048577 /media
lsof 36304 root cwd DIR 8,21 4096 1048577 /media
lsof 36305 root cwd DIR 8,21 4096 1048577 /media
Still scratching my head, it was at this point I remembered the thumb drive sticking out of the USB port. Maybe the scratching helped.
So I unmounted the thumb drive partitions (unmounting one automatically unmounted the other) and safefly unplugged the thumb drive. After doing so, I was able to unmount /dev/sda1 (having nothing mounted on it anymore), relabel it with gparted, remount both the drive and thumb drive with no issues whatsoever.
Bacon saved.
Someone has mentioned that if you are using terminal and your current directory is inside the path which you want to unmount, you will get the error.
As a complementary, in this case, your lsof | grep path-to-be-unmounted must have below output:
bash ... path-to-be-unmounted
sudo fusermount -u -z <mounted path>
NB: do not use completition for the path as this will also freeze the terminal.
Another alternative when anything works is editing /etc/fstab, adding noauto flag and rebooting the machine. The device won't be mounted, and when you're finished doing whatever, remove flag and reboot again.
Niche Answer:
If you have a zfs pool on that device, at least when it's a file-based pool, lsof will not show the usage. But you can simply run
sudo zpool export mypool
and then unmount.
Multiple mounts inside a folder
An additional reason could be a secondary mount inside your primary mount folder, e.g. after you worked on an SD card for an embedded device:
# mount /dev/sdb2 /mnt # root partition which contains /boot
# mount /dev/sdb1 /mnt/boot # boot partition
Unmounting /mnt will fail:
# umount /mnt
umount: /mnt: target is busy.
First we have to unmount the boot folder and then the root:
# umount /mnt/boot
# umount /mnt
In my case, I couldn't unmount a partition that was mounted to a directory that was an AFP share. (sharing into an Apple bonjour/avahi mdns world)
I moved all the logins on the server to their home directory; I moved all the remotely connected Macs to some other directory.
I still couldn't unmount the partition even with umount -f
So I restarted the netatalk daemon on the server.
(/etc/netatalk/afp.conf has in it the share assignment)
After the netatalk restart, umount succeeded without the -f.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 4 years ago.
Improve this question
Linux: Which process is causing "device busy" when doing umount?
Look at the lsof command (list open files) -- it can tell you which processes are holding what open. Sometimes it's tricky but often something as simple as sudo lsof | grep (your device name here) could do it for you.
Just in case... sometimes happens that you are calling umount from the terminal, and your current directory belongs to the mounted filesystem.
You should use the fuser command.
Eg. fuser /dev/cdrom will return the pid(s) of the process using /dev/cdrom.
If you are trying to unmount, you can kill theses process using the -k switch (see man fuser).
Check for open loop devices mapped to a file on the filesystem with "losetup -a". They wont show up with either lsof or fuser.
Also check /etc/exports. If you are exporting paths within the mountpoint via NFS, it will give this error when trying to unmount and nothing will show up in fuser or lsof.
lsof +f -- /mountpoint
(as lists the processes using files on the mount mounted at /mountpoint. Particularly useful for finding which process(es) are using a mounted USB stick or CD/DVD.
lsof and fuser are indeed two ways to find the process that keeps a certain file open.
If you just want umount to succeed, you should investigate its -f and -l options.
That's exactly why the "fuser -m /mount/point" exists.
BTW, I don't think "fuser" or "lsof" will indicate when a resource is held by kernel module, although I don't usually have that issue..
Open files
Processes with open files are the usual culprits. Display them:
lsof +f -- <mountpoint or device>
There is an advantage to using /dev/<device> rather than /mountpoint: a mountpoint will disappear after an umount -l, or it may be hidden by an overlaid mount.
fuser can also be used, but to my mind lsof has a more useful output. However fuser is useful when it comes to killing the processes causing your dramas so you can get on with your life.
List files on <mountpoint> (see caveat above):
fuser -vmM <mountpoint>
Interactively kill only processes with files open for writing:
fuser -vmMkiw <mountpoint>
After remounting read-only (mount -o remount,ro <mountpoint>), it is safe(r) to kill all remaining processes:
fuser -vmMk <mountpoint>
Mountpoints
The culprit can be the kernel itself. Another filesystem mounted on the filesystem you are trying to umount will cause grief. Check with:
mount | grep <mountpoint>/
For loopback mounts, also check the output of:
losetup -la
Anonymous inodes (Linux)
Anonymous inodes can be created by:
Temporary files (open with O_TMPFILE)
inotify watches
[eventfd]
[eventpoll]
[timerfd]
These are the most elusive type of pokemon, and appear in lsof's TYPE column as a_inode (which is undocumented in the lsof man page).
They won't appear in lsof +f -- /dev/<device>, so you'll need to:
lsof | grep a_inode
For killing processes holding anonymous inodes, see: List current inotify watches (pathname, PID).
lsof and fuser didn't give me anything either.
After a process of renaming all possible directories to .old and rebooting the system every time after I made changes I found one particular directory (relating to postfix) that was responsible.
It turned out that I had once made a symlink from /var/spool/postfix to /disk2/pers/mail/postfix/varspool in order to minimise disk writes on an SDCARD-based root filesystem (Sheeva Plug).
With this symlink, even after stopping the postfix and dovecot services (both ps aux as well as netstat -tuanp didn't show anything related) I was not able to unmount /disk2/pers.
When I removed the symlink and updated the postfix and dovecot config files to point directly to the new dirs on /disk2/pers/ I was able to successfully stop the services and unmount the directory.
Next time I will look more closely at the output of:
ls -lR /var | grep ^l | grep disk2
The above command will recursively list all symbolic links in a directory tree (here starting at /var) and filter out those names that point to a specific target mount point (here disk2).
If you still can not unmount or remount your device after stopping all services and processes with open files, then there may be a swap file or swap partition keeping your device busy. This will not show up with fuser or lsof. Turn off swapping with:
sudo swapoff -a
You could check beforehand and show a summary of any swap partitions or swap files with:
swapon -s
or:
cat /proc/swaps
As an alternative to using the command sudo swapoff -a, you might also be able to disable the swap by stopping a service or systemd unit. For example:
sudo systemctl stop dphys-swapfile
or:
sudo systemctl stop var-swap.swap
In my case, turning off swap was necessary, in addition to stopping any services and processes with files open for writing, so that I could remount my root partition as read only in order to run fsck on my root partition without rebooting. This was necessary on a Raspberry Pi running Raspbian Jessie.
Filesystems mounted on the filesystem you're trying to unmount can cause the target is busy error in addition to any files that are in use. (For example when you mount -o bind /dev /mnt/yourmount/dev in order to use chroot there.)
To find which file systems are mounted on the filesystem run the following:
mount | grep '/mnt/yourmount'
To find which files are in use the advice already suggested by others here:
lsof | grep '/mnt/yourmount'
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 9 years ago.
Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
I have an NFS-mounted directory on a Linux machine that has hung. I've tried to force an unmount, but it doesn't seem to work:
$ umount -f /mnt/data
$ umount2: Device or resource busy
$ umount: /mnt/data: device is busy
If I type "mount", it appears that the directory is no longer mounted, but it hangs if I do "ls /mnt/data", and if I try to remove the mountpoint, I get:
$ rmdir /mnt/data
rmdir: /mnt/data: Device or resource busy
Is there anything I can do other than reboot the machine?
You might try a lazy unmount:
umount -l
If the NFS server disappeared and you can't get it back online, one trick that I use is to add an alias to the interface with the IP of the NFS server (in this example, 192.0.2.55).
Linux
The command for that is something roughly like:
ifconfig eth0:fakenfs 192.0.2.55 netmask 255.255.255.255
Where 192.0.2.55 is the IP of the NFS server that went away. You should then be able to ping the address, and you should also be able to unmount the filesystem (use unmount -f). You should then destroy the aliased interface so you no longer route traffic to the old NFS server to yourself with:
ifconfig eth0:fakenfs down
FreeBSD and similar operating systems
The command would be something like:
ifconfig em0 alias 192.0.2.55 netmask 255.255.255.255
And then to remove it:
ifconfig em0 delete 192.0.2.55
man ifconfig(8) for more!
Try running
lsof | grep /mnt/data
That should list any process that is accessing /mnt/data that would prevent it from being unmounted.
I had the same problem, and
neither umount /path -f,
neither umount.nfs /path -f,
neither fuser -km /path,
works
finally I found a simple solution >.<
sudo /etc/init.d/nfs-common restart, then lets do the simple umount ;-)
Your NFS server disappeared.
Ideally your best bet is if the NFS server comes back.
If not, the "umount -f" should have done the trick.
It doesn't ALWAYS work, but it often will.
If you happen to know what processes are USING the NFS filesystem,
you could try killing those processes and then maybe an unmount would work.
Finally, I'd guess you need to reboot.
Also, DON'T soft-mount your NFS drives. You use hard-mounts to guarantee
that they worked. That's necessary if you're doing writes.
Couldn't find a working answer here; but on linux you can run "umount.nfs4 /volume -f" and it definitely unmounts it.