I was running a simulation on a terminal and the simulation did not go through due to disk space issue. (It reported "No space left on device")
Then we cleaned up some space and ran simulation on the same terminal.
However, it still complained the space issue.
When we ran on a new terminal, the simulation went through.
Hence I want to understand the cause of this.
Please help
Thank you.
This is a common issue in linux.
If a process has opened a file and not closed it, removing the file only removes the directory entry (think name) from the directory it is in. Until the file is closed by the process or the process terminates the disk space will not be reclaimed.
To find these files you can look through the /proc file system. Every running process can be found in there by it process id (pid).
Here I'm running a python program that opened a file and is doing nothing. If I use ps to find the pid of the process and cd int /proc/<pid>/fd I can see the open file descriptors and the names of the files that are open:
$ pwd
/proc/38246/fd
$ ls -l
total 0
lrwx------ 1 x x 64 Sep 8 15:39 0 -> /dev/pts/0
lrwx------ 1 x x 64 Sep 8 15:39 1 -> /dev/pts/0
lrwx------ 1 x x 64 Sep 8 15:39 2 -> /dev/pts/0
lr-x------ 1 x x 64 Sep 8 15:39 3 -> /tmp/test
If I remove the file /tmp/test I see this:
$ rm /tmp/test
$ ls -l
total 0
lrwx------ 1 x x 64 Sep 8 15:39 0 -> /dev/pts/0
lrwx------ 1 x x 64 Sep 8 15:39 1 -> /dev/pts/0
lrwx------ 1 x x 64 Sep 8 15:39 2 -> /dev/pts/0
lr-x------ 1 x x 64 Sep 8 15:39 3 -> /tmp/test (deleted)
Search through /proc/*/fd/ for files that say deleted.
Related
Below files are present in current directory.
-rw-r--r-- 1 kazama kazama 0 Feb 16 08:50 london_july_2001_001.jpeg
-rw-r--r-- 1 kazama kazama 0 Feb 16 08:50 london_march_2004_002.png
-rw-r--r-- 1 kazama kazama 0 Feb 16 08:50 newyork_dec_2012_005.jpeg
-rw-r--r-- 1 kazama kazama 0 Feb 16 08:50 paris_sept_2007_003.jpg
I want to filter all images except which starts with "paris" text.
I have tried below command which works as per my expectation.
ls -l !(paris*)
But, I do not understand why below 2 commands do not give me expected output. In both of the cases it shows all the 4 files.
ls -l !(paris)*.*(jpeg|jpg|png)
ls -l !(paris)*
How bash interprets this extended globbing syntax internally ?Is it first trying to match !(paris) then it tries to match (jpeg|jpg|png) ?
!(paris) matches anything but paris, which includes paris_, pari, par, pa, p, and even the empty string. Bash will find something that doesn't match paris and try to match the rest of the pathname against the rest of the pattern. See:
$ echo !(paris)london*
london_july_2001_001.jpeg london_march_2004_002.png
$ echo !(paris)_*
london_july_2001_001.jpeg london_march_2004_002.png newyork_dec_2012_005.jpeg paris_sept_2007_003.jpg
$ echo !(paris)_*_*_*
london_july_2001_001.jpeg london_march_2004_002.png newyork_dec_2012_005.jpeg
I have a bunch of MP4 files that look like this:
-rw-rw-r-- 1 116M Apr 19 06:08 lULIqx9Akn4.mp4
These are youtube videos. When I try to do anything with all of them, I get a weird error. Every command I try says that I'm using invalid options (that I am not using). Here are some examples.
$ ls *.mp4
/bin/ls: invalid option -- '7'
Try '/bin/ls --help' for more information.
$ mv *.mp4 videos/
mv: invalid option -- 'L'
Try 'mv --help' for more information.
$ cp *.mp4 videos/.
cp: invalid option -- '7'
Try 'cp --help' for more information.
It doesn't do the same thing with a different extension (*.mp3, *.txt, *.sh).
What's going on? How do I fix this?
I used this as a cheap workaround,
find . -name "*.mp4" -exec mv {} videos/. \;
but I want to understand what's happening, not just get the job done.
One of your filenames starts with a hyphen, e.g,. -7 or -L. Try ls -- *.mp4 or cp -- *.mp4 videos. Also, allow me to suggest UNIX and Linux Stack Exchange for shell questions :) .
Solution:
Either move the files,
mv -- *.mp4 ./videos
or rename the files in situ...
for file in -*.mp4; do mv -- "$file" "${file:1}"; done
Explanation:
My sense is you have a file with a leading - in the directly... most commands stop you creating such files but if you copy them from another operating system it can occur. Thus, you need to rename any files with a leading - in their filename...
Let me explain with an example...
Let's try to create a file with a leading -:
touch "-7ULIqx9Akn4.mp4"
touch: illegal option -- 7
we can get around this as follows:
>touch -- "-7ULIqx9Akn4.mp4"
> ls -al -- -*.mp4
total 0
-rw-r--r--# 1 n staff 0 Apr 29 13:02 -7ULIqx9Akn4.mp4
ok, now lets set up an example and demonstrate a solution...
> ls -la
total 0
-rw-r--r--# 1 n staff 0 Apr 29 12:49 -75438752.mp4
-rw-r--r--# 1 n staff 0 Apr 29 12:49 -85438750.mp4
drwxr-xr-x# 7 n staff 238 Apr 29 12:49 .
drwxr-xr-x# 6 n staff 204 Apr 29 11:18 ..
-rw-r--r--# 1 n staff 0 Apr 29 12:36 75438750.mp4
-rw-r--r--# 1 n staff 0 Apr 29 12:33 7ULIqx9Akn4.mp4
-rw-rw-r--# 1 n staff 0 Apr 29 11:19 lULIqx9Akn4.mp4
next:
ls -- -*.mp4
-75438752.mp4 -85438750.mp4
ok, lets now rename these files...
A little explanation here, the following command uses mv to remove the leading character. i.e. Find files with a leading - and remove the leading character {$file:1} from the filename...
for file in -*.mp4; do mv -- "$file" "${file:1}"; done
Result:
> for file in -*.mp4; do mv -- "$file" "${file:1}"; done
> ll
total 0
drwxr-xr-x# 7 n 238 Apr 29 12:52 ./
drwxr-xr-x# 6 n 204 Apr 29 11:18 ../
-rw-r--r--# 1 n 0 Apr 29 12:36 75438750.mp4
-rw-r--r--# 1 n 0 Apr 29 12:49 75438752.mp4
-rw-r--r--# 1 n 0 Apr 29 12:33 7ULIqx9Akn4.mp4
-rw-r--r--# 1 n 0 Apr 29 12:49 85438750.mp4
-rw-rw-r--# 1 n 0 Apr 29 11:19 lULIqx9Akn4.mp4
Note
The above does not account for duplicate file names...
I want to find out IP of ssh connection using kernel code. However, I'm unable to find any suitable reference on Internet.
I tried using usermodehelper API but it seems it doesn't provide the output back in kernel.
Any help is much appreciated!
Let's assume 3298 is the pid of the process you are interested in. In userpsace, I can call
ls -l /proc/3298/fd
It will list something similar like
lrwx------ 1 name name 64 Apr 6 11:27 0 -> /dev/pts/2
lrwx------ 1 name name 64 Apr 6 11:27 1 -> /dev/pts/2
lrwx------ 1 name name 64 Apr 6 11:27 2 -> /dev/pts/2
lrwx------ 1 name name 64 Apr 6 11:27 3 -> socket:[50256]
lrwx------ 1 name name 64 Apr 6 11:37 4 -> /dev/pts/2
lrwx------ 1 name name 64 Apr 6 11:37 5 -> /dev/pts/2
lrwx------ 1 name name 64 Apr 6 11:37 6 -> /dev/pts/2
We now know that fd 3 (and only fd 3) is the file descriptor you are looking for. Your information should be in fd 3. You should be able to do the same thing in kernelspace by looking at the kernel sources how /proc/pid/fd is implemented.
Mar 21 15:59:56 loong kernel: [ 364.376809] communicator[5399]: segfault at 273c6 ip 00000000000273c6 sp 00007fc0c8a15818 error 14 in communicator (deleted)[400000+29000]
When you delete a file currently in use, it gets marked deleted and is no longer listed but can still be used until the process currently having it open closes it.
$ cat > foo &
[1] 13566
$ rm foo
[1]+ Stopped cat > foo
$ ls foo
ls: cannot access foo: No such file or directory
$ ls -l /proc/13566/fd
total 0
lrwx------ 1 pterjan pterjan 64 Mar 22 21:17 0 -> /dev/pts/4
l-wx------ 1 pterjan pterjan 64 Mar 22 21:17 1 -> /tmp/foo (deleted)
lrwx------ 1 pterjan pterjan 64 Mar 22 21:17 2 -> /dev/pts/4
So, the communicator executable had been deleted before it crashed (maybe it was replaced by a new one during an update, or it was just deleted).
So, I need to write a shell script that accepts a file path as an argument and uses it within the shell script. I know that I could do something like:
$ ./script filename
With this solution, this would make it so that I could access this filename string using $1 in the shell script.However, I'm wondering if it is possible to use input redirection instead so I could pass the filename using it:
$ ./script < filename
If so, how would I access this filename in the shell script? I'm fairly new to shell script coding so I'm not sure if this is even possible.
You can extract the name of file passed with < by listing /proc/<pid>/fd directory, as follows:
ls -ltr /proc/$$/fd
Then
$ cat myscript.sh
ls -ltr /proc/$$/fd
$ ./myscript.sh < hello
total 0
lr-x------ 1 u g 64 Feb 25 08:42 255 -> /tmp/myscript.sh
lrwx------ 1 u g 64 Feb 25 08:42 2 -> /dev/pts/0
lrwx------ 1 u g 64 Feb 25 08:42 1 -> /dev/pts/0
lr-x------ 1 u g 64 Feb 25 08:42 0 -> /tmp/hello
I can't guess whether this is useful
And doesn't work when input is passed through a pipe
$ cat hello| ./myscript.sh
total 0
lr-x------ 1 u p 64 Feb 25 08:50 255 -> /tmp/myscript.sh
lrwx------ 1 u p 64 Feb 25 08:50 2 -> /dev/pts/0
lrwx------ 1 u p 64 Feb 25 08:50 1 -> /dev/pts/0
lr-x------ 1 u p 64 Feb 25 08:50 0 -> pipe:[82796]
Alternatively, you can use lsof and a little line handling to extract value
filename=$(/usr/sbin/lsof -p $$| grep " 0u"| cut -c 60-)
You cannot access the filename in the case of ./script < filename but only its contents. The reason is that the shell opens filename and set the corresponding file descriptor to be the standard input of the ./script.
With your second solution you don't access the filename, but your script's standard input is coming from the file.
With the first solution printing the file would look like this:
cat "$1"
With the second:
while read line; do echo "$line"; done