How to delete files named like these? [closed] - linux

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 8 years ago.
Improve this question
I have two files in my home folder named "'?" and "'?;?" (without the double quotes). How can I delete them? I've tried to use escape, but it doesn't work.

Use single or double quotes to avoid wildcard expansion. A ? is a wildcard which indicates to the shell to match with any one single character. By placing it in quotes you are telling the shell not to perform wildcard expansion.
rm '?' '?;?'
rm "?" "?;?"
This will remove the two files named "?" and "?;?"
You can also use a backslash to quote the individual characters that have special meaning to the shell, so you could do this
rm \? \?\;\?
Notice you have to quote the '?' to prevent pathname expansion and you have to quote the ';' so the shell doesn't interpret that as separating commands.
If you leave out the quotes, then the shell parses it differently. Here's an experiment I ran.
$ for i in {1..4}; do for j in {a..c}; do touch "$i;$j" $j '?' '?;?';done;done
$ ls
1;a 1;b 1;c 2;a 2;b 2;c 3;a 3;b 3;c 4;a 4;b 4;c ? ?;? a b c
$ rm ? ?;?
rm: cannot remove `?': No such file or directory
rm: cannot remove `a': No such file or directory
rm: cannot remove `b': No such file or directory
rm: cannot remove `c': No such file or directory
bash: ?: command not found
$ rm `echo "?" "?;?"`
rm: cannot remove `?': No such file or directory
$
What happened here is the shell did pathname expansion, so
rm ? ?;?
became
rm ? a b c ? a b c;? a b c
The rm command removed files a b c ? then complained that the following files were not found (they had already been deleted). The semicolon separated commands, so it then tried to invoke the '?' command passing arguments "a" "b" "c" ... but there is no '?' command - the file named '?' had just been deleted, and it wasn't executable anyway - so the shell complains that the "?" command is not found.
If you want to remove all files matching "?" and "?;?" you need to trick the shell into expanding those, which I did like this
rm `echo "?" "?;?"`
This was expanded by the shell in two steps, first it runs echo "?" "?;?" which results in two strings, "?" and "?;?", then it does pathname expansion using those strings to produce the arguments for rm, which results in
rm ? 1;a 1;b 1;c 2;a 2;b 2;c 3;a 3;b 3;c 4;a 4;b 4;c ?;?
Notice that the wildcard expansion for '?' didn't produce any matching files this time (they had already been previously deleted), so the shell passes '?' as an argument to rm, which successfully removes all files passed as arguments except for '?' so it complains about that.
Here's another experiment
$ for i in {1..4}; do for j in {a..c}; do touch "$i;$j" $j '?' '?;?';done;done
$ ls
1;a 1;b 1;c 2;a 2;b 2;c 3;a 3;b 3;c 4;a 4;b 4;c ? ?;? a b c
$ rm "?" "?;?"
$ ls
1;a 1;b 1;c 2;a 2;b 2;c 3;a 3;b 3;c 4;a 4;b 4;c a b c
$ rm `echo "?" "?;?"`
$ ls
$
For more information consult the man page on globbing
man 7 glob
Wildcard Matching
A string is a wildcard pattern if it contains one of the characters '?', '*' or '['. Globbing is the operation that
expands a wildcard pattern into the list of pathnames matching the pattern. Matching is defined by:
A '?' (not between brackets) matches any single character.
A '*' (not between brackets) matches any string, including the empty string.

Note that ls can report a question mark for arbitrary non-printable characters, so there's a chance that what you've got as a file name does not contain a question mark.
You can spot this with the ls -b command, or with ls | cat.
As a convoluted example, complete with remedy, I created a script convolvulus like this:
set -x
mkdir convoluted &&
(
cd convoluted
cp /dev/null "$(ls -la | sed 1d)"
ls
ls -b
ls | cat
ls -la | cat
cp /dev/null $'\n'
cp /dev/null $'\n;\n'
ls -als | cat
ls -lab
ls
ls | cat
rm $'\n' $'\n;\n' d*
ls -a
)
rm -fr convoluted
When run, it yielded:
$ bash convolvulus 2>&1 | so
+ mkdir convoluted
+ cd convoluted
++ ls -la
++ sed 1d
+ cp /dev/null 'drwxr-xr-x 2 jleffler staff 68 Mar 9 11:58 .
drwxr-xr-x 240 jleffler staff 8160 Mar 9 11:58 ..'
+ ls
drwxr-xr-x 2 jleffler staff 68 Mar 9 11:58 .
drwxr-xr-x 240 jleffler staff 8160 Mar 9 11:58 ..
+ ls -b
drwxr-xr-x 2 jleffler staff 68 Mar 9 11:58 .\ndrwxr-xr-x 240 jleffler staff 8160 Mar 9 11:58 ..
+ ls
+ cat
drwxr-xr-x 2 jleffler staff 68 Mar 9 11:58 .
drwxr-xr-x 240 jleffler staff 8160 Mar 9 11:58 ..
+ ls -la
+ cat
total 0
drwxr-xr-x 3 jleffler staff 102 Mar 9 11:58 .
drwxr-xr-x 240 jleffler staff 8160 Mar 9 11:58 ..
-rw-r--r-- 1 jleffler staff 0 Mar 9 11:58 drwxr-xr-x 2 jleffler staff 68 Mar 9 11:58 .
drwxr-xr-x 240 jleffler staff 8160 Mar 9 11:58 ..
+ cp /dev/null '
'
+ cp /dev/null '
;
'
+ ls -als
+ cat
total 0
0 -rw-r--r-- 1 jleffler staff 0 Mar 9 11:58
0 -rw-r--r-- 1 jleffler staff 0 Mar 9 11:58
;
0 drwxr-xr-x 5 jleffler staff 170 Mar 9 11:58 .
0 drwxr-xr-x 240 jleffler staff 8160 Mar 9 11:58 ..
0 -rw-r--r-- 1 jleffler staff 0 Mar 9 11:58 drwxr-xr-x 2 jleffler staff 68 Mar 9 11:58 .
drwxr-xr-x 240 jleffler staff 8160 Mar 9 11:58 ..
+ ls -lab
total 0
-rw-r--r-- 1 jleffler staff 0 Mar 9 11:58 \n
-rw-r--r-- 1 jleffler staff 0 Mar 9 11:58 \n;\n
drwxr-xr-x 5 jleffler staff 170 Mar 9 11:58 .
drwxr-xr-x 240 jleffler staff 8160 Mar 9 11:58 ..
-rw-r--r-- 1 jleffler staff 0 Mar 9 11:58 drwxr-xr-x 2 jleffler staff 68 Mar 9 11:58 .\ndrwxr-xr-x 240 jleffler staff 8160 Mar 9 11:58 ..
+ ls
;
drwxr-xr-x 2 jleffler staff 68 Mar 9 11:58 .
drwxr-xr-x 240 jleffler staff 8160 Mar 9 11:58 ..
+ ls
+ cat
;
drwxr-xr-x 2 jleffler staff 68 Mar 9 11:58 .
drwxr-xr-x 240 jleffler staff 8160 Mar 9 11:58 ..
+ rm '
' '
;
' 'drwxr-xr-x 2 jleffler staff 68 Mar 9 11:58 .
drwxr-xr-x 240 jleffler staff 8160 Mar 9 11:58 ..'
+ ls -a
.
..
+ rm -fr convoluted
$
Have fun!
The -b option to ls works for GNU ls and for Mac OS X and BSD ls (but is not defined by POSIX).
The '$'\n' notation is Bash ANSI-C Quoting.

Use quotes around the file names:
$ ls
? ?;?
$ rm '?'
$ ls
?;?
$ rm "?;?"
$ ls
$

Related

How bash inverse pattern matching works internally in extglob?

Below files are present in current directory.
-rw-r--r-- 1 kazama kazama 0 Feb 16 08:50 london_july_2001_001.jpeg
-rw-r--r-- 1 kazama kazama 0 Feb 16 08:50 london_march_2004_002.png
-rw-r--r-- 1 kazama kazama 0 Feb 16 08:50 newyork_dec_2012_005.jpeg
-rw-r--r-- 1 kazama kazama 0 Feb 16 08:50 paris_sept_2007_003.jpg
I want to filter all images except which starts with "paris" text.
I have tried below command which works as per my expectation.
ls -l !(paris*)
But, I do not understand why below 2 commands do not give me expected output. In both of the cases it shows all the 4 files.
ls -l !(paris)*.*(jpeg|jpg|png)
ls -l !(paris)*
How bash interprets this extended globbing syntax internally ?Is it first trying to match !(paris) then it tries to match (jpeg|jpg|png) ?
!(paris) matches anything but paris, which includes paris_, pari, par, pa, p, and even the empty string. Bash will find something that doesn't match paris and try to match the rest of the pathname against the rest of the pattern. See:
$ echo !(paris)london*
london_july_2001_001.jpeg london_march_2004_002.png
$ echo !(paris)_*
london_july_2001_001.jpeg london_march_2004_002.png newyork_dec_2012_005.jpeg paris_sept_2007_003.jpg
$ echo !(paris)_*_*_*
london_july_2001_001.jpeg london_march_2004_002.png newyork_dec_2012_005.jpeg

Bash shell complains of invalid options I didn't use ... only on *.mp4 file extensions

I have a bunch of MP4 files that look like this:
-rw-rw-r-- 1 116M Apr 19 06:08 lULIqx9Akn4.mp4
These are youtube videos. When I try to do anything with all of them, I get a weird error. Every command I try says that I'm using invalid options (that I am not using). Here are some examples.
$ ls *.mp4
/bin/ls: invalid option -- '7'
Try '/bin/ls --help' for more information.
$ mv *.mp4 videos/
mv: invalid option -- 'L'
Try 'mv --help' for more information.
$ cp *.mp4 videos/.
cp: invalid option -- '7'
Try 'cp --help' for more information.
It doesn't do the same thing with a different extension (*.mp3, *.txt, *.sh).
What's going on? How do I fix this?
I used this as a cheap workaround,
find . -name "*.mp4" -exec mv {} videos/. \;
but I want to understand what's happening, not just get the job done.
One of your filenames starts with a hyphen, e.g,. -7 or -L. Try ls -- *.mp4 or cp -- *.mp4 videos. Also, allow me to suggest UNIX and Linux Stack Exchange for shell questions :) .
Solution:
Either move the files,
mv -- *.mp4 ./videos
or rename the files in situ...
for file in -*.mp4; do mv -- "$file" "${file:1}"; done
Explanation:
My sense is you have a file with a leading - in the directly... most commands stop you creating such files but if you copy them from another operating system it can occur. Thus, you need to rename any files with a leading - in their filename...
Let me explain with an example...
Let's try to create a file with a leading -:
touch "-7ULIqx9Akn4.mp4"
touch: illegal option -- 7
we can get around this as follows:
>touch -- "-7ULIqx9Akn4.mp4"
> ls -al -- -*.mp4
total 0
-rw-r--r--# 1 n staff 0 Apr 29 13:02 -7ULIqx9Akn4.mp4
ok, now lets set up an example and demonstrate a solution...
> ls -la
total 0
-rw-r--r--# 1 n staff 0 Apr 29 12:49 -75438752.mp4
-rw-r--r--# 1 n staff 0 Apr 29 12:49 -85438750.mp4
drwxr-xr-x# 7 n staff 238 Apr 29 12:49 .
drwxr-xr-x# 6 n staff 204 Apr 29 11:18 ..
-rw-r--r--# 1 n staff 0 Apr 29 12:36 75438750.mp4
-rw-r--r--# 1 n staff 0 Apr 29 12:33 7ULIqx9Akn4.mp4
-rw-rw-r--# 1 n staff 0 Apr 29 11:19 lULIqx9Akn4.mp4
next:
ls -- -*.mp4
-75438752.mp4 -85438750.mp4
ok, lets now rename these files...
A little explanation here, the following command uses mv to remove the leading character. i.e. Find files with a leading - and remove the leading character {$file:1} from the filename...
for file in -*.mp4; do mv -- "$file" "${file:1}"; done
Result:
> for file in -*.mp4; do mv -- "$file" "${file:1}"; done
> ll
total 0
drwxr-xr-x# 7 n 238 Apr 29 12:52 ./
drwxr-xr-x# 6 n 204 Apr 29 11:18 ../
-rw-r--r--# 1 n 0 Apr 29 12:36 75438750.mp4
-rw-r--r--# 1 n 0 Apr 29 12:49 75438752.mp4
-rw-r--r--# 1 n 0 Apr 29 12:33 7ULIqx9Akn4.mp4
-rw-r--r--# 1 n 0 Apr 29 12:49 85438750.mp4
-rw-rw-r--# 1 n 0 Apr 29 11:19 lULIqx9Akn4.mp4
Note
The above does not account for duplicate file names...

Bash - Filter out directories from ls -la output, but leave directory "."

I am trying to count size of all files and subdirectories starting from ./ using oneliner:
ls -laR | grep -v "\.\." | awk '{total += $5} END {print total}'
but this counts size of subdirectories twice because output of ls -laR | grep -v "\.\." is:
.:
total 32
drwxr-xr-x 3 root root 4096 Nov 29 22:59 .
-rw-r--r-- 1 root root 55 Nov 29 02:19 131
-rw-r--r-- 1 root root 50 Nov 29 01:28 abc
-rw-r--r-- 1 root root 1000 Nov 29 01:27 access.log
drwxr-xr-x 2 root root 4096 Nov 29 22:24 asd
-rwx------ 1 root root 458 Nov 29 02:54 oneliners.sh
-rwx------ 1 root root 2136 Nov 29 17:56 regexp.sh.skript
./asd:
total 32
drwxr-xr-x 2 root root 4096 Nov 29 22:24 .
-rw-r--r-- 1 root root 21298 Nov 29 22:26 asd
so it counts directory asd twice. once in listing of directory .: as:
drwxr-xr-x 2 root root 4096 Nov 29 22:24 asd
and 2nd time in listing of directory ./asd: as:
drwxr-xr-x 2 root root 4096 Nov 29 22:24 .
I expect, this will happen for every subdirectory. Is there a way to remove them once from ls output? Usint grep -v '^d' removes all directories, so they wont be counted at all. I know I can do it simply bu using du -sb, but I need it to be done with fancy oneliner.
ls -FlaR |grep -v '\s\.\{1,\}/$' |awk '{total += $5} END {print total}'
includes the size of folders inside '.', but not the size of '.' itself. Comparing with du, the answer is quite different -as du is about the space on disk (relates to blocks).
The answer I get using your awk script is closer to what the OS reports -if you subtract the directory sizes you get a match, which suggests that MacOS X uses a method similar to
ls -FlaR |grep -v '^d.*/$' |awk '{total += $5} END {print total}'
for calculating the size of the content of a folder.

How to tar the n most recent files

I am trying to create a script that foreach directoy in the folder folder, only the n most recent files are to be compressed.
However, I am having trouble with the multiple word files. I need a way to wrap them in quote marks so the tar command knows wich is each file.
Here is my script so far:
#!/bin/bash
if [ ! -d ~/backup ]; then
mkdir ~/backup
fi
cd ~/folder
for i in *; do
if [ -d "$i" ]; then
original=`pwd`
cd $i
echo tar zcf ~/backup/"$i".tar.gz "`ls -t | head -10`"
cd $original
fi
done
echo "Backup copied in $HOME/backup/"
exit 0
if [ ! -d ~/backup ]; then
mkdir ~/backup
fi
You can simplify by this :
[[ ! -d ~/backup ]] && mkdir ~/backup
Now to answer your question :
$ ls -t|head -10
file with spaces
file
test.txt
test
test.sh
$ lstFiles=""; while read; do lstFiles="$lstFiles \"$REPLY\""; done <<< "$(ls -t|head -10)"
$ echo $lstFiles
"file with spaces" "file" "test.txt" "test" "test.sh"
See how to read a command output or file content with a loop in Bash to read more details.
Several workarounds if you want to stick to one-liners - simplest is probably to use 'tr' and introduce wildcard for spaces:
echo tar zcf ~/backup/"$i".tar.gz "ls -t | head -10| tr ' ' '?'"
-rw-rw-r-- 1 dale dale 35 Apr 6 09:11 test 1_dummy.txt
-rw-rw-r-- 1 dale dale 35 Apr 6 09:11 test 2_dummy.txt
-rw-rw-r-- 1 dale dale 35 Apr 6 09:11 test 3_dummy.txt
-rw-rw-r-- 1 dale dale 35 Apr 6 09:11 test 4_dummy.txt
-rw-rw-r-- 1 dale dale 35 Apr 6 09:11 test 5_dummy.txt
-rw-rw-r-- 1 dale dale 35 Apr 6 09:11 test 6_dummy.txt
-rw-rw-r-- 1 dale dale 35 Apr 6 09:11 test 7_dummy.txt
-rw-rw-r-- 1 dale dale 35 Apr 6 09:11 test 8_dummy.txt
-rw-rw-r-- 1 dale dale 35 Apr 6 09:11 test 9_dummy.txt
-rw-rw-r-- 1 dale dale 35 Apr 6 09:11 test 10_dummy.txt
-rw-rw-r-- 1 dale dale 35 Apr 6 09:11 test 11_dummy.txt
$ tar cvf TEST.tar $(ls -t | head -5 | tr ' ' '?')
test 11_dummy.txt
test 10_dummy.txt
test 9_dummy.txt
test 8_dummy.txt
test 7_dummy.txt
Another option might be to redirect to a file and then use '-T':
ls -t | head > /tmp/10tarfiles.txt
echo tar zcf ~/backup/"$i".tar.gz -T /tmp/10tarfiles.txt"
rm /tmp/10tarfiles.txt

Tail latest file that matches a selected rule

Have a directory that multiple processes log to and I want to tail the latest file of a selected process.
in ~/bashrc I have added the following
function __taillog {
tail -f $(find $1 -maxdepth 1 -type f -printf "%T# %p\n" | sort -n | tail -n 1 | cut -d' ' -f 2-)
}
alias taillog='__taillog'
Taken from: https://superuser.com/questions/117596/how-to-tail-the-latest-file-in-a-directory
An example of the log file directory
-rw-r--r-- 1 genesys genesys 2284 Mar 19 16:34 gdalog.20130319_163436_906.log
-rw-r--r-- 1 genesys genesys 131072 Mar 19 16:34 gdalog.20130319_163436_906.snapshot.log
-rw-r--r-- 1 genesys genesys 10517 Mar 19 16:54 lcalog.20130319_163332_719.log
-rw-r--r-- 1 genesys genesys 131072 Mar 19 16:54 lcalog.20130319_163332_719.snapshot.log
-rw-r--r-- 1 genesys genesys 3792 Mar 19 16:37 StatServer_TLSTest.20130319_163700_703.log
-rw-r--r-- 1 genesys genesys 160562 Mar 19 16:52 StatServer_TLSTest.20130319_163712_045.log
-rw-r--r-- 1 genesys genesys 49730 Mar 19 16:54 StatServer_TLSTest.20130319_165217_402.log
-rw-r--r-- 1 genesys genesys 53960 Mar 20 09:55 StatServer_TLSTest.20130319_165423_702.log
-rw-r--r-- 1 genesys genesys 131072 Mar 20 09:56 StatServer_TLSTest.20130319_165423_702.snapshot.log
So to tail the all StatServer the command would be
taillog /home/user/logs/StatServer*
and it would tail the latest file for that application in the given path
The issue is the tail displays some of the file output but does not show any updates when the log file is appended. If the following command is run the log is tailed correctly
tail -f $(find /home/user/logs/StatServer* -maxdepth 1 -type f -printf "%T# %p\n" | sort -n | tail -n 1 | cut -d' ' -f 2-)
Some how adding this command as a bash function then calling it from an alias causes it to not operate as desired.
Any suggestion on a better way are welcome.
I believe you should be running this command:
taillog /home/user/logs
When you say /home/user/logs/this_app* you're passing all the files that match the pattern as argument to taillog and only using the first argument i.e. $1, and the command eventually translates to tail -f $1.
Instead $1 should be the directory where find should look for the files at that directory level (i.e. /home/user/logs in your case), then pipe the results to sort, tail and cut.
I didn't have any problems running your taillog function on linux/bash. Perhaps the log output is being buffered, so changes aren't being written right away? You might try turning off the [log]buffering option for this StatServer.

Resources