I'm using Ubuntu 16.04.1 LTS
I found a script to delete everything but the 'n' newest files in a directory.
I modified it to this:
sudo rm /home/backup/`ls -t /home/backup/ | awk 'NR>5'`
It deletes only one file. It reports the following message about the rest of the files it should have deleted:
rm: cannot remove 'delete_me_02.tar': No such file or directory
rm: cannot remove 'delete_me_03.tar': No such file or directory
...
I believe that the problem is the path. It's looking for delete_me_02.tar (and subsequent files) in the current directory, and it's somehow lost its reference to the correct directory.
How can I modify my command to keep looking in the /home/backup/ directory for all 'n' files?
Maybe find could help you do what you want:
find /home/backup -type f | xargs ls -t | head -n 5 | xargs rm
But I would first check what find would return (just remove | xargs rm) and check what is going to be removed.
The command in the backticks will be expanded to the list of relative file paths:
%`ls -t /home/backup/ | awk 'NR>5'`
a.txt b.txt c.txt ...
so the full command will now look like this:
sudo rm /home/backup/a.txt b.txt c.txt
which, I believe, makes it pretty obvious on why only the first file is removed.
There is also a limit on a number of arguments you can pass to rm, so
you better modify your script to use xargs instead:
ls -t|tail -n+5|xargs -I{} echo rm /home/backup/'{}'
(just remove echo, once you verify that it produces an expected results for you)
After the command substitution expands, your command line looks like
sudo rm /home/backup/delete_me_01.tar delete_me_02.tar delete_me_03.tar etc
/home/backup is not prefixed to each word from the output. (Aside: don't use ls in a script; see http://mywiki.wooledge.org/ParsingLs.)
Frankly, this is something most shells just doesn't make easy to do properly. (Exception: with zsh, you would just use sudo rm /home/backup/*(Om[1,-6]).) I would use some other language.
Related
This is a little anecdote from earlier on why not running root is vital.
I was sorting my home directory and deleted a few compressed files I had, I wrote
ls . | grep -P 'zip|tar|7z' | xargs rm and thought, hey I could also write this as rm -r $(ls . | grep -P '...') I suppose.
The second part I didn't mean to use it since there was nothing to delete, it was morelike a mental exercise, I wrote it next to the last command with a 'divider' to visually compare them.
ls . | grep -P 'zip|tar|7z' | xargs rm **//** rm -r $(ls . | grep -P '...')
Being **//** the "divider" and ... the mental "substitute" for 'zip|tar..'
I thought this wouldn't run but to my surprise, it acted as rm -r /and tried to delete everything, luckily permissions saved me and nothing was deleted.
But I'm curious why it'd work that way,
my guess is that rm **//** somehow translated to rm / but I'm not sure.
In the zsh shell, **//** would expand to all names under / as well to all names below the current directory (recursively).
From an empty directory on my system:
$ echo **//**
/altroot /bin /boot /bsd /bsd.booted /bsd.rd /bsd.sp /dev /etc /extra /home /mnt /root /sbin /sys /tmp /tmp_mnt /usr /var /vol
Why? Well, **/ matches all directories recursively under the current directory. More importantly, it matches the current directory, but since the current directory's name is not available inside the current directory, there's no entry returned for that.
However, when you add a / to that to create **//, then you get a lone / back for the current directory. Again in an empty directory:
$ echo **//
/
Then, if you add a further ** to make **//**, you pick up all names from the root directory, together with all names from the current directory and below (directory names from the current directory and below would occur twice in the list).
Your xargs is calling
rm **//** rm -r $(ls . | grep -P '...')
If you're using GNU rm, it will helpfully rearrange the command line so that it is interpreted the same as
rm -r **//** rm $(ls . | grep -P '...')
What this does should now be clear.
If you want to delete all regular files in the current directory that have filename suffixes .zip, .tar or .7z, use
rm ./*.(zip|tar|7z)(.)
in the zsh shell. If want to do that recursively down into subdirectories, use
rm ./**/*.(zip|tar|7z)(.)
The glob qualifier (.) makes the globbing pattern only match regular files. You could even restrict it to files above a certain size, say 10MB, with ./**/*.(zip|tar|7z)(.Lm+10).
One difference is that the ls ... | xargs .... solution also works if there are really a lot of files involved, while your rm $( .... ) might produce a argument list too long error. But if this is not an issue in your case, an even simpler attempt would be (assuming here Zsh; I don't understand why you tagged this bash, since you explicitly refer to Zsh only in your question)
rm *(zip|tar|7z)*(N)
which would express your original statement; I believe however that you really meant
rm -- *.(zip|tar|7z)(N)
because the solution you posted would also remove a file tarpit.txt, for instance. The (N) flag is a frail attempt to treat the case, that you don't have any file matching the pattern. Without the (N), you would get an error message from Zsh, and rm would receive the unexpanded file pattern, and, since it is unlikely that a file of this name exists, would output a second error message. By using (N), Zsh would simply pass nothing in this case (without complaining), and in fact rm would be invoked without arguments. Of course you would then get a rm: missing operand on stderr, and if you don't like this, you can filter this message.
UPDATES:
As Kusalananda has pointed out in his/her comment, omitting the (N) would, by default, make zsh only print an error message, if no files match the pattern, but not cause rm to be invoked.
Also added the -- flag to rm to allow removal of, i.e., a file called -rf.tar.
I want to delete oldest files in a directory when the number of files is greater than 5. I'm using
(ls -1t | tail -n 3)
to get the oldest 3 files in the directory. This works exactly as I want. Now I want to delete them in a single command with rm. As I'm running these commands on a Linux server, cd into the directory and deleting is not working so I need to use either find or ls with rm and delete the oldest 3 files. Please help out.
Thanks :)
If you want to delete files from some arbitrary directory, then pass the directory name into the ls command. The default is to use the current directory.
Then use $() parameter expansion to transfer the result of tail into rm like this
rm $(ls -1t dirname| tail -n 3)
rm $(ls -1t | tail -n 3) 2> /dev/null
ls may return No such file or directory error message, which may cause rm to run unnessesary with that value.
With the help of following answer: find - suppress "No such file or directory" errors and https://unix.stackexchange.com/a/140647/198423
find $dirname -type d -exec ls -1t {} + | tail -n 3 | xargs rm -rf
This question already has answers here:
bash script to remove directories based on modified file date
(3 answers)
Closed 8 years ago.
Hi so I'm trying to remove old backup files from a sub directory if the number of files exceeds the maximum and I found this command to do that
ls -t | sed -e '1,10d' | xargs -d '\n' rm
And my changes are as follows
ls -t subdirectory | sed -e '1,$f' | xargs -d '\n' rm
Obviously when I try running the script it gives me an error saying unknown commands: f
My only concern right now is that I'm passing in the max number of files allowed as an argument so I'm storing that in f but now I'm not too sure how to use that variable in the command above instead of having to set condition to a specific number.
Can anyone give me any pointers? And is there anything else I'm doing wrong?
Thanks!
The title of your question says "based on modification date". So why not simply using find with mtime option?
find subdirectory -mtime +5d -exec rm -v {} \;
Will delete all files older than 5 days.
The problem is that the file list you are passing to xargs does not contain the needed path information to delete the files. When called from the current directory, no path is needed, but if you call it with subdirectory, you need to then rm subdirectory/file from the current directory. Try it:
ls -t subdirectory # returns files with no path info
What you need to do is change to the subdirectory, call the removal script, then change back. In one line it could be done with:
pushd subdirectory &>/dev/null; ls -t | sed -e '1,$f' | xargs -d '\n' rm; popd
Other than doing it in a similar manner, you are probably better writing a slightly longer and more flexible script forming the list of files with the find command to insure the path information is retained.
I am very new to the unix. I ran the following command.
ls -l | xargs rm -rf bark.*
and above command removed every directory in the folder.
Can any one explained me why ?
The -r argument means "delete recursively" (ie descend into subdirectories). The -f command means "force" (in other words, don't ask for confirmation). -rf means "descend recursively into subdirectories without asking for confirmation"
ls -l lists all files in the directory. xargs takes the input from ls -l and appends it to the command you pass to xargs
The final command that got executed looked like this:
rm -rf bark.* <output of ls -l>
This essentially removed bark.* and all files in the current directory. Moral of the story: be very careful with rm -rf. (You can use rm -ri to ask before deleting files instead)
rm(1) deleted every file and directory in the current working directory because you asked it to.
To see roughly what happened, run this:
cd /etc ; ls -l | xargs echo
Pay careful attention to the output.
I strongly recommend using echo in place of rm -rf when constructing command lines. Only if the output looks fine should you then re-run the command with rm -rf. When in doubt, maybe just use rm -r so that you do not accidentally blow away too much. rm -ir if you are very skeptical of your command line. (I have been using Linux since 1994 and I still use this echo trick when constructing slightly complicated command lines to selectively delete a pile of files.)
Incidentally, I would avoid parsing ls(1) output in any fashion -- filenames can contain any character except ASCII NUL and / chars -- including newlines, tabs, and output that looks like ls -l output. Trying to parse this with tools such as xargs(1) can be dangerous.
Instead, use find(1) for these sorts of things. To delete all files in all directories named bark.*, I'd run a command like this:
find . -type d -name 'bark.*' -print0 | xargs -0 rm -r
Again, I'd use echo in place of rm -r for the first execution -- and if it looked fine, then I'd re-run with rm -r.
The ls -l command gave a list of all the subdirectories in your current present-working-directory (PWD).
The rm command can delete multiple files/directories if you pass them to it as a list.
eg: rm test1.txt test2.txt myApp will delete all three of the files with names:
test1.txt
test2.txt
myApp
Also, the flags for the rm command you used are common in many a folly.
rm -f - Force deletion of files without asking or confirming
rm -r - Recurse into all subdirectories and delete all their contents and subdirectories
So, let's say you are in /home/user, and the directory structure looks like so:
/home/user
|->dir1
|->dir2
`->file1.txt
the ls -l command will provide the list containing "dir1 dir2 file1.txt", and the result of the command ls -l | xargs rm -rf will look like this:
rm -rf dir1 dir2 file1.txt
If we expand your original question with the example above, the final command that gets passed to the system becomes:
rm -rf di1 dir2 file1.txt bark.*
So, everything in the current directory gets wiped out, so the bark.* is redundant (you effectively told the machine to destroy everything in the current directory anyway).
I think what you meant to do was delete all files in the current directory and all subdirectories (recurse) that start with bark. To do that, you just have to do:
find -iname bark.* | xargs rm
The command above means "find all files in this directory and subdirectories, ignoring UPPERCASE/lowercase/mIxEdCaSe, that start with the characters "bark.", and delete them". This could still be a bad command if you have a typo, so to be sure, you should always test before you do a batch-deletion like this.
In the future, first do the following to get a list of all the files you will be deleting first to confirm they are the ones you want deleted.
find -iname bark.* | xargs echo
Then if you are sure, delete them via
find -iname bark.* | xargs rm
Hope this helps.
As a humorous note, one of the most famous instances of "rm -rf" can be found here:
https://github.com/MrMEEE/bumblebee-Old-and-abbandoned/commit/a047be85247755cdbe0acce6f1dafc8beb84f2ac
An automated script runs something like rm -rf /usr/local/........., but due to accidentally inserting a space, the command became rm -rf /usr /local/......, so this effectively means "delete all root folders that start with usr or local", effectively destroying the system of anyone who uses it. I feel bad for that developer.
You can avoid these kinds of bugs by quoting your strings, ie:
rm -rf "/usr/ local/...." would have provided an error message and avoided this bug, because the quotes mean that everything between them is the full path, NOT a list of separate paths/files (ie: you are telling rm that the file/folder has a SPACE character in its name).
If we want to delete all files and directories we use, rm -rf *.
But what if i want all files and directories be deleted at a shot, except one particular file?
Is there any command for that? rm -rf * gives the ease of deletion at one shot, but deletes even my favourite file/directory.
Thanks in advance
find can be a very good friend:
$ ls
a/ b/ c/
$ find * -maxdepth 0 -name 'b' -prune -o -exec rm -rf '{}' ';'
$ ls
b/
$
Explanation:
find * -maxdepth 0: select everything selected by * without descending into any directories
-name 'b' -prune: do not bother (-prune) with anything that matches the condition -name 'b'
-o -exec rm -rf '{}' ';': call rm -rf for everything else
By the way, another, possibly simpler, way would be to move or rename your favourite directory so that it is not in the way:
$ ls
a/ b/ c/
$ mv b .b
$ ls
a/ c/
$ rm -rf *
$ mv .b b
$ ls
b/
Short answer
ls | grep -v "z.txt" | xargs rm
Details:
The thought process for the above command is :
List all files (ls)
Ignore one file named "z.txt" (grep -v "z.txt")
Delete the listed files other than z.txt (xargs rm)
Example
Create 5 files as shown below:
echo "a.txt b.txt c.txt d.txt z.txt" | xargs touch
List all files except z.txt
ls|grep -v "z.txt"
a.txt
b.txt
c.txt
d.txt
We can now delete(rm) the listed files by using the xargs utility :
ls|grep -v "z.txt"|xargs rm
You can type it right in the command-line or use this keystroke in the script
files=`ls -l | grep -v "my_favorite_dir"`; for file in $files; do rm -rvf $file; done
P.S. I suggest -i switch for rm to prevent delition of important data.
P.P.S You can write the small script based on this solution and place it to the /usr/bin (e.g. /usr/bin/rmf). Now you can use it as and ordinary app:
rmf my_favorite_dir
The script looks like (just a sketch):
#!/bin/sh
if [[ -z $1 ]]; then
files=`ls -l`
else
files=`ls -l | grep -v $1`
fi;
for file in $files; do
rm -rvi $file
done;
At least in zsh
rm -rf ^filename
could be an option, if you only want to preserve one single file.
If it's just one file, one simple way is to move that file to /tmp or something, rm -Rf the directory and then move it back. You could alias this as a simple command.
The other option is to do a find and then grep out what you don't want (using -v or directly using one of finds predicates) and then rming the remaining files.
For a single file, I'd do the former. For anything more, I'd write something custom similar to what thkala said.
In bash you have the !() glob operator, which inverts the matched pattern. So to delete everything except the file my_file_name.txt, try this:
shopt -s extglob
rm -f !(my_file_name.txt)
See this article for more details:
http://karper.wordpress.com/2010/11/17/deleting-all-files-in-a-directory-with-exceptions/
I don't know of such a program, but I have wanted it in the past for some times. The basic syntax would be:
IFS='
' for f in $(except "*.c" "*.h" -- *); do
printf '%s\n' "$f"
done
The program I have in mind has three modes:
exact matching (with the option -e)
glob matching (default, like shown in the above example)
regex matching (with the option -r)
It takes the patterns to be excluded from the command line, followed by the separator --, followed by the file names. Alternatively, the file names might be read from stdin (if the option -s is given), each on a line.
Such a program should not be hard to write, in either C or the Shell Command Language. And it makes a good excercise for learning the Unix basics. When you do it as a shell program, you have to watch for filenames containing whitespace and other special characters, of course.
I see a lot of longwinded means here, that work, but with
a/ b/ c/ d/ e/
rm -rf *.* !(b*)
this removes everything except directory b/ and its contents (assuming your file is in b/.
Then just cd b/ and
rm -rf *.* !(filename)
to remove everything else, but the file (named "filename") that you want to keep.
mv subdir/preciousfile ./
rm -rf subdir
mkdir subdir
mv preciousfile subdir/
This looks tedious, but it is rather safe
avoids complex logic
never use rm -rf *, its results depend on your current directory (which could be / ;-)
never use a globbing *: its expansion is limited by ARGV_MAX.
allows you to check the error after each command, and maybe avoid the disaster caused by the next command.
avoids nasty problems caused by space or NL in the filenames.
cd ..
ln trash/useful.file ./
rm -rf trash/*
mv useful.file trash/
you need to use regular expression for this. Write a regular expression which selects all other files except the one you need.