I have RaspBerry Pi project where A/D converter collects 5 minutes long data files into data folder and all data except newest one will be removed in every hour by dataFlush.py script and crontab. The path of dataFlush.py is "/home/pi" and the path of the data folder is "/home/pi/dataLog". So how do I add dataLog path into this command:
os.system('ls -t | tail -n +2 | xargs rm --')
I tried something like this but it didn't work:
os.system('ls /home/pi/dataLog -t | tail -n +2 | xargs rm --')
ls(1) won't generate full-path outputs in your case, so you might want to prepend every line with the "prefix":
os.system('ls /home/pi/dataLog -t | tail -n +2 | sed 's|^|/home/pi/dataLog/|' | xargs rm --')
Another options is to use find(1):
os.system('find /home/pi/dataLog | xargs ls -t -- | tail -n +2 | xargs rm --')
Related
Here is my directory structure.
./TEST1/automation
./TEST2_1/automation
./TEST3.4/automation
./general/automation
I want to preserve only the latest 5 sub-folders under all directories that starts with TEST*/automation.
Currently, my script goes into each directory as below and executes the command:
./TEST1/automation
ls -dt */ | tail -n +5 | xargs rm -rf
./TEST2_1/automation
ls -dt */ | tail -n +5 | xargs rm -rf
./TEST3.4/automation
ls -dt */ | tail -n +5 | xargs rm -rf
Everytime we add a new folder that starts with TEST, I've to manually update the script.
Basically, go into all directories that starts with TEST*/automation and preseve only latest 5 folders.
Try this one:
find -regex '.*/TEST.*/automation' -print0 |xargs -0 -I {} -n1 bash -c 'cd "{}"; ls -rt | tail -n +4 | xargs -I {} echo rm -rf -- "{}"'
If the output looks alright (check that it does indeed show "rm ... " for all files/directories you want to get rid of) remove the echo.
Caveat: the second part of execution (the first xargs does not explicitly look for directories, it will also list files. From your description it's unclear whether your automation directories contain both files and directories or just directories.
I would like to find files in my home directory that start with '~', sort them numerically, print the first five and delete them using find command and pipes in Linux. I have a bash script:
#!/bin/bash
find ~/ -name "~*" | sort -n | head -5 | tee | xargs rm
This works fine for deleting files, but I was expecting tee command to print deleted files to standard output. All this command does is delete files, but there in so output in terminal. What should I add/ change?
Thank you.
You could just use the verbose flag on rm and it will tell you what it's deleting
find ~/ -name "~*" | sort -n | head -5 | xargs rm -v
Use man rm to see the docs
-v, --verbose
explain what is being done
You can use rm -v to print each deleting filename:
find ~ -name '~*' -print0 | sort -zn | head -z -n 5 | xargs -0 rm -v
Also note use -print0 and all corresponding options in sort. head, xargs to address filenames with whitespace and glob characters.
How would I do the following command? This is the following pseudocode I have:
ls -U | head -4 < rm *
I want to delete the below four files
ubuntu#ip:/tmp$ ls -U | head -4
c9e5aff2-f4f6-459e-9752-466625165fa0.jpg
815626f8-2650-4596-94b9-796a82780390.jpg
b9259a6b-b421-4b64-ba4d-72fd632703a3.jpg
78c243e7-43df-417d-9dcf-a4578f1c1f20.jpg
xargs should do the trick:
$ ls -U | head -4 | xargs rm
If you can't remember bash syntax a simple for loop often works like a charm:
for file in `ls -U | head -4`; do rm $file; done
I have changed this post down to just what I think the main problem is..
There is a need to leave the two latest
how to delete all files except the latest three in a folder
ls -t1 /home/jdoe/checks/downloads/*.md5 | head -n +2 | xargs rm -r
This removes the oldest files..
And to test:
ls -t1 /home/jdoe/checks/downloads/*.md5 | head -n +2
We really want to leave the two (2) newest files:
ls -t1 /home/jdoe/checks/downloads/*.md5 | tail -n +2 | xargs rm -r
This does not seem to work..
And to test:
ls -t1 /home/jdoe/checks/downloads/*.md5 | tail -n +2
Thanks!
I was able to get some assistance from one of my co-workers and this appears to be what we need.
ls -1tr /home/jdoe/checks/downloads/*.md5 | head -n -2 | while read f; do
#rm -f "$f"
print "file to delete is $f"
done
This is a backup script that copies files from one directory to another. I use a for loop to check if there are more than five files. If there are, the loop should delete the oldest entries first.
I tried ls -tr | head -n -5 | xargs rm from the command line and it works successfully to delete older files if there are more than 5 in the directory.
However, when I put it into my for loop, I get an error rm: missing operand
Here is the full script. I don't think I am using the for loop correctly in the script, but I'm really not sure how to use the commands ls -tr | head -n -5 | xargs rm in a loop that iterates over the files in the directory.
timestamp=$(date +"%m-%d-%Y")
dest=${HOME}/mybackups
src=${HOME}/safe
fname='bu_'
ffname=${HOME}/mybackups/${fname}${timestamp}.tar.gz
# for loop for deletion of file
for f in ${HOME}/mybackups/*
do
ls -tr | head -n -5 | xargs rm
done
if [ -e $ffname ];
then
echo "The backup for ${timestamp} has failed." | tee ${HOME}/mybackups/Error_${timestamp}
else
tar -vczf ${dest}/${fname}${timestamp}.tar.gz ${src}
fi
Edit: I took out the for loop, so it's now just:
[...]
ffname=${HOME}/mybackups/${fname}${timestamp}.tar.gz
ls -tr | head -n -5 | xargs rm
if [ -e $ffname ];
[...]
The script WILL work if it is in the mybackups directory, however, I continue to get the same error if it is not in that directory. The script gets the file names but tries to remove them from the current directory, I think... I tried several modifications but nothing has worked so far.
I get an error rm: missing operand
The cause of that error is that there are no files left to be deleted. To avoid that error, use the --no-run-if-empty option:
ls -tr | head -n -5 | xargs --no-run-if-empty rm
In the comments, mklement0 notes that this issue is peculiar to GNU xargs. BSD xargs will not run with an empty argument. Consequently, it does not need and does not support the --no-run-if-empty option.
More
Quoting from a section of code in the question:
for f in ${HOME}/mybackups/*
do
ls -tr | head -n -5 | xargs rm
done
Note that (1) f is never used for anything and (2) this runs the ls -tr | head -n -5 | xargs rm several times in a row when it needs to be run only once.
Obligatory Warning
Your approach parses the output of ls. This makes for a simple and easily understood command. It can work if all your files are sensibly named. It will not work in general. For more on this, see: Why you shouldn't parse the output of ls(1).
Safer Alternative
The following will work with all manner of file names, whether they contains spaces, tabs, newlines, or whatever:
find . -maxdepth 1 -type f -printf '%T# %i\n' | sort -n | head -n -5 | while read tstamp inode
do
find . -inum "$inode" -delete
done
SMH. I ended up coming up to the simplest solution in the world by just cd-ing into the directory before I ran ls -tr | head -n -5 | xargs rm . Thanks for everyone's help!
timestamp=$(date +"%m-%d-%Y")
dest=${HOME}/mybackups
src=${HOME}/safe
fname='bu_'
ffname=${HOME}/mybackups/${fname}${timestamp}.tar.gz
cd ${HOME}/mybackups
ls -tr | head -n -5 | xargs rm
if [ -e $ffname ];
then
echo "The backup for ${timestamp} has failed." | tee ${HOME}/mybackups/Error_${timestamp}
else
tar -vczf ${dest}/${fname}${timestamp}.tar.gz ${src}
fi
This line ls -tr | head -n -5 | xargs rm came from here
ls -tr displays all the files, oldest first (-t newest first, -r
reverse).
head -n -5 displays all but the 5 last lines (ie the 5 newest files).
xargs rm calls rm for each selected file
.