I tried the two following cmd to show content of all files under current directory. I want to know why one works, the other does not.
ls | xargs cat # does not work, No such file or directory
find . | xargs cat # works
cat is just an example, it can be any cmd which takes a file name as its parameter.
---------------------------------Update---------------------------------
Here are some observation from my PC.
$ echo 1 > test1.txt
$ echo 2 > test2.txt
$ echo 3 > test3.txt
$ ls
test1.txt test2.txt test3.txt
$ ls *.txt | xargs cat
cat: test1.txt: No such file or directory
cat: test2.txt: No such file or directory
cat: test3.txt: No such file or directory
$ find . -name '*.txt' | xargs cat
2
1
3
For others that might see this, we found the issue in the comments. Hao's problem was that ls was an alias, causing issues piping xargs to cat.
With 'type ls', they saw it was aliased, and using '\ls' to remove the alias solved the problem.
Related
I have a list of files inside directory1 and I want to move n number of files from that directory1 to directory2. When I try xargs like this it did not work.
ls -ltr | head -20 | mv xargs /directory2
Why we can't use xargs in middle? how to move n number of files to another directory in command line ?
First, you should not use ls in scripts. ls is for humans, not for scripting. Second, the last command of your pipe tries to move a file named xargs to /directory2. Third, xargs appends its inputs to the command. Even if you swap mv and xargs this will lead to execute mv /directory2 file1 file2 file3... file20, instead of what you want: mv file1 file2 file3... file20 /directory2. Finally, the -l option of ls will print more than just the file name (permissions, owner, group...); you cannot use it as mv argument.
Your ls options suggest that you want to move the 20 oldest files. Try:
while read -d '' -r time file; do
mv -- "$file" "/directory2"
done < <( stat --printf '%Y %n\0' * | sort -zn | head -zn20 )
stat --printf '%Y %n\0' * prints the last modification time as seconds since epoch, followed by the file name, of all files in the current directory. Each record is terminated by the NUL character, the only character that cannot be found in a file name. The -z option of sort and head and the -d '' option of read instruct these utilities to use NUL as the record separator instead of the default (newline). This way, the script should work even if some of your file names contain newlines.
If you prefer xargs:
stat --printf '%Y\t%n\0' * | sort -zn | head -zn20 | cut -zf2- |
xargs -0I{} mv -- {} /directory2
You can try this:
n=20
cd /path/to/directory1 || exit
files=(*)
mv -- "${files[#]:0:n}" /path/to/directory2
Also, you may consider reading this article: Why you shouldn't parse the output of ls
This is a backup script that copies files from one directory to another. I use a for loop to check if there are more than five files. If there are, the loop should delete the oldest entries first.
I tried ls -tr | head -n -5 | xargs rm from the command line and it works successfully to delete older files if there are more than 5 in the directory.
However, when I put it into my for loop, I get an error rm: missing operand
Here is the full script. I don't think I am using the for loop correctly in the script, but I'm really not sure how to use the commands ls -tr | head -n -5 | xargs rm in a loop that iterates over the files in the directory.
timestamp=$(date +"%m-%d-%Y")
dest=${HOME}/mybackups
src=${HOME}/safe
fname='bu_'
ffname=${HOME}/mybackups/${fname}${timestamp}.tar.gz
# for loop for deletion of file
for f in ${HOME}/mybackups/*
do
ls -tr | head -n -5 | xargs rm
done
if [ -e $ffname ];
then
echo "The backup for ${timestamp} has failed." | tee ${HOME}/mybackups/Error_${timestamp}
else
tar -vczf ${dest}/${fname}${timestamp}.tar.gz ${src}
fi
Edit: I took out the for loop, so it's now just:
[...]
ffname=${HOME}/mybackups/${fname}${timestamp}.tar.gz
ls -tr | head -n -5 | xargs rm
if [ -e $ffname ];
[...]
The script WILL work if it is in the mybackups directory, however, I continue to get the same error if it is not in that directory. The script gets the file names but tries to remove them from the current directory, I think... I tried several modifications but nothing has worked so far.
I get an error rm: missing operand
The cause of that error is that there are no files left to be deleted. To avoid that error, use the --no-run-if-empty option:
ls -tr | head -n -5 | xargs --no-run-if-empty rm
In the comments, mklement0 notes that this issue is peculiar to GNU xargs. BSD xargs will not run with an empty argument. Consequently, it does not need and does not support the --no-run-if-empty option.
More
Quoting from a section of code in the question:
for f in ${HOME}/mybackups/*
do
ls -tr | head -n -5 | xargs rm
done
Note that (1) f is never used for anything and (2) this runs the ls -tr | head -n -5 | xargs rm several times in a row when it needs to be run only once.
Obligatory Warning
Your approach parses the output of ls. This makes for a simple and easily understood command. It can work if all your files are sensibly named. It will not work in general. For more on this, see: Why you shouldn't parse the output of ls(1).
Safer Alternative
The following will work with all manner of file names, whether they contains spaces, tabs, newlines, or whatever:
find . -maxdepth 1 -type f -printf '%T# %i\n' | sort -n | head -n -5 | while read tstamp inode
do
find . -inum "$inode" -delete
done
SMH. I ended up coming up to the simplest solution in the world by just cd-ing into the directory before I ran ls -tr | head -n -5 | xargs rm . Thanks for everyone's help!
timestamp=$(date +"%m-%d-%Y")
dest=${HOME}/mybackups
src=${HOME}/safe
fname='bu_'
ffname=${HOME}/mybackups/${fname}${timestamp}.tar.gz
cd ${HOME}/mybackups
ls -tr | head -n -5 | xargs rm
if [ -e $ffname ];
then
echo "The backup for ${timestamp} has failed." | tee ${HOME}/mybackups/Error_${timestamp}
else
tar -vczf ${dest}/${fname}${timestamp}.tar.gz ${src}
fi
This line ls -tr | head -n -5 | xargs rm came from here
ls -tr displays all the files, oldest first (-t newest first, -r
reverse).
head -n -5 displays all but the 5 last lines (ie the 5 newest files).
xargs rm calls rm for each selected file
.
I have about 2000 files in a folder.
All the files contain the string test in the name.
What I need to do is move all those files ~1250 to a folder called trash within the same directory and append _scrap to the end of each file.
mv *test* trash/
What I want is something like this:
[root#server] ls
test1.txt test2.txt test3.txt trash video1.txt video2.txt video3.txt
[root#server] mv *test* trash/*_scrap
[root#server] ls
trash vidoe1.txt video2.txt video3.txt
[root#server] ls trash/
test1.txt_scrap test2.txt_scrap test3.txt_scrap
I can move all files, however I cannot figure out how to append the _scrap to the end.
As I have to do this on a number of machines, a one liner would be preferable over a small script.
$ touch test1.txt test2.txt test3.txt vidoe1.txt vidoe2.txt vidoe3.txt
$ mkdir trash
$ for file in *test*; do mv "$file" "trash/${file}_scrap"; done
$ ls
trash vidoe1.txt vidoe2.txt vidoe3.txt
$ ls trash
test1.txt_scrap test2.txt_scrap test3.txt_scrap
$
You could also use xargs
$ ls *test* | xargs -t -I{} mv {} trash/{}_scrap
mv test1.txt trash/test1.txt_scrap
mv test2.txt trash/test2.txt_scrap
mv test3.txt trash/test3.txt_scrap
$
You could use find
$ find . -name '*test*' -maxdepth 1 -exec mv {} trash/{}_scrap \;
You can use rename to avoid shell for loops. It's a perl script but it comes installed with many common distros (including Ubuntu 14):
$ mv *test* trash/
$ rename 's/$/_scrap/g' trash/*
$ ls trash/
test1.txt_scrap test3.txt_scrap test2.txt_scrap
I would like to find the most recently changed file in a directory, excluding hidden files (the ones that start with .) and also excluding directories.
This question is headed in the right direction, but not exactly what I need:
Linux: Most recent file in a directory
The key here is to exclude directories...
Like the answer there except without -A
ls -rt | tail -n 1
Look at man ls for more info.
To make it exclude directories, we use the -F option to add a "/" to each directory, and then filter for those that don't have the "/":
ls -Frt | grep "[^/]$" | tail -n 1
This does what you want, excluding directories:
stat --printf='%F %Y %n\n' * | sort | grep -v ^directory | head -n 1
same one, not very clean but: ls -c1 + tail if you want => ls -c1 | tail -1
$ touch a .b
$ ls -c1
a
$ ls -c1a
a
.b
$ touch d
$ ls -c1
d
a
$ ls -c1a
.
d
a
.b
..
$ touch .b
$ ls -c1a
.b
.
d
a
..
As you can see, without a arg, only visible files are listed.
probably the same as the answer in the other post but with a small difference (excluding directories) -
ls --group-directories-first -rt | tail -n 1
i have a lot of different type of files in one folder. i need to delete the files but except the pdf file.
I tried to display the pdf file only. but i need to delete the other than pdf files
ls -1 | xargs file | grep 'PDF document,' | sed 's/:.*//'
You could do the following - I've used echo rm instead of rm for safety:
for i in *
do
[ x"$(file --mime-type -b "$i")" != xapplication/pdf ] && echo rm "$i"
done
The --mime-type -b options to file make the output of file easier to deal with in a script.
$ ls
aa.txt a.pdf bb.cpp b.pdf
$ ls | grep -v .pdf | xargs rm -rf
$ ls
a.pdf b.pdf
:) !
ls |xargs file|awk -F":" '!($2~/PDF document/){print $1}'|xargs rm -rf
Try inverting the grep match:
ls -1 | xargs file | grep -v 'PDF document,' | sed 's/:.*//'
It's rare in my experience to encounter PDF files which don't have a .pdf extension. You don't state why "file" is necessary in the example, but I'd write this as:
# find . -not -name '*.pdf' -delete
Note that this will recurse into subdirectories; use "-maxdepth 1" to limit to the current directory only.