I'm trying to use du command for every directory in the current one. So I'm trying to use code like this:
ls | du -sb
But its not working as expected. It outputs only size of current '.' directory and thats all.
The same thing is with echo
ls | echo
Outputs empty line. Why is this happening?
Using a pipe sends the output (stdout) of the first command, to stdin (input) of the child process (2nd command). The commands you mentioned don't take any input on stdin. This would work, for example, with cat (and by work, I mean work like cat run with no arguments, and just pass along the input you give it):
ls | cat
For your applications, this is where xargs comes in. It takes piped input and gives it as arguments to the command specified. So, you can make it work like:
ls | xargs du -sb
Beware that by default xargs will break its input on spaces, so if your filenames contain spaces this won't work as you want. So, in this particular case, this would be better:
du -sb *
Use command substitution, like this:
du -sb $(ls -d */)
$ find . -type d -maxdepth 1 -exec du -sb {} \;
or
$ ls -d */ | xargs du -sb
Related
I wrote an alias to show me 10 biggest files and 10 biggest directories, so I tried to hide "du cannot access" and "no such file or directory" errors, without success.
Below the alias that I wrote:
alias big="echo 'Big Files:';find . -type f -print0 | xargs -0 du -h | sort -hr | head -10 2>/dev/null;echo 'Big Directories:';du -sh * | sort -hr | head -n10 2>/dev/null"
Thanks for help
First: Use a function instead. This kind of application is entirely the wrong use case for an alias. A function will let you write your logic over multiple lines; can have conditional logic at execution (for instance, you can look at whether your function is given arguments, and behave differently if they're present); and can be called with the same syntax (it's just big to invoke in either case).
Second: Redirect stderr for the whole thing, not just the last segment.
big() {
{ # <-- open a block we can redirect
echo 'Big Files:'
find . -type f -exec du -h -- '{}' + |
sort -hr |
head -10
echo 'Big Directories:'
du -sh -- */ |
sort -hr |
head -n10
} 2>/dev/null # <-- close, and redirect, that block
}
You can also do the same thing while continuing to use an alias (though, again, you shouldn't):
alias big='{ echo "Big Files:";find . -type f -print0 | xargs -0 du -h | sort -hr | head -10;echo "Big Directories:";du -sh * | sort -hr | head -n10; } 2>/dev/null'
Here, wrapping the entire code in { } lets us perform a single redirection across the whole block without the expense of a subshell (as would be created by wrapping the code in ( ).
Other notes:
find -exec ... {} + is POSIX-specified, and has the same efficiency gains as find ... -print0 | xargs -0 (in terms of spawning no more instances of the child process than necessary), and in fact is slightly more efficient, as it avoids the need for an external xargs process.
Using */ in the "Big Directories" segment tells the glob to expand only to directory names. Using the -- argument prior tells du to interpret all arguments given past that point as file or directory names, even if you have a name that starts with a -.
For folks trying to test this answer on MacOS, note that it depends on GNU sort. I personally have gsort installed via macports (port install coreutils), so used sort() { gsort "$#"; } to wrap the sort command while testing.
Try piping the stderr stream of your du command to /dev/null
du -sh * 2>/dev/null | sort -hr
I am trying to move all directories less than 1GB. I am trying to use this command:
du -h -d 1 -t -1G | xargs -0 mv -it /destination/dir/
But I get an error message:
mv: cannot stat [...] File name too long
Help would be greatly appreciated :)
I'm not sure why you're using the -0 argument to xargs as this specifies that the filenames are separated by null bytes rather than spaces. The output of du won't contain any null bytes so the entire output will be treated as a single filename, causing the error that you're seeing.
Anyway, I would suggest using find:
find /path/to/source -type d -size -1024M -exec mv -it /path/to/destination {} +
If you're happy that du is already producing the output that you want and want to use it instead, you can add the -0 switch so that it uses null byte separators, then your current xargs command should work.
So here is a workaround that serves my needs. Perhaps somebody can expand on it? Anyway, IF you don't need to worry about sub directories then the following works.
du -Sb -t -1G | cut -f 2- | xargs -d "\n" mv -t /path/to/destination/
I have a directory with a large number of files. I am attempting to search for text located in at least one of the files. The text is likely located in one of the more recent files. What is the command to do this? I thought it would look something like ls -t | head -5 | grep abaaba.
For example, if I have 5 files returned from ls -t | head -5:
- file1, file2, file3, file4, file5, I need to know which of those files contains abaaba.
It's not really clear what you are trying to do. But I assume the efficiency is your main goal. I would use something like:
ls -t | while read -r f; do grep -lF abaaba "$f" && break;done
This will print only first file containing the string and stops the search. If you want to see actual lines use -H instead of -l. And if you have regex instead of mere string drop -F which will make grep run slower however.
ls -t | while read -r f; do grep -H abaaba "$f" && break;done
Of course if you want to continue the search I'd suggest dropping "&& break".
ls -t | while read -r f; do grep -HF abaaba "$f";done
If you have some ideas about the time frame, it's good idea to try find.
find . -maxdepth 1 -type f -mtime -2 -exec grep -HF abaaba {} \;
You can raise the number after -mtime to cover more than last 2 days.
If you're just doing this interactively, and you know you don't have spaces in your filenames, then you can do:
grep abaaba $(ls -t | head -5) # DO NOT USE THIS IN A SCRIPT
If writing this in an alias or for repeat future use, do it the "proper" way that takes more typing, but that doesn't break on spaces and other things in filenames.
If you have spaces but not newlines, you can also do
(IFS=$'\n' grep abaaba $(ls -t | head -5) )
I was trying to execute a script n times with a different file as argument each time using this:
ls /user/local/*.log | xargs script.pl
(script.pl accepts file name as argument)
But the script is executed only once. How to resolve this ? Am I not doing it correctly ?
ls /user/local/*.log | xargs -rn1 script.pl
I guess your script only expects one parameter, you need to tell xargs about that.
Passing -r helps if the input list would be empty
Note that something like the following is, in general, better:
find /user/local/ -maxdepth 1 -type f -name '*.log' -print0 |
xargs -0rn1 script.pl
It will handle quoting and directories safer.
To see what xargs actually executes use the -t flag.
I hope this helps:
for i in $(ls /usr/local/*.log)
do
script.pl $i
done
I would avoid using ls as this is often an alias on many systems. E.g. if your ls produces colored output, then this will not work:
for i in ls; do ls $i;done
Rather, you can just let the shell expand the wildcard for you:
for i in /usr/local/*.log; do script.pl $i; done
How can i execute a certain command for every file/folder in the current folder?
I've started with this as a base script, but this seems that its only working when using temporary files, and i dont really like the ideea. Is there any other way?
FOLDER=".";
DIRS=`ls -1 "$FOLDER">/tmp/DIRS`;
echo >"/tmp/DIRS1";
while read line ; do
SIZE=`du "$FOLDER$line"`;
echo $SIZE>>"/tmp/DIRS1";
done < "/tmp/DIRS";
For anyone interested, i wanted to make a list of folders, sorted by their size. Here is the final result
FOLDER="$1";
for f in $FOLDER/*; do
du -sb "$f";
done | sort -n | sed "s#^[0-9]*##" | sed "s#^[^\./]*##" | xargs -L 1 du -sh | sed "s|$FOLDER||";
which leads to du -sb $FOLDER/* | sort -n | sed "s#^[0-9]*##" | sed "s#^[^\./]*##" | xargs -L 1 du -sh | sed "s|$FOLDER||";
Perhaps xargs, which reinvokes the command specified after it for each additional line of parameters received on stdin...
ls -1 $FOLDER | xargs du
But, in this case, why not...
du *
...? Or...
for X in *; do
du $X
done
(Personally, I use zsh, where you can modify the glob pattern to only find say regular files, or only directories, only symlinks etc - I'm pretty sure there's something similar in bash - can dig for details if you need that).
Am I missing part of your requirement?
The find command will let you execute a command for each item it finds, too. Without further arguments it will find all files and folders in the current directory, like this:
$ find -exec du -h {} \;
The {} part is the "variable" where the match is placed, here as the argument to du. \; ends the command.
It is useless to parse output of ls to cycle over files. Bash can do it with wildcard expansion.
Storing the result of du in a variable to output it to a file is also a useless use of a variable.
What I suggest:
for i in ./tmp/DIRS/*
do
du "$i" >> "/tmp/DIRS1"
done
What's wrong with something like this?
function process() {
echo "Processing $1"
}
for i in *
do
process $i
done
You can put all the "work" you want done inside the function process. This will do it for your current directory.
This works for every file in the current directory:
do
/usr/local/mp3unicode/bin/mp3unicode -s cp1251 --id3v2-encoding unicode "$file"
done
The invocation of action exec can be done by two ways:
find . -type d -exec du -ch {} \;
find . -type d -exec du -ch {} +
In the first command, the substitution {} occurs for each folder found. In the second one all the results of find are passed to exec at once, which matters, to obtain a final total.
https://www.eovao.com/en/a/bash%20find%20exec%20linux/2/bash-execute-action-on-find-(-exec)-for-each-file