So, I have a list of files that I want to use to generate a new set of groups of files. I want to open up these groups (multiple files) together at once. Edit them. Then go back to the terminal, hit enter, and open up the next group of files.
I've got it working, but I'm using a temporary file to do it like so
cat failingAll | awk '{print $2}' | xargs -I {} -L 1 sh -c "find {} | grep xml | xargs echo;" | perl -pe "s/^(.*)$/open \1;read;/g" > command.sh ; sh command.sh
Is this possible to do with just the xargs? Really, I mean without a temporary file. I tried this
cat failingAll | awk '{print $2}' | xargs -I {} -L 1 sh -c "find {} | grep xml | xargs open ; read;"
but it does not pause in between the groups, it just opens them all at once (and crashes my xml editor.)
Try something like this and just quit the editor when you finish editing each group of files?
while IFS= read -r dir; do
files=()
while IFS= read -r -d '' xmlfile; do
files+=("$xmlfile")
done < <(find "$dir" -name "*.xml" -type f -print0)
open -W "${files[#]}"
done < <(awk '{print $2}' failingAll)
Assuming you don't want to have to quit the editor and assuming it can take a new set of files via a subsequent call to open then the following might also work:
while IFS= read -u 3 -r dir; do
files=()
while IFS= read -r -d '' xmlfile; do
files+=("$xmlfile")
done < <(find "$dir" -name "*.xml" -type f -print0)
open "${files[#]}"
read
done 3< <(awk '{print $2}' failingAll)
In light of Etan Reisner's answer, I found this also works
cat failingAll | awk '{print $2}' | xargs -I {} -L 1 sh -c "find {} | grep xml | xargs open -W"
But it doesn't quite do what I wanted either which is wait for an enter. I believe its because xargs keeps feeding it the next line which breaks the read command.
I'm trying to get the most recently modified file's (datetime - as a unixtimestamp) from a folder structure. There are many files but I only need the datetime of the most recently updated.
I'ved tried the following but I think I'm way of the mark:
stat --printf="%y %n\n" $(ls -tr $(find * -type f))
Try this:
ls -trF | grep -v '\/\|#' | tail -1 | xargs -i date +%s -r {}
ls -trF gives you symbols to filter out, '/' for directories and '#' for links. After that, grep out those files, pick the last one, and pass it to date command.
EDIT: Of note as well is the date -r option, which will display the last modified date of file given as argument.
something like this?
ls -ltr | tail -n1 | awk '{print "date -d\"" $6FS$7FS$8 "\" +%s"}' | sh
EDIT:
actually better yet,try the following
find -type f -exec ls -l --time-style +%s {} \+ | sort -n -k6 | tail -n1
this will iterate over the folder structure you desired, print the time as a unix timestamp and sort it so the newest is at the end. (hence tail -n1)
I am able to list all the directories by
find ./ -type d
I attempted to list the contents of each directory and count the number of files in each directory by using the following command
find ./ -type d | xargs ls -l | wc -l
But this summed the total number of lines returned by
find ./ -type d | xargs ls -l
Is there a way I can count the number of files in each directory?
This prints the file count per directory for the current directory level:
du -a | cut -d/ -f2 | sort | uniq -c | sort -nr
Assuming you have GNU find, let it find the directories and let bash do the rest:
find . -type d -print0 | while read -d '' -r dir; do
files=("$dir"/*)
printf "%5d files in directory %s\n" "${#files[#]}" "$dir"
done
find . -type f | cut -d/ -f2 | sort | uniq -c
find . -type f to find all items of the type file, in current folder and subfolders
cut -d/ -f2 to cut out their specific folder
sort to sort the list of foldernames
uniq -c to return the number of times each foldername has been counted
You could arrange to find all the files, remove the file names, leaving you a line containing just the directory name for each file, and then count the number of times each directory appears:
find . -type f |
sed 's%/[^/]*$%%' |
sort |
uniq -c
The only gotcha in this is if you have any file names or directory names containing a newline character, which is fairly unlikely. If you really have to worry about newlines in file names or directory names, I suggest you find them, and fix them so they don't contain newlines (and quietly persuade the guilty party of the error of their ways).
If you're interested in the count of the files in each sub-directory of the current directory, counting any files in any sub-directories along with the files in the immediate sub-directory, then I'd adapt the sed command to print only the top-level directory:
find . -type f |
sed -e 's%^\(\./[^/]*/\).*$%\1%' -e 's%^\.\/[^/]*$%./%' |
sort |
uniq -c
The first pattern captures the start of the name, the dot, the slash, the name up to the next slash and the slash, and replaces the line with just the first part, so:
./dir1/dir2/file1
is replaced by
./dir1/
The second replace captures the files directly in the current directory; they don't have a slash at the end, and those are replace by ./. The sort and count then works on just the number of names.
Here's one way to do it, but probably not the most efficient.
find -type d -print0 | xargs -0 -n1 bash -c 'echo -n "$1:"; ls -1 "$1" | wc -l' --
Gives output like this, with directory name followed by count of entries in that directory. Note that the output count will also include directory entries which may not be what you want.
./c/fa/l:0
./a:4
./a/c:0
./a/a:1
./a/a/b:0
Slightly modified version of Sebastian's answer using find instead of du (to exclude file-size-related overhead that du has to perform and that is never used):
find ./ -mindepth 2 -type f | cut -d/ -f2 | sort | uniq -c | sort -nr
-mindepth 2 parameter is used to exclude files in current directory. If you remove it, you'll see a bunch of lines like the following:
234 dir1
123 dir2
1 file1
1 file2
1 file3
...
1 fileN
(much like the du-based variant does)
If you do need to count the files in current directory as well, use this enhanced version:
{ find ./ -mindepth 2 -type f | cut -d/ -f2 | sort && find ./ -maxdepth 1 -type f | cut -d/ -f1; } | uniq -c | sort -nr
The output will be like the following:
234 dir1
123 dir2
42 .
Everyone else's solution has one drawback or another.
find -type d -readable -exec sh -c 'printf "%s " "$1"; ls -1UA "$1" | wc -l' sh {} ';'
Explanation:
-type d: we're interested in directories.
-readable: We only want them if it's possible to list the files in them. Note that find will still emit an error when it tries to search for more directories in them, but this prevents calling -exec for them.
-exec sh -c BLAH sh {} ';': for each directory, run this script fragment, with $0 set to sh and $1 set to the filename.
printf "%s " "$1": portably and minimally print the directory name, followed by only a space, not a newline.
ls -1UA: list the files, one per line, in directory order (to avoid stalling the pipe), excluding only the special directories . and ..
wc -l: count the lines
This can also be done with looping over ls instead of find
for f in */; do echo "$f -> $(ls $f | wc -l)"; done
Explanation:
for f in */; - loop over all directories
do echo "$f -> - print out each directory name
$(ls $f | wc -l) - call ls for this directory and count lines
This should return the directory name followed by the number of files in the directory.
findfiles() {
echo "$1" $(find "$1" -maxdepth 1 -type f | wc -l)
}
export -f findfiles
find ./ -type d -exec bash -c 'findfiles "$0"' {} \;
Example output:
./ 6
./foo 1
./foo/bar 2
./foo/bar/bazzz 0
./foo/bar/baz 4
./src 4
The export -f is required because the -exec argument of find does not allow executing a bash function unless you invoke bash explicitly, and you need to export the function defined in the current scope to the new shell explicitly.
My answer is a little different, due to the options of find, you can actually be much more flexible. Just try:
find . -type f -printf "%h\n" | sort | uniq -c
With the "%h" option to "-printf", find prints only the directory of the files it found. Then sort and count with "uniq -c". This prints the number of search result entries with the same directory, per directory.
Using further options on find, you can be much more flexible. For example, to get an overview how many files in which directory have been modified at a certain date, use:
find . -newermt "2022-01-01 00:00:00" -type f -printf "%TY-%Tm-%Td %h\n" | sort | uniq -c
This finds all files that have been modified since 1. January 2022, prints (with "-printf") the modification date and the directory, then sorts and counts them. In this example, each line in the result has the number of files, the date of modification (without time), and the directory.
Note that "-printf" may not be available in all versions of find I think.
I combined #glenn jackman's answer and #pcarvalho's answer(in comment list, there is something wrong with pcarvalho's answer because the extra style control function of character '`'(backtick)).
My script can accept path as an augument and sort the directory list as ls -l, also it can handles the problem of "space in file name".
#!/bin/bash
OLD_IFS="$IFS"
IFS=$'\n'
for dir in $(find $1 -maxdepth 1 -type d | sort);
do
files=("$dir"/*)
printf "%5d,%s\n" "${#files[#]}" "$dir"
done
FS="$OLD_IFS"
My first answer in stackoverflow, and I hope it can help someone ^_^
THis could be another way to browse through the directory structures and provide depth results.
find . -type d | awk '{print "echo -n \""$0" \";ls -l "$0" | grep -v total | wc -l" }' | sh
find . -type f -printf '%h\n' | sort | uniq -c
gives for example:
5 .
4 ./aln
5 ./aln/iq
4 ./bs
4 ./ft
6 ./hot
I tried with some of the others here but ended up with subfolders included in the file count when I only wanted the files. This prints ./folder/path<tab>nnn with the number of files, not including subfolders, for each subfolder in the current folder.
for d in `find . -type d -print`
do
echo -e "$d\t$(find $d -maxdepth 1 -type f -print | wc -l)"
done
This will give the overall count.
for file in */; do echo "$file -> $(ls $file | wc -l)"; done | cut -d ' ' -f 3| py --ji -l 'numpy.sum(l)'
A super fast miracle command, which recursively traverses files to count the number of images in a directory and organize the output by image extension:
find . -type f | sed -e 's/.*\.//' | sort | uniq -c | sort -n | grep -Ei '(tiff|bmp|jpeg|jpg|png|gif)$'
Credits: https://unix.stackexchange.com/a/386135/354980
I edited the script in order to exclude all node_modules directories inside the analyzed one.
This can be used to check if the project number of files is exceeding the maximum number that the file watcher can handle.
find . -type d ! -path "*node_modules*" -print0 | while read -d '' -r dir; do
files=("$dir"/*)
printf "%5d files in directory %s\n" "${#files[#]}" "$dir"
done
To check the maximum files that your system can watch:
cat /proc/sys/fs/inotify/max_user_watches
node_modules folder should be added to your IDE/editor excluded paths in slow systems, and the other files count shouldn't ideally exceed the maximum (which can be changed though).
Easy Method:
find ./|grep "Search_file.txt" |cut -d"/" -f2|sort |uniq -c
In my case I needed the count at subfolder level, so I did:
du -a | cut -d/ -f3 | sort | uniq -c | sort -nr
Easy way to recursively find files of a given type. In this case, .jpg files for all folders in current directory:
find . -name *.jpg -print | wc -l
omg why the complex commands. just use something like
find whatever_folder | wc -l
This question already has answers here:
How to pass command output as multiple arguments to another command
(5 answers)
Closed 6 years ago.
I'm taking a unix/linux class and we have yet to learn variables or functions. We just learned some basic utilities like the flag and pipeline, output and append to file. On the lab assignment he wants us to find the largest files and copy them to a directory.
I can get the 5 largest files but I don't know how to pass them into cp in one command
ls -SF | grep -v / | head -5 | cp ? Directory
It would be:
cp `ls -SF | grep -v / | head -5` Directory
assuming that the pipeline is correct. The backticks substitute in the line the output of the commands inside it.
You can also make your tests:
cp `echo a b c` Directory
will copy all a, b, and c into Directory.
I would do:
cp $(ls -SF | grep -v / | head -5) Directory
xargs would probably be the best answer though.
ls -SF | grep -v / | head -5 | xargs -I{} cp "{}" Directory
Use backticks `like this` or the dollar sign $(like this) to perform command substitution. Basically this pastes each line of standard ouput of the backticked command into the surrounding command and runs it. Find out more in the bash manpage under "Command Substitution."
Also, if you want to read one line at a time you can read individual lines out of a pipe stream using "while read" syntax:
ls | while read varname; do echo $varname; done
If your cp has a "-t" flag (check the man page), that simplifies matters a bit:
ls -SF | grep -v / | head -5 | xargs cp -t DIRECTORY
The find command gives you more fine-grained ability to get what you want, instead of ls | grep that you have. I'd code your question like this:
find . -maxdepth 1 -type f -printf "%p\t%s\n" |
sort -t $'\t' -k2 -nr |
head -n 5 |
cut -f 1 |
xargs echo cp -t DIRECTORY
I am new to Linux shell and I found a way to get the name of the file I want:
ls *.*g -S| grep -v ^d | head -1
I am going to be repeating this for a number of file. I am trying to copy this file to another directory (cp command?). But the below code is failing.
I am trying this, but its not working:
cp ls -S| grep -v ^d | head -1 ../directory
Also, I was wondering how to loop through directorys that are in a particular directory.
Any help is much appreciated.
Thanks,
Bryan
cp $(ls *.*g -S| grep -v ^d | head -1) ../directory
cp "$(find . -type f -name "*.*" -printf "%f:%s\n" | sort -t":" -k2 -n | awk -F":" 'END{print $1}')" ../directory
Use quotes in case there are white spaces in your file. add -maxdepth 1 if you don't want to recurse subdirectories