I am attempting to list files in a folder with sub folders recursively, but trying to avoid going into one folder as they are duplicates.
This is the command I run but it doesn't do anything.
ls -lR /opt/elk/data/syslogs | grep -v .log. | grep --exclude-dir="cam" * > /tmp/logs.log
If there any changes I can make to this?
Thanks.
Options to different versions of find vary greatly, but you may try:
find /opt/elk/data/syslogs -name cam -prune -o -print
On RHEL, you probably have gnu find, and if you want file size and modification time, you might try:
find /opt/elk/data/syslogs -name cam -prune -o -printf "%p %s %t\n"
Related
I've created a script to quickly analyze some logs and automatically provide advices to solve problems based on errors found.
All works as expected.
However, it's appears that folders structure containing these logs can change (depends on system configuration) and my script not work any more.
I would like to find a way to find the directory containing a specifics files like logs or appinfo.txt file.
Once obtains I could use it as variable and finally solve my problem.
Here is an example:
AppLogDirectory ='Your_Special_Command_You_Will_HelpMe_To_Find'
grep -i "Error" $AppLogDirectory/esl*.log
Log format is: ESL.randomValue.log
Files analyzed : appinfo.txt,
system.txt etc ..
A suggested in comment section, I edit my orginal post with more detail to clarify the context, below an example:
Log files (esl.xxx.tt.ss.log ) can be in random directory, like:
/var/log/ApplicationName/logs/
/opt/ApplicationName/logs/
/var/data/ApplicationName/Extended/logs/
Because of random directory, I need to find a solution to print the directory names of the files that match esl*.log patter (without esl filename)
Use find and pass the output to xargs with grep, like so, which runs grep on multiple files and prints the output together with the file name where the pattern was found:
find /path/to/files /another/path/to/other/files \( -name 'appinfo.txt' -o -name 'system.txt' -o -name 'esl*.log' \) -print0 | xargs -0 grep -i 'Error'
Or simply use -exec ... \+, which gives the same effect, without the need for xargs:
find /path/to/files /another/path/to/other/files \( -name 'appinfo.txt' -o -name 'system.txt' -o -name 'esl*.log' -exec grep -i 'Error' \+
To find the directories which contain the files that contain the desired pattern, use grep -l to print file names only (not the lines that match), and pipe the results to xargs dirname to print the directory names. If you need the unique dir names, pipe it further to sort -u:
find /path/to/files /another/path/to/other/files \( -name 'appinfo.txt' -o -name 'system.txt' -o -name 'esl*.log' -exec grep -il 'Error' \+ | xargs dirname | sort -u
SEE ALSO:
GNU find manual
To search for files based on their contents
xargs
Solution found thanks to you thank you again!
#Ask for extracted tar.gz folder
read -p "Where did you extract the tar.gz file? r1
#directory path where esl files is located
logpath=`find $r1 -name "esl*.log" | xargs dirname | sort -u`
#Search value (here "Error") into all esl*.log
grep 'Error' $logpath/esl*.log | awk '{print $8}'
I am trying to find all the directories and files owned by user with the following command.
find / -type d -user greg | grep -v proc
it is working fine sometimes and hanging up sometimes. Is there any performance issues associated with it or is there any better way to executing this.
To keep it from descending into /proc, use -prune. It's better than filtering out entries with grep -v as it'll avoid descending into /proc at all.
find / -path /proc -prune -o -type d -user greg -print
Read -o as "or". If the path is /proc, prune it, i.e. don't go in there. Otherwise, match directories owned by greg. (If you want files too then get rid of the -type d test.)
When you use -prune, you also have to use -print to print matches. -print's normally implied, but using -prune changes that.
I am new to Stack Overflow and am somewhat of a newbie with Linux. I have been trying to filter specific files within a parent directory and it's children using the following command as an example:
ls -R | grep '*.jpg' | wc -l
Which I have found great when looking for individual files but I will need to do this on a monthly basis and looking for quicker ways to list several types in one command. I purposely want to exclude hidden files.
I have tried this but to no avail —
Count number of specific file type of a directory and its sub dir in mac
I've seen different methods across the web from list, find, tree, echo etc. so any help with this would be much appreciated and if there is a better way of doing this than what I am currently doing then that's not a problem as I am open to suggestions. I'm just not sure what's the best way to skin this cat at the moment!
Thank you very much
you can do this with help of find as it was mentioned under the link from your initial post.
Just something like this:
find . -name \*.jpg -or -name \*.png -not -path \*/\.\* | wc -l
If you arrive here looking for more of a summary, here's a way to count all file extensions recursively in a folder:
find . -type f -name '*.*' -not -name '.*' | sed -Ee 's,.*/.+\.([^/]+)$,\1,' | sort | uniq -ci | sort -n
This gives a summary like:
422 mov
1043 mp4
3266 png
6738 CR3
9417 RAF
29679 cr2
60949 jpg
You can have grep filter for more than one pattern. You should learn about manpages in linux, just type man grep in a terminal and you will see of what this program is capable of and how.
For your issue, you could e.g. use this to filter for png and jpeg files (ingoring case-sensitivity, thus getting PNG and png files):
ls -R | grep -i '*.jpg\|*.png' | wc -l
the -i will ignore the case of the names, the \| is an or-concatenation.
Thank you all for contributing, in case this proves to be useful to someone out there, I've had some help from a developer friend who's kindly looked into it for me and what I've found that works best in my particular case is the following:
find . -type f \( -iname "*.jpg" ! -iname ".*.png" ! -path "*/.HSResource/*" \) |wc -l
This skips over the resource folders and hidden files and appears to return me the correct results.
I am successfully using find to create a list of all files in the current subdirectory, excluding those in the subdirectory "cache." Here's my first bit of code:
find . -wholename './cach*' -prune -o -print
I now wish to pipe this into a grep command. It seems like that should be simple:
find . -wholename './cach*' -prune -o -print | xargs grep -r -R -i "samson"
... but this is returning results that are mostly from the cache directory. I've tried removing the xargs reference, but that does what you'd expect, running the grep on text of the file names, rather than on the files themselves. My goal is to find "samson" in any files that aren't cached content.
I'll probably get around this issue by just using doubled greps in this instance, but I'm very curious about why this one-liner behaves this way. I'd love to hear thoughts on a way to modify it while still using these two commands (as there are speed advantages to doing it this way).
(This is in CentOS 5, btw.)
The wholename match may be the reason why it's still including "cache" files. If you're executing the find command in the directory that contains the "cache" folder, it should work. If not, try changing it to -name '*cache*' instead.
Also, you do not need the -r or -R for your grep, that tells it to recurse through directories - but you're testing individual files.
You can update your command using the piped version, or a single-command:
find . -name '*cache*' -prune -o -print0 | xargs -0 grep -il "samson"
or
find . -name '*cache*' -prune -o -exec grep -iq "samson" {} \; -print
Note, the -l in the first command tells grep to "list the file" and not the line(s) that match. The -q in the second does the same; it tells grep to respond quietly so find will then just print the filename.
You've told grep itself to recurse (twice! -r and -R are synonyms). Since one of the arguments you're passing is . (the top directory), grep is searching in every file (some of them twice, or even more if they're in subdirectories).
If you're going to use find and grep, do this:
find . -path './cach*' -prune -o -print0 | xargs -0 grep -i "samson"
Using -print0 and -0 makes your script work even with file names that contain spaces or punctuation characters.
However, you probably don't need to bother with find here, since GNU grep is capable of excluding directories:
grep -R --exclude-dir='cach*' -i "samson" .
(This also excludes ./deeply/nested/directory/cache. If you only want to exclude cache directories at the toplevel, use find as you did.)
Use the -exec option on find instead of piping them to another command. From there you can use grep "samson" {} \; to look for samson in each file listed.
For example:
find . -wholename './cach*' -prune -o -exec grep "samson" "{}" +
I'd like to construct a Linux command to list all files (with their full paths) within a specific directory (and subdirectories) ordered by access time.
ls can order by access time, but doesn't give the full path. find gives the full path, but the only control you have over the access time is to specify a range with -atime N (accessed at least 24*N hours ago), which isn't what I want.
Is there a way to order by access time and get the full path at once? I could just write a script, but it seems there should be a way to do this with the standard Linux programs.
find . -type f -exec ls -l {} \; 2> /dev/null | sort -t' ' -k +6,6 -k +7,7
This will find all files, and sort them by date and then time. You can then use awk or cut to extract the dates and files name from the ls -l output
you could try:
ls -l $(find /foo/bar -type f )
you can add other options (e.g. -t for sorting) to ls command to achieve your goal.
also you could add your searching criteria to find cmd
find . -type f | xargs ls -ldt should do the trick as long as there's not so many files that you hit the command like argument limit and spawn 2 instances of ls.
pwd | xargs -I % find % -type f
find . -type f -exec ls -l --full-time {} \; 2> /dev/null | sort -t' ' -k +6,6 -k +7,7
Alex's answer did not work for me since I had files older than one year and the sorting got messed up. The above adds the --full-time parameter which nuetralizes the date/time values and makes them sortable regardless of how old they are.