I am writing a script which does a checksum (md5sum) on a forum web directory.
It is a bash script. With the idea being to do a checksum on all the files in the directory, and then compare it to a text file which has a list of checksums.
The script works if I pass it into a text file, and then do a diff command between the text file and my list of known checksums, but I would like to not have it write to a text file and then have to remove the text file at the end of the script, hence why I am using a variable
The script below fails with the error:
/usr/bin/diff: Argument list too long
cd /var/www/html/forum/
VAR1=$(find . -type d \( -name store_sitemap \) -prune -o -type f -exec md5sum {} \; | grep -v "files\|that\|change")
/usr/bin/diff "${VAR1}" "/root/scripts/forum_checkum_original.txt"
How can I pass my variable along so that I can runn the diff command on it?
EDIT: with the help of the user devnull (thank you again) here is the completed and working script:
cd /var/www/html/forum/
MAIL=$(/usr/bin/diff <(find . -type d \( -name store_sitemap \) -prune -o -type f -exec md5sum {} \; | grep -v "files\|that\|change") /root/scripts/forum_checkum_original.txt)
if [[ -n $(/usr/bin/diff <(find . -type d \( -name store_sitemap \) -prune -o -type f -exec md5sum {} \; | grep -v "files that change") /root/scripts/forum_checkum_original.txt) ]]; then
echo "$MAIL" | mail -s "Forum Checksum" yourmailaddress#yourdomain.com
else
echo "no files have been changed"
fi
diff compares files, not variables. Use Process Substitution instead.
An equivalent of what you're trying to do would be:
/usr/bin/diff <(find . -type d \( -name store_sitemap \) -prune -o -type f -exec md5sum {} \; | grep -v "bidorbuy.log") /root/scripts/forum_checkum_original.txt
If you want to keep it in a variable you can give diff the variable as a filedescriptor by doing:
diff <(echo "$MAIL") "/root/scripts/forum_checkum_original.txt"
Related
Am trying to write a script that finds the files that are older than 10 hours from the sub-directories that are in the "HS_client_list". And send the Output to a file "find.log".
#!/bin/bash
while IFS= read -r line; do
echo Executing cd /moveit/$line
cd /moveit/$line
#Find files less than 600 minutes old.
find $PWD -type f -iname "*.enc" -mmin +600 -execdir basename '{}' ';' | xargs ls > /home/infa91punv/find.log
done < HS_client_list
However, the script is able to cd to the folders from HS_client_list(this file contents the name of the subdirectories) but, the find command (find $PWD -type f -iname "*.enc" -mmin +600 -execdir basename '{}' ';' | xargs ls > /home/infa91punv/find.log) is not working. The Output file is empty. But when I run find $PWD -type f -iname "*.enc" -mmin +600 -execdir basename '{}' ';' | xargs ls > /home/infa91punv/find.log as a command it works and from the script it doesn't.
You are overwriting the file in each iteration.
You can use xargs to perform find on multiple directories; but you have to use an alternate delimiter to avoid having xargs populate the {} in the -execdir command.
sed 's%^%/moveit/%' HS_client_list |
xargs -I '<>' find '<>' -type f -iname "*.enc" -mmin +600 -execdir basename {} \; > /home/infa91punv/find.log
The xargs ls did not seem to perform any useful functionality, so I took it out. Generally, don't use ls in scripts.
With GNU find, you could avoid the call to an external utility, and use the -printf predicate to print just the part of the path name that you care about.
For added efficiency, you could invoke a shell to collect the arguments:
sed 's%^%/moveit/%' HS_client_list |
xargs sh -c 'find "$#" -type f -iname "*.enc" -mmin +600 -execdir basename {} \;' _ >/home/infa91punv/find.log
This will run as many directories as possible in a single find invocation.
If you want to keep your loop, the solution is to put the redirection after done. I would still factor out the cd, and take care to quote the variable interpolation.
while IFS= read -r line; do
find /moveit/"$line" -type f -iname "*.enc" -mmin +600 -execdir basename '{}' ';'
done < HS_client_list >/home/infa91punv/find.log
I need to get all the texts in the matching file in the folder. However, at the same time need to get the matching file path as well. How can I get the matching file path as well using the following command.
find . -type f -name release.txt | xargs cat
try
find . -type f -name release.txt -exec grep -il {} \; | xargs cat
Skip xargs, just do:
find . -type f -name release.txt -exec sh -c 'echo "$1"; cat "$1"' _ {} \;
I'm trying to reencode all files in a directory and put the results in a subdirectory.
I'm using
find . -type f -name '*.txt' -execdir iconv -f utf-16 -t utf-8 {} > reencoded/{} \;
But the filename does not replace the second occurrence of '{}', there is a result in reencoded/{} instead.
Wrap the command inside a call to sh -c, which can then reference the {} as $0:
find . -type f -name '*.txt' -execdir sh -c 'iconv -f utf-16 -t utf-8 "$0" > reencoded/"$0"' {} \;
I have lots of files, directories and sub-directories at my file system.
For example:
/path/to/file/test-poster.jpg
/anotherpath/my-poster.jpg
/tuxisthebest/ohyes/path/exm/bold-poster.jpg
I want to switch all file names from *-poster.jpg to folder.jpg
I have tried with sed and awk with no success.
little help?
You can do it with find:
find -name "*poster.jpg" -exec sh -c 'mv "$0" "${0%/*}/folder.jpg"' '{}' \;
Explanation
Here, for each filename matched, executes:
sh -c 'mv "$0" "${0%/*}/folder.jpg"' '{}'
Where '{}' is the filename passed as an argument to the command_string:
mv "$0" "${0%/*}/folder.jpg"
So, at the end, $0 will have the filename.
Finally, ${0%/*}/folder.jpg expands to the path of the old filename and adds /folder.jpg.
Example
Notice I'm replacing mv with echo
$ find -name "*poster.jpg" -exec sh -c 'echo "$0" "${0%/*}/folder.jpg"' '{}' \;
./anotherpath/my-poster.jpg ./anotherpath/folder.jpg
./path/to/file/test-poster.jpg ./path/to/file/folder.jpg
./tuxisthebest/ohyes/path/exm/bold-poster.jpg ./tuxisthebest/ohyes/path/exm/folder.jpg
Try this script, it should rename all the files as required.
for i in $(find . -name "*-poster.jpg") ; do folder=`echo $i | awk -F"-poster.jpg" {'print $1'}`; mv -iv $i $folder.folder.jpg; done
You can replace . to the directory where these files are placed in the command find . -name "*-poster.jpg" in the script. Let me know if it is working fine for you.
you can try it like
find -name '*poster*' -type f -exec sh -c 'mv "{}" "$(dirname "{}")"/folder.jpg' \;
find all files containing poster == find -name '*poster*' -type f
copy the directory path of the file and store it in a temporary variable and afterwards affix "folder.jpg" to directory path == -exec sh -c 'mv "{}" "$(dirname "{}")"/folder.jpg' \;
I am trying to find all the files of type *.gz and cat them to total.gz and I think I am quite close on this.
This is the command I am using to list all *.gzfiles:
find /home/downloaded/. -maxdepth 3 -type d \( ! -name . \) \
-exec bash -c "ls -ltr '{}' " \
How to modify it so that it will concatenate all of them and write to ~/total.gz
Directory structure under downloaded is as follows
/downloaded/wllogs/303/07252014/SysteOut.gz
/downloaded/wllogs/301/07252014/SystemOut_13.gz
/downloaded/wllogs/302/07252014/SystemOut_14.gz
Use cat in -exec and redirect output of find:
find /home/downloaded/ -type f -name '*.gz' -exec cat {} \; > output
Use echo in -exec and redirect the output:
find /home/downloaded/ -name "*.gz" -exec echo {} \; > output