I have a simple problem with binary operator but I can't resolve it. Can anyone help me why this shell script not work:
set -o nounset -o pipefail -o errexit
if [ -e /root/mom/*.php ]; then
find /root/mom/*.php -exec gpg --clearsign {} \;
else
echo "Hello world"
fi
If you want to do something for all php files in a directory, just use find:
find /root/mom -name "*.php" -exec gpg --clearsign {} \;
Note that it takes a list of directories to search in, not a list of plain files. There's no need to try to see if files exist before using it; it's not an error if they don't.
Related
Variable causing issue while doing the test command in unix Command is::
find $PWD -type d -exec sh -c 'test "{}" ">" "$PWD/$VersionFolders"' \; -print|wc -l`
Input Values-
Here $PWD- Current Directory
b1_v.1.0
b1_v.1.2
b1_v.1.3
b1_v.1.4
Given Version folder as $VersionFolders
b1_v.1.2
The Command should check if any folders exist in current directory which is greater than the give version folder and it should count or display.
This approach has to be consider with out date or time created of folders.
Expected Output-
b1_v.1.3
b1_v.1.4
If I give hard code Directories its working fine. But when I pass it like as variable.it give all folders.
working fine this commend-
find $PWD -type d -exec sh -c 'test "{}" ">" "$PWD/b1_v.1.2"' \; -print|wc -l`
Not working this command with variable-
find $PWD -type d -exec sh -c 'test "{}" ">" "$PWD/$VersionFolders"' ; -print|wc -l`
The variable $VersionFolders won't get expanded inside the single quotes, and apparently you did not export it to make it visible to subprocesses.
An obscure but common hack is to put it in $0 (which nominally should contain the name of the shell itself, and is the first argument after sh -c '...') because that keeps the code simple.
find . -type d \
-exec sh -c 'test "$1" ">" "$0"' \
"$VersionFolders" {} \; \
-print | wc -l
But as #chepner remarks, you can run -exec test {} ">" "$VersionFolders" \; directly.
The shell already does everything in the current directory, so you don't need to spell out $PWD. Perhaps see also What exactly is current working directory?
I would like to preface this with I am a complete noob with scripting. So I have a situation where I need to manually look for a phone number that could live in one of hundreds of files.
so the logs live in the following directory.
/actlogs/sbclogger_archive
The logs file names are in directories numbered 01-31 inside of that directory and all the files are zipped.
Inside of those numbered directories are tons of files but the only ones I want to search are "sipd.logthenthedate.gz" and "sipmsg.logthenthedate.gz".
So I need to look in all the files in the following directory.
"/actlogs/sbclogger_archive"
Which has 31 directories labeled "01-31"
Then in each 01-31 there is hundreds of files the only ones I want to look are are "sipd.logthenthedate.gz" and "sipmsg.logthenthedate.gz".
The script I am using is below, please let me know what I could do to make this work.
#!/bin/bash
read -p "Enter a phone number: " text
read -p "Enter directory of log file's, Hint it should be /actlogs/sbclogger_archive: " directory
#arr=( $(find $directory -type f -exec grep -l "$text" {} \; | sort -r) )
#find $directory -type f -exec grep -qe "$text" {} \; -exec bash -c '
file=$(find $directory -type f -name 'sipd.log*' -exec grep -qe "$text" {} \; -exec bash -c 'select f; do echo $f; break; done' find-sh {} +;)
if [ -z "$file" ]; then
echo "No matches found."
else
echo "select tool:"
tools=("nano" "less" "vim" "quit")
select tool in "${tools[#]}"
do
case $tool in
"quit")
break
;;
*)
$tool $file
break
;;
esac
done
fi
This would give you the list of files matching:
find \( -name 'sipd.log[0-9]*.gz' -o -name 'sipmsg.log[0-9]*.gz' \) \
-exec sh -c 'gunzip -c {}| grep -m1 -q 888333' \; -print
./18/sipd.log20200118.gz
./7/sipd.log20200107.gz
Note: -m1 tells grep to stop after first match, since you need only the file name in this case, it's enough.
If you have zgrep, you can shorten it to:
find \( -name 'sipd.log[0-9]*.gz' -o -name 'sipmsg.log[0-9]*.gz' \) \
-exec zgrep -l '888333' {} \;
./18/sipd.log20200118.gz
./7/sipd.log20200107.gz
Also, some of the tools you are suggesting do not support gzip files (nano and some variants of less for example). In which case you might need to decompress the file and compress it again when done.
And, you might want to consider a loop if you want to "quit". Feeding the file list to the tool doesn't make sense.
Note: AFAIK zgrep doesn't do recursive:
DESCRIPTION
Zgrep invokes grep on compressed or gzipped files. These grep options will cause zgrep to terminate with an
error code:
(-[drRzZ]|--di*|--exc*|--inc*|--rec*|--nu*). All other options specified are passed directly to grep. If no file is specified, then
the
standard input is decompressed if necessary and fed to grep. Otherwise the given files are uncompressed if necessary and fed to
grep.
so zgrep -rl "$text" "$directory" or zgrep -rl --include 'simpd.log*.gz' "$test" {01..31} won't work except if you have a special zgrep
As you must unzip before using your tool, i would divide the problem in two blocks.
Firstly, i would expand the paths you need (looking under <directory> for the phone <text>), and then iterate to apply the tool (because some tools like vim or nano cannot be piped).
Try something like this:
#!/bin/bash
#...
# text/directory input stuff
#...
tmpdir=$(mktemp -d)
trap 'rm -rf ${tmpdir}' EXIT
while IFS= read -r file; do
unzipped=${tmpdir}/$(basename "${file}" .gz)
gunzip -c "${file}" > "${unzipped}"
${tool} "${unzipped}"
done < <(zgrep -lw "${text}" "${directory}"/{01..31}/{sipd.logthenthedate.gz,sipmsg.logthenthedate.gz} 2>/dev/null)
Above is the proposed invert-form by Charles Duffy following this Bash FAQ.
If you prefer to iterate an array, you could build in this way:
# shellcheck disable=SC2207
files=( $(zgrep -lw "${text}" "${directory}"/{01..31}/{sipd.logthenthedate.gz,sipmsg.logthenthedate.gz} 2>/dev/null) )
for file in "${files[#]}"; do
# etc.
as in our particular case, the files to match have no spaces in their names and shellcheck warning is not so important (hidden above).
BRs
I wrote a find command, which finds the files, but excludes other files/directories. I did echo this code and copied it. If I paste it in the terminal, it works. Some files were excluded. But if I execute it from the script, it does not work as expected.
I tried to escape my variables, quote them, between brackets like $() or ${} but nothing worked.
My find code looks like this:
find ${StartDirs[*]} $pat -print
In fact it will be executed like:
find ./bin -wholename './bin/backup2' -prune -o -wholename './bin/backup3' -prune -o -print
The second code above works in the terminal but not in the script.
What did I do wrong?
For more info I will try to paste necessary code below
I am trying to make a backup and want to do that with find and cp. Most code of my script are omitted. I think the code below is the necessary minimal code for this problem.
StartDirs=();
ExcludedFiles=(); #files/directories which needs to be excluded
#check and store excluded files
CheckExcludedFile(){ #this function will be called over and over again by another function (getopts). It depends on the chain of -x options. With the -x option I can decide which file i want to exclude. The Getopts function is also omitted.
exclFile=`find $1 2>/dev/null | wc -l`
if [ $exclFile -lt 1 ]; then
echo $FILEMAPNOTEXIST | sed 's~-~'$1'~g' # FILEMAPNOTEXIST is a variable from another script with error messages
exit 0
else
ExcludedFiles+=($1) #add excluded file/dir path to array
fi
}
MakeBackup(){
for i in ${ExcludedFiles[*]}
do
s=" -wholename $i -prune -o"
pat=$pat$s
done
# the code above edits the array elements of the EcludedFIles[]
#For example by calling the script with the -x option (-x fileA -x fileB -x fileC) wordt als volgt: -wholename 'fileA' -prune -o -wholename 'fileB' -prune -o -wholename 'fileC' -prune -o.
#the pat variable will be used for the find command to ignore files/directories
mkdir -p ~/var
echo "Start-time $(date '+%F %T')" >> ~/var/dq.log
find ./bin -wholename './bin/backup2' -prune -o -wholename './bin/backup3' -prune -o -print
#the code above should work like in terminal. That is not the case..
# find ${StartDirs[*]} $pat -print #this should work also.
# cp -av ${StartDirs[#]} $Destination >> ~/var/dq.log find command not working therefore this rule is commented
echo "end-time $(date '+%F %T')" >> ~/var/dq.log
}
The expected result should simply be some files/directories being excluded if given.
If a full script is necessary, let me know.
The command find ./bin -wholename './bin/backup2' -prune -o -wholename './bin/backup3' -prune -o -print should work as intended, provided the current directory is directly above bin/. This may be the cause of your problems: If in the real script you assemble path names which do not match the prefixes in the found paths then e.g. the prune will not work. Example: You have a dir /home/me; in it is bin/backup2/, bin/backup3/ and stuff-to-backup/. Now if you are in /home/me and execute find . it finds e.g. ./bin/backup2 which will be pruned.
But if you put this in a script and call the script with path arguments, e.g. /home/me, it will find the same files but the paths will be different, e.g. /home/me/bin/backup2, and will not prune it because it does not match the supplied exclude pattern, even though they are the same files. Likewise no patterns supplied with -wholename will be found. Here is a question which addresses this problem.
I was helped out today with a command, but it doesn't seem to be working. This is the command:
find /home/me/download/ -type f -name "*.rm" -exec ffmpeg -i {} -sameq {}.mp3 && rm {}\;
The shell returns
find: missing argument to `-exec'
What I am basically trying to do is go through a directory recursively (if it has other directories) and run the ffmpeg command on the .rm file types and convert them to .mp3 file types. Once this is done, remove the .rm file that has just been converted.
A -exec command must be terminated with a ; (so you usually need to type \; or ';' to avoid interpretion by the shell) or a +. The difference is that with ;, the command is called once per file, with +, it is called just as few times as possible (usually once, but there is a maximum length for a command line, so it might be split up) with all filenames. See this example:
$ cat /tmp/echoargs
#!/bin/sh
echo $1 - $2 - $3
$ find /tmp/foo -exec /tmp/echoargs {} \;
/tmp/foo - -
/tmp/foo/one - -
/tmp/foo/two - -
$ find /tmp/foo -exec /tmp/echoargs {} +
/tmp/foo - /tmp/foo/one - /tmp/foo/two
Your command has two errors:
First, you use {};, but the ; must be a parameter of its own.
Second, the command ends at the &&. You specified “run find, and if that was successful, remove the file named {};.“. If you want to use shell stuff in the -exec command, you need to explicitly run it in a shell, such as -exec sh -c 'ffmpeg ... && rm'.
However you should not add the {} inside the bash command, it will produce problems when there are special characters. Instead, you can pass additional parameters to the shell after -c command_string (see man sh):
$ ls
$(echo damn.)
$ find * -exec sh -c 'echo "{}"' \;
damn.
$ find * -exec sh -c 'echo "$1"' - {} \;
$(echo damn.)
You see the $ thing is evaluated by the shell in the first example. Imagine there was a file called $(rm -rf /) :-)
(Side note: The - is not needed, but the first variable after the command is assigned to the variable $0, which is a special variable normally containing the name of the program being run and setting that to a parameter is a little unclean, though it won't cause any harm here probably, so we set that to just - and start with $1.)
So your command could be something like
find -exec bash -c 'ffmpeg -i "$1" -sameq "$1".mp3 && rm "$1".mp3' - {} \;
But there is a better way. find supports and and or, so you may do stuff like find -name foo -or -name bar. But that also works with -exec, which evaluates to true if the command exits successfully, and to false if not. See this example:
$ ls
false true
$ find * -exec {} \; -and -print
true
It only runs the print if the command was successfully, which it did for true but not for false.
So you can use two exec statements chained with an -and, and it will only execute the latter if the former was run successfully.
Try putting a space before each \;
Works:
find . -name "*.log" -exec echo {} \;
Doesn't Work:
find . -name "*.log" -exec echo {}\;
I figured it out now. When you need to run two commands in exec in a find you need to actually have two separate execs. This finally worked for me.
find . -type f -name "*.rm" -exec ffmpeg -i {} -sameq {}.mp3 \; -exec rm {} \;
You have to put a space between {} and \;
So the command will be like:
find /home/me/download/ -type f -name "*.rm" -exec ffmpeg -i {} -sameq {}.mp3 && rm {} \;
Just for your information:
I have just tried using "find -exec" command on a Cygwin system (UNIX emulated on Windows), and there it seems that the backslash before the semicolon must be removed:
find ./ -name "blabla" -exec wc -l {} ;
For anyone else having issues when using GNU find binary in a Windows command prompt. The semicolon needs to be escaped with ^
find.exe . -name "*.rm" -exec ffmpeg -i {} -sameq {}.mp3 ^;
You need to do some escaping I think.
find /home/me/download/ -type f -name "*.rm" -exec ffmpeg -i {} \-sameq {}.mp3 \&\& rm {}\;
Just in case anyone sees a similar "missing -exec args" in Amazon Opsworks Chef bash scripts, I needed to add another backslash to escape the \;
bash 'remove_wars' do
user 'ubuntu'
cwd '/'
code <<-EOH
find /home/ubuntu/wars -type f -name "*.war" -exec rm {} \\;
EOH
ignore_failure true
end
Also, if anyone else has the "find: missing argument to -exec" this might help:
In some shells you don't need to do the escaping, i.e. you don't need the "\" in front of the ";".
find <file path> -name "myFile.*" -exec rm - f {} ;
Both {} and && will cause problems due to being expanded by the command line. I would suggest trying:
find /home/me/download/ -type f -name "*.rm" -exec ffmpeg -i \{} -sameq \{}.mp3 \; -exec rm \{} \;
In my case I needed to execute "methods" from by bash script, which does not work when using -exec bash -c, so I add another solution I found here, as well:
UploadFile() {
curl ... -F "file=$1"
}
find . | while read file;
do
UploadFile "$file"
done
This thread pops up first when searching for solutions to execute commands for each file from find, so I hope it's okay that this solution does not use the -exec argument
I got the same error when I left a blank space after the ending ; of an -exec command.So, remove blank space after ;
If you are still getting "find: missing argument to -exec" try wrapping the execute argument in quotes.
find <file path> -type f -exec "chmod 664 {} \;"
Good day,
I've found an easy way to find files that have certain content, but I would like to create a bash script to do it quickier,
The script is:
#!/bin/bash
DIRECTORY=$(cd `dirname .` && pwd)
ARGUMENTS="'$#'"
echo find: $ARGUMENTS on $DIRECTORY
find $DIRECTORY -iname '*' | xargs grep $ARGUMENTS -sl
So if I write:
$ script.sh text
It should find in that directory files that contains 'text'
But when I execute this script it always fails, but the echo command shows exactly what I need, what's wrong with this script?
Thank you!
Luis
References: http://www.liamdelahunty.com/tips/linux_find_string_files.php
There are problems with quoting that will break in this script if either the current directory or the search pattern contains a space. The following is more simply, and fixes both issues:
find . -maxdepth 1 -type f -exec grep "$#" {} +
With the proper quoting of $#, you can even pass options to grep, such as -i.
./script -i "some text"
Try this version, with the following changes:
1.Use $1 instead of $# unless you intend to run multiple find/grep to search for multiple patterns.
2.Use find $DIR -type f to find all files instead of find $DIR -iname '*'
3.Avoid piping by using the -exec command line option of find.
4.Do not single quote the command line arguments to your script, this was the main problem with the version you had. Your grep string had escaped single quotes \'search_string\'
#!/bin/bash
DIRECTORY=$(cd `dirname .` && pwd)
ARGUMENTS="$1"
echo find: $ARGUMENTS on $DIRECTORY
find $DIRECTORY . -type f -exec grep -sl "$ARGUMENTS" {} \;
There is no point extracting all the command line arguments and passing it to grep. If you want to search for a string with spaces, pass the string within single quotes from the command line as follows:
/home/user/bin/test-find.sh 'i need to search this'
Why not just run the following?:
grep -R text .