I am trying to grep a particular pattern from a group of files stored in a directory thru shell script. However script scans through the files but it is not fetching me the result. I have 5 files with the pattern as finishing with status COMPLETE
Here is the code,could you please help on the issue
#!/bin/bash
dt=$(date --d='7 day ago' +'%Y%m%d')
countT=$("find /home/arun/file_status -type f -name "product_feed_*stock*_${dt}*.out" -exec grep "finishing with status COMPLETE" {} \;" | wc -l)
echo $countT
if [ $countT -eq 5 ]
then
echo "Hello"
else
echo "Hai"
fi
The below is the error:
find /home/arun/file_status -type f -name product_feed_stock_20170504*.out -exec grep finishing: No such file or directory 0 Hai
Need to remove quotes in find command.
#!/bin/bash
dt=$(date --d='7 day ago' +'%Y%m%d')
countT=$(find /home/arun/file_status -type f -name "product_feed_stock_${dt}*.out" -exec grep "finishing with status COMPLETE" {} \;| wc -l)
echo $countT
if [ $countT -eq 5 ]
then
echo "Hello"
else
echo "Hai"
fi
Related
I have a bash script that I want to list all my files older than x hours. But I have some errors. What would be the correct syntax?!
find: missing argument to-exec'
./script.sh: line 9: [[0: command not found [root#localhost home]#
prev_count=0
path=/home/alex/
find $path -type f -mmin +2 -exec ls -ltrh\; > /home/test.txt
count=$(cat /home/test.txt | wc -l)
if ["$prev_count" -lt "$count"]
then
echo "This are the files that need to be deleted" >> $MESSAGE
echo "+----------------------------------------+" >> $MESSAGE
echo "" >> $MESSAGE
fi```
I have created a script to zip and move log files from one directory to another directory to free space. This is the script:
#!/bin/bash
logsDirectory="/test//logs/"
email=""
backupDirectory="/test/backup"
pid="/data/test/scripts/backup.pid"
usage=$(df | grep /data/logs | awk '{ print $2 }')
space=450000000
getBackup ()
{
if [[ ! -e $pid ]] then
if [[ $usage -le $space ]]
then
touch $pid
find $backupDirectory -mtime +15 -type f -delete;
for i in $(find $logsDirectory -type f -not -path "*/irws/*")
do
/sbin/fuser $i > /dev/null 2>&1
if [ $? -ne 0 ]
then
gzip $i
mv -v $i.gz $backupDirectory
else
continue
fi
done
[[ ! -z $email ]] && echo "Backup is ready" | mas"Backup" $email
rm -f $pid
fi
fi
}
getBackup
I am getting this error:
gzip: /data/logs/log01.log.gz already has .gz suffix -- unchanged
mv: cannot stat `/data/logs/log01.log.gz': No such file or directory
I got the error every time I ran the script in my DEV and PROD (CentOS servers) environments. To analyse it, I ran the same script in a VM (Ubuntu) in my laptop, and I don't get the error there.
My questions:
How can I prevent this error?
What I have done wrong in the script?
Your script contains a number of common clumsy or inefficient antipatterns. Here is a refactoring. The only real change is skipping any *.gz files.
#!/bin/bash
logsDirectory="/test//logs/"
email=""
backupDirectory="/test/backup"
pid="/data/test/scripts/backup.pid"
# Avoid useless use of grep -- awk knows how to match a regex
# Better still run df /data/logs
usage=$(df /data/logs/ | awk '{ print $2 }')
space=450000000
getBackup ()
{
# Quote variables
if [[ ! -e "$pid" ]]; then
if [[ "$usage" -le "$space" ]]; then
touch "$pid"
find "$backupDirectory" -mtime +15 -type f -delete;
# Exclude *.gz files
# This is still not robust against file names with spaces or wildcards in their names
for i in $(find "$logsDirectory" -type f -not -path "*/irws/*" -not -name '*.gz')
do
# Avoid useless use of $?
if /sbin/fuser "$i" > /dev/null 2>&1
then
gzip "$i"
mv -v "$i.gz" "$backupDirectory"
# no need for do-nothing else
fi
done
[[ ! -z "$email" ]] &&
echo "Backup is ready" | mas"Backup" "$email"
rm -f "$pid"
fi
fi
}
getBackup
With a slightly more intrusive refactoring, the proper fix to the find loop would perhaps look something like
find "$logsDirectory" -type f \
-not -path "*/irws/*" -not -name '*.gz' \
-exec sh -c '
for i; do
if /sbin/fuser "$i" > /dev/null 2>&1
then
gzip "$i"
mv -v "$i.gz" "$backupDirectory"
fi
done' _ {} +
where the secret sauce is to have find ... -exec + pass in the arguments to the sh -c script in a way which does not involve exposing the arguments to the current shell at all.
What I have done wrong in the script?
Your script tries to zip every file but the gzip command is rejecting files already zipped
How can I prevent this error?
Have the script check whether the file is zipped or not and only gzip if it corresponds (1). Alternatively, you could force re-compression even if it is already compressed (2).
Going with option number 1):
getBackup ()
{
if [[ ! -e $pid ]] then
if [[ $usage -le $space ]]
then
touch $pid
find $backupDirectory -mtime +15 -type f -delete;
for i in $(find $logsDirectory -type f -not -path "*/irws/*")
do
/sbin/fuser $i > /dev/null 2>&1
if [ $? -ne 0 ]
then
if [[ $i =~ \.gz$ ]]
# File is already zipped
mv -v $i $backupDirectory
else
gzip $i
mv -v $i.gz $backupDirectory
fi
else
continue
fi
done
[[ ! -z $email ]] && echo "Backup is ready" | mas"Backup" $email
rm -f $pid
fi
fi
}
I want to use a find command in order to obtain files older than 8640 min and send the result in a email body. I used this script that makes use of a file - ATTACH_FILE - containing the results of the find command:
#!/bin/sh
ATTACH_FILE="/pub/email_attach.txt"
WORK_DIR="/pub/"
rm -f $ATTACH_FILE
find $WORK_DIR -maxdepth 1 -name '*x.rsd' -type f -daystart -mmin +8640 -exec echo {} >> $ATTACH_FILE \;
if [ ! -z $ATTACH_FILE ]; then
FILESIZE=$(stat -c%s "$ATTACH_FILE" 2>> getLatestErr.log)
echo $ATTACH_FILE "size $FILESIZE bytes"
if [ $FILESIZE -gt 0 ]; then
cat $ATTACH_FILE | mail -s "Test "$TODAY mmm#server.com
fi
fi
How can I get the same result by putting a message in the body of the email without using the auxiliary file ATTACH_FILE ?
You can use the -e option to mail. That tells it not to do anything if the input is empty.
find $WORK_DIR -maxdepth 1 -name '*x.rsd' -type f -daystart -mmin +8640 -print | mail -e -s "Test "$TODAY mmm#server.com
To expand on my comment above:
Assign to an array variable and use printf to separate the found items with a newline character:
#!/bin/bash
WORK_DIR="/pub/"
FILE_LIST=($(find $WORK_DIR -maxdepth 1 \
-name '*x.rsd' -type f \
-daystart -mmin +8640 ))
if [ -n "${FILE_LIST[0]}" ]; then
printf '%s\n' "${FILE_LIST[#]}" | mail -s "Test "$TODAY mmm#server.com
fi
I exchanged /bin/sh with /bin/bash, as the question is tagged with [bash].
I am trying to create a script that will find all the files in a folder that contain, for example, the string 'J34567' and process them. Right now I can process all the files in the folder with my code, however, my script will not just process the contained string it will process all the files in the folder. In other words once I run the script even with the string name ./bashexample 'J37264' it will still process all the files even without that string name. Here is my code below:
#!/bin/bash
directory=$(cd `dirname .` && pwd)
tag=$1
echo find: $tag on $directory
find $directory . -type f -exec grep -sl "$tag" {} \;
for files in $directory/*$tag*
do
for i in *.std
do
/projects/OPSLIB/BCMTOOLS/sumfmt_linux < $i > $i.sum
done
for j in *.txt
do
egrep "device|Device|\(F\)" $i > $i.fail
done
echo $files
done
Kevin, you could try the following:
#!/bin/bash
directory='/home'
tag=$1
for files in $directory/*$tag*
do
if [ -f $files ]
then
#do your stuff
echo $files
fi
done
where directory is your directory name (you could pass it as a command-line argument too) and tag is the search term you are looking for in a filename.
Following script will give you the list of files that contain (inside the file, not in file name) the given pattern.
#!/bin/bash
directory=`pwd`
tag=$1
for file in $(find "$directory" -type f -exec grep -l "$tag" {} \;); do
echo $file
# use $file for further operations
done
What is the relevance of .std, .txt, .sum and .fail files to the files containing given pattern?
Its assumed there are no special characters, spaces, etc. in file names.
If that is the case following should help working around those.
How can I escape white space in a bash loop list?
Capturing output of find . -print0 into a bash array
There are multiple issues in your script.
Following is not required to set the operating directory to current directory.
directory=$(cd `dirname .` && pwd)
find is executed twice for the current directory due to $directory and ..
find $directory . -type f -exec grep -sl "$tag" {} \;
Also, result/output of above find is not used in for loop.
For loop is run for files in the $directory (sub directories not considered) with their file name having the given pattern.
for files in $directory/*$tag*
Following for loop will run for all .txt files in current directory, but will result in only one output file due to use of $i from previous loop.
for j in *.txt
do
egrep "device|Device|\(F\)" $i > $i.fail
done
This is my temporary solution. Please check if it follows your intention.
#!/bin/bash
directory=$(cd `dirname .` && pwd) ## Should this be just directory=$PWD ?
tag=$1
echo "find: $tag on $directory"
find "$directory" . -type f -exec grep -sl "$tag" {} \; ## Shouldn't you add -maxdepth 1 ? Are the files listed here the one that should be processed in the loop below instead?
for file in "$directory"/*"$tag"*; do
if [[ $file == *.std ]]; then
/projects/OPSLIB/BCMTOOLS/sumfmt_linux < "$file" > "${file}.sum"
fi
if [[ $file == *.txt ]]; then
egrep "device|Device|\(F\)" "$file" > "${file}.fail"
fi
echo "$file"
done
Update 1
#!/bin/bash
directory=$PWD ## Change this to another directory if needed.
tag=$1
echo "find: $tag on $directory"
while IFS= read -rd $'\0' file; do
echo "$file"
case "$file" in
*.std)
/projects/OPSLIB/BCMTOOLS/sumfmt_linux < "$file" > "${file}.sum"
;;
*.txt)
egrep "device|Device|\(F\)" "$file" > "${file}.fail"
;;
*)
echo "Unexpected match: $file"
;;
esac
done < <(exec find "$directory" -maxdepth 1 -type f -name "*${tag}*" \( -name '*.std' -or -name '*.txt' \) -print0) ## Change or remove the maxdepth option as wanted.
Update 2
#!/bin/bash
directory=$PWD
tag=$1
echo "find: $tag on $directory"
while IFS= read -rd $'\0' file; do
echo "$file"
/projects/OPSLIB/BCMTOOLS/sumfmt_linux < "$file" > "${file}.sum"
done < <(exec find "$directory" . -maxdepth 1 -type f -name "*${tag}*" -name '*.std' -print0)
while IFS= read -rd $'\0' file; do
echo "$file"
egrep "device|Device|\(F\)" "$file" > "${file}.fail"
done < <(exec find "$directory" -maxdepth 1 -type f -name "*${tag}*" -name '*.txt' -print0)
I would like that if this command outputs anything
find /var/www/cgi-bin -name touch -cmin 10
then should "ok" be echoed.
Have tried
if [ $(find /var/www/cgi-bin -name touch -cmin 10) ]; then echo "ok";fi
but it never echoes anything.
Put double quotes around $(..):
if [ "$(find /var/www/cgi-bin -name touch -cmin 10)" ]; then echo "ok"; fi
This will interpret the output of find as a single word.
This should work for you
if [ -n "$(find ./var/www/cgi-bin -name touch -cmin 10)" ];then echo ok;fi
Even better you can do it like this:
find /var/www/cgi-bin -name touch -cmin 10 -exec echo "ok" \;
HTH
if find ... | grep . > /dev/null; then
echo found something
fi
If you need the output:
if h=$(find ... | grep . ); then
echo found $h
fi