I am working on a script that take 1 string argument and a file. I want it so that if a file is put in that doesn't exist, then it will display the "filename cannot be read" message.
That part does work however it also displays a "grep: grep.txt: No such file or directory" message. Is there any way to stop the grep message from posting and ending the script if the first if statement is true?
#! /bin/sh
if [ ! -f "$2" ]
then
echo "$0" cannot be read 1>&2
fi
if [ $# -eq 2 ]
then
grep "$1" $2
else
echo there is more or less than 2 arguments 1>&2
fi
Exit the script with a non-zero exit code to indicate failure and stop it from continuing on to the grep.
if [ ! -f "$2" ]
then
echo "$0" cannot be read 1>&2
exit 1
fi
You can add /dev/null in grep command it will suppress the error part.
grep "$1" $2 2>/dev/null
The > operator redirects the output usually to a file but it can be to a device. You can also use >> to append.
2> file redirects stderr to file
/dev/null is the null device it takes any input you want and throws it away. It can be used to suppress any output.
You could redirect all errors from grep, for example:
grep "$1" $2 2>/dev/null
(the 2> means redirect standard error, as opposed to standard output with > or 1>).
That introduces a race condition, however: if the file disappears while your script as running, it might still exist when you check that it exists, but be gone by the time grep runs.
You could handle that by checking the exit status...
grep "$1" $2 2>/dev/null
if [[ $? -gt 1 ]]; then
echo "grep failed unexpectedly" >&2
fi
IMHO, in this example it would be better to just let grep print the error.
Related
I've been trying to understand a problem that's cropped up with some of the scripts we use at work.
To generate many of our script logs, we utilize the exec command and file redirects to print all output from the script to both the terminal and a log file. Occasionally, for information that doesn't need to be displayed to the user, we do a straight redirect to the log file.
The issue we're seeing occurs on the last line of output to the file when we're printing the number of errors that occurred during that execution: The text doesn't get printed to the file.
In an attempt to diagnose the problem, I wrote a simplified version of our production script (script1.bash) and a test script (script2.bash) to try to tease out the problem.
script1.bash
#!/bin/bash
log_name="${USER}_`date +"%Y%m%d-%H%M%S"`_${HOST}_${1}.log"
log="/tmp/${log_name}"
log_tmp="/tmp/temp_logs"
err_count=0
finish()
{
local ecode=0
if [ $# -eq 1 ]; then
ecode=${1}
fi
# This is the problem line
echo "Error Count: ${err_count}" >> "${log}"
mvlog
local success=$?
exec 1>&3 2>&4
if [ ${success} -ne 0 ]; then
echo ""
echo "WARNING: Failed to save log file to ${log_tmp}"
echo ""
ecode=$((ecode+1))
fi
exit ${ecode}
}
mvlog()
{
local ecode=1
if [ ! -d "${log_tmp}" ]; then
mkdir -p "${log_tmp}"
chmod 775 "${log_tmp}"
fi
if [ -d "${log_tmp}" ]; then
rsync -pt --bwlimit=4096 "${log}" "${log_tmp}/${log_name}" 2> /dev/null
[ $? -eq 0 ] && ecode=0
if [ ${ecode} -eq 0 ]; then
rm -f "${log}"
fi
fi
}
exec 3>&1 4>&2 >(tee "${log}") 2>&1
ecode=0
echo
echo "Some text"
echo
finish ${ecode}
script2.bash
#!/bin/bash
runs=10000
logdir="/tmp/temp_logs"
if [ -d "${logdir}" ]; then
rm -rf "${logdir}"
fi
for i in $(seq 1 ${runs}); do
echo "Conducting run #${i}/${runs}..."
${HOME}/bin/script1.bash ${i}
done
echo "Scanning logs from runs..."
total_count=`find "${logdir}" -type f -name "*.log*" | wc -l`
missing_count=`grep -L 'Error Count:' ${logdir}/*.log* | grep -c /`
echo "Number of runs performed: ${runs}"
echo "Number of log files generated: ${total_count}"
echo "Number of log files missing text: ${missing_count}"
My first test indicated roughly 1% of the time the line isn't written to the log file. I then proceeded to try several different methods of handling this line of output.
Echo and Wait
echo "Error Count: ${err_count}" >> "${log}"
wait
Alternate print method
printf "Error Count: %d\n" ${err_count} >> "${log}"
No Explicit File Redirection
echo "Error Count: ${err_count}"
Echo and Sleep
echo "Error Count: ${err_count}" >> "${log}"
sleep 0.2
Of these, #1 and #2 each had a 1% fail rate while #4 had a staggering 99% fail rate. #3 was the only methodology that had a 0% fail rate.
At this point, I'm at a loss for why this behavior is occurring, so I'm asking the gurus here for any insight.
(Note that the simple solution is to implement #3, but I want to know why this is happening.)
Without testing, this looks like a race condition between your script and tee. It's generally better to avoid multiple programs writing to the same file at the same time.
If you do insist on having multiple writers, make sure they are all in append mode, in this case by using tee -a. Appends to the local filesystem are atomic, so all writes should make it (this is not necessarily true for networked file systems).
I'm a rookie in bash scripting, and here's basically my bash script:
Z=`diff -Z $ref_out $exec_out | grep "[<>]" | wc -l` 2>/dev/null
if [ $Z -gt 0 ]; then
echo "*** testcase: [ stdout - FAILED ]"
else
echo "*** testcase: [ stdout - PASSED ]"
fi
I would like to suppress the error message from diff such as:
diff: No such file or directory
This could either result from no $ref_out or $exec_out file, though I'm redirecting to /dev/null, this error message still shows up.
Any help?
You need diff's stderr to go to /dev/null, so it should instead be:
Z=`diff -Z $ref_out $exec_out 2> /dev/null | grep "[<>]" | wc -l`
Your redirection isn't working because it is being applied to the parent shell, not the subshell that runs the pipeline.
If you want to send the stderr of a bunch of commands to /dev/null, you could do it this way - I am using $() instead of backticks:
Z=$( { diff -Z $ref_out $exec_out | grep "[<>]" | wc -l; } 2>/dev/null )
Here, 2>/dev/null applies to all the commands inside { }.
There are many issues in your code. You could rewrite it in a better way:
if diff -Z "$ref_out" "$exec_out" 2>/dev/null | grep -q "[<>]"; then
echo "*** testcase: [ stdout - FAILED ]"
else
echo "*** testcase: [ stdout - PASSED ]"
fi
grep -q is a better way to do this check and you won't need a wc -l unless you want to know the exact number of matches
you need to quote your variables
if statement can include commands; you don't need to capture the output in order to use it in the if statement
You can use shellcheck to validate your shell script and see if you are making the usual mistakes that can break your code.
So I am writing a bash script which will run through all of the process ids in /proc/[pid] and read the executable that was used to run it.
From what I have had a looked at, the /proc filesystem contains the /proc/[pid]/exe symbolic link. Within the bash script I am trying work out how to read the value of "readlink /proc/[pid]/exe" to check if (deleted) or nothing is returned to find out whether the original executable exists on the disk or not.
Is there a way of doing this, so far I have?
#!/bin/bash
pid = "0"
while [ $pid -lt 32769 ]
do
if [-d /proc/$pid]; then
if [-f /proc/$pid/exe]; then
echo $pid
readlink /proc/$pid/exe
fi
fi
pid = $[$pid+1]
done
This fails to work and always returns nothing.I am trying to list all of the processes that no longer have their executables available on disk.
Will this work for you?
#!/bin/bash
for i in $(ls /proc | awk '/^[[:digit:]]+/{print $1}'); do
if [ -h /proc/$i/exe ]; then
echo -n "$i: "
if readlink /proc/$i/exe >/dev/null 2>&1 ; then
echo "executable exists"
else
echo "executable not found"
fi
fi
done
I've updated your script to make it work. Notice that -f checks whether a file name represents a regular file. I would return false for a symbolic link:
pid="0"
while [ $pid -lt 32769 ]
do
if [ -d /proc/$pid ]; then
if [ -h /proc/$pid/exe ]; then
echo $pid
readlink /proc/$pid/exe
fi
fi
pid=$[$pid+1]
done
you can read returned value after any command in shell by printing $? variable:
readlink
echo $?
if link is invalid, $? will be bigger than 0.
however if link exist and actual file is deleted, you can use something like:
ls `readlink somelink`
readlink -f `ls --dereference /proc/$pid/exe`
I'm trying to run programs (for example mv file1.txt file2.txt) in my .sh script and I need to hide errors, but handle it with my script.
Currently I'm trying to do something like
EXECUTE="mv -v $VOL $BACKUP_YESTERDAY_CRYPT"
{
EXEC_ERROR=$($EXECUTE)
} &2>> $LOG_FILE
if [[ -n $EXEC_ERROR ]]; then
echo "There is an error!"
fi
But it doesn't work at all - it shows an error (for example mv: cannot stat 'file1.txt': No such file or directory) and $EXEC_ERROR variable is empty.
Is there any way to get output to variable + to log file?
How about something like:
mv -v $VOL $BACKUP_YESTERDAY_CRYPT 2>> $LOG_FILE
if [[ ! ( $? -eq 0 ) ]] ; then
echo "There is an error\!"
fi
Though $? is good for saving and processing exit codes, the if statement is designed to take any command, not just [ or [[:
if ! mv -v "$VOL" "$BACKUP_YESTERDAY_CRYPT" 2>> $LOG_FILE; then
echo "There is an error!"
fi
This includes saving variables:
if OUTPUT=$(mv -v "$VOL" "$BACKUP_YESTERDAY_CRYPT" 2>> $LOG_FILE); then
echo ">>> $OUTPUT <<<"
fi
In fact, if can take more than one command, as its man page describes. Documentation on boolean operators such as !, &&, and || is hidden within the description of shell commands, where they form pipelines (!) and lists (&&, ||).
Try this:
mv sourcefile destfile 2> /dev/null 1>logfile
returnstatus=`echo $?`
if [[ $returnstatus -ne 0 ]]; then
echo "There was an error!"
fi
I have a bash program that will write to an output file. This file may or may not exist, but the script must check permissions and fail early. I can't find an elegant way to make this happen. Here's what I have tried.
set +e
touch $file
set -e
if [ $? -ne 0 ]; then exit;fi
I keep set -e on for this script so it fails if there is ever an error on any line. Is there an easier way to do the above script?
Why complicate things?
file=exists_and_writeable
if [ ! -e "$file" ] ; then
touch "$file"
fi
if [ ! -w "$file" ] ; then
echo cannot write to $file
exit 1
fi
Or, more concisely,
( [ -e "$file" ] || touch "$file" ) && [ ! -w "$file" ] && echo cannot write to $file && exit 1
Rather than check $? on a different line, check the return value immediately like this:
touch file || exit
As long as your umask doesn't restrict the write bit from being set, you can just rely on the return value of touch
You can use -w to check if a file is writable (search for it in the bash man page).
if [[ ! -w $file ]]; then exit; fi
Why must the script fail early? By separating the writable test and the file open() you introduce a race condition. Instead, why not try to open (truncate/append) the file for writing, and deal with the error if it occurs? Something like:
$ echo foo > output.txt
$ if [ $? -ne 0 ]; then die("Couldn't echo foo")
As others mention, the "noclobber" option might be useful if you want to avoid overwriting existing files.
Open the file for writing. In the shell, this is done with an output redirection. You can redirect the shell's standard output by putting the redirection on the exec built-in with no argument.
set -e
exec >shell.out # exit if shell.out can't be opened
echo "This will appear in shell.out"
Make sure you haven't set the noclobber option (which is useful interactively but often unusable in scripts). Use > if you want to truncate the file if it exists, and >> if you want to append instead.
If you only want to test permissions, you can run : >foo.out to create the file (or truncate it if it exists).
If you only want some commands to write to the file, open it on some other descriptor, then redirect as needed.
set -e
exec 3>foo.out
echo "This will appear on the standard output"
echo >&3 "This will appear in foo.out"
echo "This will appear both on standard output and in foo.out" | tee /dev/fd/3
(/dev/fd is not supported everywhere; it's available at least on Linux, *BSD, Solaris and Cygwin.)