output of shell script to console and file - linux

I have shell script in Linux like below
#!/bin/bash
LOG_LOCATION=/home/$USER/logs
exec > >(tee /home/$USER/logs/"$1") 2>&1
[ $# -ne 1 ] && { echo "Usage : $0 table ";exit 1; }
table=$1
TIMESTAMP=`date "+%Y-%m-%d"`
touch /home/$USER/logs/${TIMESTAMP}.success_log
touch /home/$USER/logs/${TIMESTAMP}.fail_log
success_logs=/home/$USER/logs/${TIMESTAMP}.success_log
failed_logs=/home/$USER/logs/${TIMESTAMP}.fail_log
#Function to get the status of the job creation
function log_status
{
status=$1
message=$2
if [ "$status" -ne 0 ]; then
echo "`date +\"%Y-%m-%d %H:%M:%S\"` [ERROR] $message [Status] $status : failed" | tee -a "${failed_logs}"
#echo "Please find the attached log file for more details"
exit 1
else
echo "`date +\"%Y-%m-%d %H:%M:%S\"` [INFO] $message [Status] $status : success" | tee -a "${success_logs}"
fi
}
`hive -e "create table testing.${table} as select * from fishing.${table}"`
cp /home/$USER/logs/"$1" /home/$USER/debug/"$1"
g_STATUS=$?
log_status $g_STATUS "Hive create ${table}"
echo "***********************************************************************************************************************************************************************"
If I have this in my shell script
exec 2>&1 | tee /home/logging/"$1"
Then I am getting logs only on console not on the redirected file.
If I have this in my script
exec> /home/logging/"$1" 2>&1
Then I am having logs on the redirected file but not on the console.
How can I have logs both on console and redirected file

You can use process substitution with exec builtin:
exec > >(tee trace.log) 2>&1
to redirect both stdout and stderr to a file as as well as show it in terminal.

The purpose of the tee command is specifically intended to direct the output to both a file and to the terminal, which is what it sounds like you're wanting. This can be replicated pretty easily with something like the following:
script.sh:
#!/usr/bin/bash
date 2>&1 | tee "$1"
Then, running the command with ./script.sh abc.txt will produce the output of the date command to the terminal as well as the file abc.txt
In your case, exec 2>&1 | tee /home/logging/"$1" should correctly produce the results you want, but you will need to call the script with that argument carefully. That assumes the /home/logging directory exists, and you call the script above with something like ./script log.txt

Related

Use of echo >> produces inconsistent results

I've been trying to understand a problem that's cropped up with some of the scripts we use at work.
To generate many of our script logs, we utilize the exec command and file redirects to print all output from the script to both the terminal and a log file. Occasionally, for information that doesn't need to be displayed to the user, we do a straight redirect to the log file.
The issue we're seeing occurs on the last line of output to the file when we're printing the number of errors that occurred during that execution: The text doesn't get printed to the file.
In an attempt to diagnose the problem, I wrote a simplified version of our production script (script1.bash) and a test script (script2.bash) to try to tease out the problem.
script1.bash
#!/bin/bash
log_name="${USER}_`date +"%Y%m%d-%H%M%S"`_${HOST}_${1}.log"
log="/tmp/${log_name}"
log_tmp="/tmp/temp_logs"
err_count=0
finish()
{
local ecode=0
if [ $# -eq 1 ]; then
ecode=${1}
fi
# This is the problem line
echo "Error Count: ${err_count}" >> "${log}"
mvlog
local success=$?
exec 1>&3 2>&4
if [ ${success} -ne 0 ]; then
echo ""
echo "WARNING: Failed to save log file to ${log_tmp}"
echo ""
ecode=$((ecode+1))
fi
exit ${ecode}
}
mvlog()
{
local ecode=1
if [ ! -d "${log_tmp}" ]; then
mkdir -p "${log_tmp}"
chmod 775 "${log_tmp}"
fi
if [ -d "${log_tmp}" ]; then
rsync -pt --bwlimit=4096 "${log}" "${log_tmp}/${log_name}" 2> /dev/null
[ $? -eq 0 ] && ecode=0
if [ ${ecode} -eq 0 ]; then
rm -f "${log}"
fi
fi
}
exec 3>&1 4>&2 >(tee "${log}") 2>&1
ecode=0
echo
echo "Some text"
echo
finish ${ecode}
script2.bash
#!/bin/bash
runs=10000
logdir="/tmp/temp_logs"
if [ -d "${logdir}" ]; then
rm -rf "${logdir}"
fi
for i in $(seq 1 ${runs}); do
echo "Conducting run #${i}/${runs}..."
${HOME}/bin/script1.bash ${i}
done
echo "Scanning logs from runs..."
total_count=`find "${logdir}" -type f -name "*.log*" | wc -l`
missing_count=`grep -L 'Error Count:' ${logdir}/*.log* | grep -c /`
echo "Number of runs performed: ${runs}"
echo "Number of log files generated: ${total_count}"
echo "Number of log files missing text: ${missing_count}"
My first test indicated roughly 1% of the time the line isn't written to the log file. I then proceeded to try several different methods of handling this line of output.
Echo and Wait
echo "Error Count: ${err_count}" >> "${log}"
wait
Alternate print method
printf "Error Count: %d\n" ${err_count} >> "${log}"
No Explicit File Redirection
echo "Error Count: ${err_count}"
Echo and Sleep
echo "Error Count: ${err_count}" >> "${log}"
sleep 0.2
Of these, #1 and #2 each had a 1% fail rate while #4 had a staggering 99% fail rate. #3 was the only methodology that had a 0% fail rate.
At this point, I'm at a loss for why this behavior is occurring, so I'm asking the gurus here for any insight.
(Note that the simple solution is to implement #3, but I want to know why this is happening.)
Without testing, this looks like a race condition between your script and tee. It's generally better to avoid multiple programs writing to the same file at the same time.
If you do insist on having multiple writers, make sure they are all in append mode, in this case by using tee -a. Appends to the local filesystem are atomic, so all writes should make it (this is not necessarily true for networked file systems).

How to get logs of individual argument passed to shell script from a file

I have a shell script. In this script I am reading table names for a file and executing a command.
The script is working fine. I am able execute the command for all the tables in the file.
shell script
#!/bin/bash
[ $# -ne 1 ] && { echo "Usage : $0 input file "; exit 1; }
args_file=$1
TIMESTAMP=`date "+%Y-%m-%d"`
touch /home/$USER/logs/${TIMESTAMP}.success_log
touch /home/$USER/logs/${TIMESTAMP}.fail_log
success_logs=/home/$USER/logs/${TIMESTAMP}.success_log
failed_logs=/home/$USER/logs/${TIMESTAMP}.fail_log
#Function to get the status of the job creation
function log_status
{
status=$1
message=$2
if [ "$status" -ne 0 ]; then
echo "`date +\"%Y-%m-%d %H:%M:%S\"` [ERROR] $message [Status] $status : failed" | tee -a "${failed_logs}"
#echo "Please find the attached log file for more details"
#exit 1
else
echo "`date +\"%Y-%m-%d %H:%M:%S\"` [INFO] $message [Status] $status : success" | tee -a "${success_logs}"
fi
}
while read table ;do
spark-submit hive.py $table
done < ${args_file}
g_STATUS=$?
log_status $g_STATUS "Spark ${table}"
In this script I want to collect status logs and stdout logs. I want to collect the logs for each table in the file individually.
I want to know if the execution of spark-submit has been successful or failed for each table in the file. Say the status logs
How can I collect stdout files for each table individually and store them at a location in Linux.
What are the changes I need to do to achieve my results.
Make sure just to re-direct (stdout) of the logs generated for each of the table instance in your script to a folder under /var/log/ may be call it as myScriptLogs
mkdir -p /var/log/myScriptLogs || { echo "mkdir failed"; exit; }
while read -r table ;do
spark-submit hive.py "$table" > /var/log/myScriptLogs/"${table}_dump.log" 2>&1
done < "${args_file}"
The script will fail if you are not able to create a new directory using mkdir for some reason. So this creates a log for each table being processed under /var/log as <table_name>_dump.log which you can change it to however way you want.
Couple of best practices would be to use -r flag in read and double-quote shell variables.
Answer updated to redirect stderr also to the log file.

I can I echo error message and send it into a log file?

I'm trying to echo the error message as well as writing it in a log file at the same time but I'm not sure how to do it. I've used 1>&2, but it just sends it to the log file and doesn't echo the message. Here's my code:
while read -r username password; do
egrep "^$username" /etc/passwd >/dev/null
if [ $? -eq 0 ]; then
echo "ERROR BLABLAH $DATE" 1>&2 >> /var/log/error.log
Try
echo "ERROR BLABLAH $DATE" | tee -a /var/log/error.log 1>&2
Description:
tee # will repeat the std input.
-a /var/log/error.log # will append to the error.log file
1>&2 # will send the stdin to stderr.
You want to use the 'tee' command:
NAME
tee - read from standard input and write to standard output
and files
SYNOPSIS
tee [OPTION]... [FILE]...
e.g.
$echo "Hello world!" | tee test.txt
Hello world!
$cat test.txt
Hello world!

Shell script to check if the process is already running and exit if yes

I have a shell script with methods status() and start(). The code is below:
#function to check the jmeter processes running
status(){
PID=$(ps -ef | grep jmeter|grep -v grep)
echo "The jmeter processes running are: \n$PID"
}
#function to run the .jmx file given by the user at run time
start(){
echo "Please enter the file name .jmx extension"
read file
echo "Please enter the log file name .jtl extension"
read log_file
sh /home/ubuntu/apache-jmeter-3.0/bin/jmeter.sh -n -t $file -l $log_file &
}
while [ "$1" != "" ]; do
case "$1" in
start)
jmeter_start
;;
status)
jmeter_status
;;
*)
echo $"Usage: $0 {start|status}"
exit 1
esac
shift
done
now when I run this script, I have to check if it is already running and if it is running I have to exit. Let me know how to do this.
Add a flag at the beginning of the function and set it to 1, before the end of the function set it to 0, query it however you like.
#function to check the jmeter processes running
status(){
PID=$(ps -ef | grep jmeter|grep -v grep)
echo "The jmeter processes running are: \n$PID"
}
#function to run the .jmx file given by the user at run time
start(){
export start_flag=1
echo "Please enter the file name .jmx extension"
read file
echo "Please enter the log file name .jtl extension"
read log_file
sh /home/ubuntu/apache-jmeter-3.0/bin/jmeter.sh -n -t $file -l $log_file &
export start_flag=0
}
Another option would be writing to an external file and query it.
You actually have most of it already. You should be able to use the code from status that gets the PID and just check if it exists. If it does, output some error and exit. Otherwise, do what you already have.
start(){
PID=$(ps -ef | grep jmeter|grep -v grep)
if [ -z $PID ]; then
echo "Error: JMeter already running"
exit
fi
echo "Please enter the file name .jmx extension"
read file
echo "Please enter the log file name .jtl extension"
read log_file
sh /home/ubuntu/apache-jmeter-3.0/bin/jmeter.sh -n -t $file -l $log_file &
}

BASH: redirect for log dillema / duplicate redirection for each loop iteration

I've got a redirect dilemma that I can't get past in a bash backup script I'm developing in CentOS 6.4. I want to redirect all output to two separate files: one tmp and one permanent. The script loops through an external source list and I'd like for the tmp log files to be specific to the source, so that I can send an email if that specific source had errors containing that log (and conversely remove the tmp if the backup completes without error).
I'm using exec to tee my output:
exec > >(tee -a ${templog} /var/log/rob/rob.log) 2>&1
This works if I place at the top of the script, but here the variable isn't defined yet, so I can't do source-specific logs.
If I place this within the while loop, it grabs the variable, but writes a copy of each line determined by the total iterations of the loop; for the example below, I have four sources it iterates through, so I get output for each source in quadruplicate:
-S-07/11/14 09:15:35 ROB-Source Process for cc2-gamma has started-S-
-S-07/11/14 09:15:35 ROB-Source Process for cc2-gamma has started-S-
-S-07/11/14 09:15:35 ROB-Source Process for cc2-gamma has started-S-
-S-07/11/14 09:15:35 ROB-Source Process for cc2-gamma has started-S-
Share cc2-gamma is not Mounted. Try 1 of 5 to mount...
Share cc2-gamma is not Mounted. Try 1 of 5 to mount...
Share cc2-gamma is not Mounted. Try 1 of 5 to mount...
Share cc2-gamma is not Mounted. Try 1 of 5 to mount...
Is there a different way to tee the output within the loop to prevent this (without touching each line of course)? Or is there something rotten in my loops that I'm not seeing? Here's the whole script. Please excuse the mess and style.. I'm clearly not finished. I didn't include the config.conf and backup source file as they don't affect the output. Let me know if needed. Thanks.
#!/bin/bash
#V.2014.0723 - Radation Oncology Backup script
#declarations
SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/rob
source /rob/conf/config.conf
while read smbdir 'smbpath' exclfile drive foldername; do
#loop declarations
mountedfile=/rob/${smbdir}.MOUNTED
runningfile=/rob/${smbdir}.RUNNING
lastrunfile=/rob/${smbdir}_${foldername}.LASTRUN
templog=/rob/${smbdir}_${foldername}.TMPLOG
errorfile=/rob/${smbdir}_${foldername}.HAD_ERRORS
backupfile=/rob/${baname:0:3}_rtbackup.sql.bz2 # for the -l seccton below -- sql backup of backup.sql
#exec > >(tee -a ${templog} /var/log/rob/rob.log) 2>&1
### SOURCE BACKUP ##############################################################################################
if [ "$1" == "-s" ]
then
exec > >(tee -a /var/log/rob/rob.log ${templog}) 2>&1
#Write Source STDOUT and STDERR to both permanent and temporary log file. Must be in loop to use variables.
#exec > >(tee -a ${templog} /var/log/rob/rob.log) 2>&1
#exec > >(tee -a /var/log/rob.log ${templog}) 2>
if [ "${sources_active}" == "1" ]
then
echo "-S-$(date "+%m/%d/%y %T") ROB-Source Process for $smbdir has started-S-"
# unmount all cifs shares, due to duplicate mounts, write file to prevent concurrentcy
umount -a -t cifs > /dev/null
# The following will test to see if the souce is mounted, and if not, mount it.
for i in {1..5}
do
if mountpoint -q /mnt/${smbdir}/${drive}/${foldername}
then
echo "Share ${smbdir} is Mounted."
touch $mountedfile
break
else
sleep 2
echo "Share ${smbdir} is not Mounted. Try $i of 5 to mount..."
mkdir -p /mnt/${smbdir}/${drive}/${foldername} > /dev/null
mount -t cifs ${smbpath} -o ro,username=<USER>,password=<PW>,workgroup=<DOMAIN> /mnt/${smbdir}/${drive}/${foldername}
fi
done
# Test to see if above was successful, and if rob is not already running, run the backup.
if [[ -f ${mountedfile}&& ! -f ${runningfile} ]]
then
src="/mnt/${smbdir}/$drive"
dst="/backup/rob/"
touch ${runningfile}
/root/bin/rtbackup -m /mnt -p ${src}/${foldername} -b ${dst} -x #${exclfile}
if [ "$?" -ne "0" ]; then
#Errors Running RTBackup
rm -f ${runningfile} > /dev/null 2>&1
rm -f ${mountedfile}> /dev/null 2>&1
echo "$(date "+%m/%d/%y %T") Source Process for ${smbdir} had errors running:-SSS"
echo "$errors" >&2
touch ${errorfile}
exit 1
else
echo "What the hell is this doing?"
fi
#NO Errors Running RTBACKUP
rm -f ${templog}
rm -f ${runningfile} > /dev/null 2>&1
rm -f ${mountedfile} > /dev/null 2>&1
echo "$(date "+%m/%d/%y %T") Source Process for ${smbdir} did not have any errors"
else
#backup will *NOT* run, cleaning up and logging
rm -f ${mountedfile} > /dev/null 2>&1
echo "$(date "+%m/%d/%y %T") ${smbdir} could not be mounted, or is already in progress. Backup could not complete."
touch ${errorfile}
tail /var/log/rob/robso.log | mail -s "ROBSO Failed to run for ${smbdir} on ${baname}" ${email}
fi
echo "-F-$(date "+%m/%d/%y %T") ROB-Source Process for ${smbdir} has finished-F-"
#break
elif [[ "${sources_active}" == "0" ]]
then
echo "***$(date "+%m/%d/%y %T") ROB-Source Process for ${smbdir} did not run because the job is not set as active***"
#break
fi
done < /rob/conf/${baname}.conf
if [ $? -eq 10 ]; then exit 0; fi
You can use curly braces to redirect a set of commands; as it says in the bash manual about command grouping, "When commands are grouped, redirections may be applied to the entire command list". It behaves more-or-less like an anonymous function.
{
command1
command2
} > >(tee -a ${templog} /var/log/rob/rob.log) 2>&1
You can do the same with a named function, too, if you're so inclined, but I don't know offhand what environment would be used to expand the redirections. (If you do, please edit this answer!)
# Untested. This MIGHT work.
your_log_command() {
command1
command2
} > >(tee -a $1 /var/log/rob/rob.log) 2>&1
your_log_command $templog
your_log_command $something_else

Resources