script to log the completion of an LSF (bsub) job - cron

I have a script called by cron to run an LSF job.
I would like to be told when that job is submitted and when it completes. The-Powers-That-Be have decided to disable email notifications. So I am writing this script to output the relevant information to a logfile.
It almost works. Here is some code:
crontab:
00 12 * * * my_script.sh my_job.bsub
my_job.bsub:
#!/bin/bash
#BSUB -q my_queue
#BSUB -W 06:00
echo "I am a long process"
sleep 60
my_script.sh:
#!/bin/sh
BSUB_SCRIPT=$1
# run bsub_script (and get the LSF job id while you're at it)...
JOB_ID=`bsub < $BSUB_SCRIPT | awk -F[\<,\>] '{print $2}'`
# log that job was submitted...
echo "`date +%Y-%m%d %T` submitted '$BSUB_SCRIPT' [$JOB_ID]" >> $HOME/my_logfile.txt
# and log when job completes...
bsub -w "ended($JOB_ID)" << EOF
#!/bin/bash
#BSUB -q my_queue
#BSUB -W 00:30
echo "`date +%Y-%m-%d %T` completed '$BSUB_SCRIPT' [$JOB_ID]" >> $HOME/my_logfile.txt
EOF
(I found this answer helpful in figuring out how to submit a job that waits until an earlier one has completed.)
The issue is that the 2nd call to date, in the heredoc, gets evaluated immediately, so I wind up with a logfile that looks like this:
my_logfile.txt:
2018-01-30 13:15:14 submitted 'my_job.bsub' [1234567]
2018-01-30 13:15:14 completed 'my_job.bsub' [1234567]
Notice how the times are exactly the same.
How can I ensure that evaluation of the content of the heredoc is deferred until the LSF job runs?

The date command in the heredoc is being expanded before being passed to bsub. You need to quote the EOF in your heredoc expression or escape the date command. See the answer to this question:
How does "cat << EOF" work in bash?
In particular:
The format of here-documents is:
<<[-]word
here-document
delimiter
...
If word is unquoted, all lines of the here-document are
subjected to parameter expansion, command substitution, and arithmetic
expansion.
So, for example when I run
$ cat << EOF
> echo `date`
> EOF
The output is
echo Tue Jan 30 11:57:32 EST 2018
Note that the date command is expanded, which is what's happening in your script. However, if I quote the delimiter in the heredoc:
$ cat << "EOF"
> echo `date`
> EOF
You get the unexpanded output you want:
echo `date`
Similarly, escaping date would preserve the other variables you want to expand:
$ cat << EOF
> echo \$(date)
> EOF
Output:
echo $(date)

Related

Output of a command within here-document

I have a script which contains the code block:
cat << EOF > new_script.sh
...
echo "$(pwd)" >> log.txt
...
EOF
The script new_script.sh is set to run at a later time. Bash recognizes the $(pwd) within the script and evaluates it before it looks at the entire EOF block, so the pwd of the current directory is output instead of the pwd of new_script.sh when it is run. Why is this the case (what logic does bash use to know to evaluate $(command)) and what is the best solution to this?
By adding an escape $, \$ , you can solve this issue.
cat << EOF > new_script.sh
...
echo "\$(pwd)" >> log.txt
...
EOF
Unless you put single quotes around the EOF marker, the contents of the here-doc are treated like a double-quoted string, so all variables and command substitutions are expanded immediately.
To leave them as literals, use
cat << 'EOF' > new_script.sh
...
echo "$(pwd)" >> log.txt
...
EOF

How to make multiple commands executed sequentially in atd

echo "echo `date +'%Y%m%d-%H%M%S'`' hello' >> tmp.log;
sleep 3s;
echo `date +'%Y%m%d-%H%M%S'`' world' >> tmp.log" |
at now
I hope that these 3 commands will be executed in sequence use atd, but it is counterproductive. The 3 commands are executed in parallel. How can I execute these 3 commands in sequence?
atd does run commands sequentially, the problem is that both date commands run at the time you submit the job due to expansion, try :
#!/usr/bin/env bash
echo 'date +"%Y%m%d-%H%M%S hello" > /tmp/at.log;
sleep 3s;
date +"%Y%m%d-%H%M%S world" >> /tmp/at.log' |
at now

How to add threading to the bash script?

#!/bin/bash
cat input.txt | while read ips
do
cmd="$(snmpwalk -v2c -c abc#123 $ips sysUpTimeInstance)"
echo "$ips ---> $cmd"
echo "$ips $cmd" >> out_uptime.txt
done
How can i add threading to this bash script, i have around 80000 input and it takes lot of time?
Simple method. Assuming the order of the output is unimportant, and that snmpwalk's output is of no interest if it should fail, put a && at the end of each of the commands to background, except the last command which should have a & at the end:
#!/bin/bash
while read ips
do
cmd="$(nice snmpwalk -v2c -c abc#123 $ips sysUpTimeInstance)" &&
echo "$ips ---> $cmd" &&
echo "$ips $cmd" >> out_uptime.txt &
done < input.txt
Less simple. If snmpwalk can fail, and that output is also needed, lose the && and surround the code with curly braces,{}, followed by &. To redirect the appended output to include standard error use &>>:
#!/bin/bash
while read ips
do {
cmd="$(nice snmpwalk -v2c -c abc#123 $ips sysUpTimeInstance)"
echo "$ips ---> $cmd"
echo "$ips $cmd" &>> out_uptime.txt
} &
done < input.txt
The braces can contain more complex if ... then ... else ... fi statements, all of which would be backgrounded.
For those who don't have a complex snmpwalk command to test, here's a similar loop, which prints one through five but sleeps for random durations between echo commands:
for f in {1..5}; do
RANDOM=$f &&
sleep $((RANDOM/6000)) &&
echo $f &
done 2> /dev/null | cat
Output will be the same every time, (remove the RANDOM=$f && for varying output), and requires three seconds to run:
2
4
1
3
5
Compare that to code without the &&s and &:
for f in {1..5}; do
RANDOM=$f
sleep $((RANDOM/6000))
echo $f
done 2> /dev/null | cat
When run, the code requires seven seconds to run, with this output:
1
2
3
4
5
You can send tasks to the background by &. If you intend to wait for all of them to finish you can use the wait command:
process_to_background &
echo Processing ...
wait
echo Done
You can get the pid of the given task started in the background if you want to wait for one (or few) specific tasks.
important_process_to_background &
important_pid=$!
while i in {1..10}; do
less_important_process_to_background $i &
done
wait $important_pid
echo Important task finished
wait
echo All tasks finished
On note though: the background processes can mess up the output as they will run asynchronously. You might want to use a named pipe to collect the output from them.

Assigning the output of the "at" command in a bash script

I'm trying to capture the results of the "at" command inside a Bash script. The various ways of capturing command output don't seem to work, but I'm not sure if it's the pipe in the command or something else.
echo $cmd | at $deployat
produces the output
job 42 at 2014-04-03 12:00
And I'm trying to get at the time the job was set for.
However, I expected something like
v=$($cmd | at $deployat)
echo $v
Would work, or
v=$(echo $cmd | at $deployat)
echo $v
Or
v=`$cmd | at $deployat`
echo $v
But all of those leave the script hung, looking like it's waiting for some input.
What is the proper way to do this to end up with a variable like:
2014-04-03 12:00
============================
Edit:
One possible complication is that the $cmd has flags with it:
ls -l
for example.
The expanded command could be something like:
echo ls -l | at noon tomorrow
Solution:
v=$(echo $cmd | at $deployat 2>&1)
echo $v
at prints its output to stderr not stdout. Use 2>&1 to pipe the stderr of at into stdout. Example:
~$ out=$(echo cat Hello | at -v 2014-04-03 2>&1 | head -n 1)
~$ echo $out
Thu Apr 3 01:21:00 2014
With -v it prints the execution time on the first line which is taken by head -n 1.

How to output the start and stop datetime of shell script (but no other log)?

I am still very new to shell scripting (bash)...but I have written my first one and it is running as expected.
What I am currently doing is writing to the log with sh name-of-script.sh >> /cron.log 2>&1. However this writes everything out. It was great for debugging but now I don't need that.
I now only want to see the start date and time along with the end date and time
I would still like to write to cron.log but just the dates as mentioned above But I can't seem to figure out how to do that. Can someone point me in the right direction to do this...either from within the script or similar to what I've done above?
A simple approach would be to add something like:
echo `date`: Myscript starts
to the top of your script and
echo `date`: Myscript ends
to the bottom and
echo `date`: Myscript exited because ...
wherever it exits with an error.
The backticks around date (not normal quotes) cause the output of the date command to be interpolated into the echo statement.
You could wrap this in functions and so forth to make it neater, or use date -u to print in UTC, but this should get you going.
You ask in the comments how you would avoid the rest of the output appearing.
One option would be to redirect the output and error of everything else in the script to /dev/null, by adding '>/dev/null 2>&1' to every line that output something, or otherwise silence them. EG
if fgrep myuser /etc/password ; then
dosomething
fi
could be written:
if fgrep myuser /etc/password >/dev/null 2>&1 ; then
dosomething
fi
though
if fgrep -q myuser /etc/password ; then
dosomething
fi
is more efficient in this case.
Another option would be to put the date wrapper in the crontab entry. Something like:
0 * * * * sh -c 'echo `date`: myscript starting ; /path/to/myscript >/dev/null 2>&1; echo `date`: myscript finished'
Lastly, you could use a subshell. Put the body of your script into a function, and then call that in a subshell with output redirected.
#!/bin/bash
do_it ()
{
... your script here ...
}
echo `date`: myscript starting
( do_it ) >/dev/null 2>&1
echo `date`: myscript finished
Try the following:
TMP=$(date); name-of-scipt.sh; echo "$TMP-$(date)"
or with formatted date
TMP=$(date +%Y%m%d.%H%M%S); name-of-scipt.sh; echo "$TMP-$(date +%Y%m%d.%H%M%S)"

Resources