I have a giant crontab with many scripts on a linux machine. I need to be able to a) change the subject and/or from of cronjob result emails, because the default is unreadably long. b) Do so via a centralized solution. c) Only require minimal changes to the crontab itself.
For example this crontab line:
0 */3 * * * /path/to/script1 | /path/to/script2 | /path/to/script3
Creates this email subject:
Cron <cronuser#myserver> /path/to/script1 | /path/to/script2 | /path/to/script3
Which in my inbox gets cut off somewhere in the path to script1. (And many lines in the crontab are significantly longer.)
Options I've tried:
Piping to mail and setting subject etc per line (The -E preserves cron's default only-send-on-output behavior):
0 */3 * * * /path/to/script1 | /path/to/script2 | /path/to/script3 2>&1 | mail -E -s "test subject" -S from="Cron Script2 <cronuser#myserver.com>" recipient#myserver.com
This "works", but I want to centralize my changes in one place, and minimize how much I add to each cron line for readability
Using shell : (noop) command, which shows up first in the subject (note that the space after the : is important!):
0 */3 * * * : Descriptive Words; /path/to/script1 | /path/to/script2 | /path/to/script3
Unfortunately there's still too much unnecessary that cron puts before "Descriptive Words" on the subject line, so this is still unusable.
What I want to create:
Something generic, like this:
0 */3 * * * /path/to/script1 | /path/to/script2 | /path/to/script3 2>&1 | coolmailer.pl
coolmailer.pl would build the subject line by getting the commands on that cron line, strip out the paths and arguments, and email me this (optionally only if there's a fail in any of the scripts):
SUBJECT: script1 | script2 | script3
FROM: Cron Script3<noreply#myserver.com>
(actual results of the command /path/to/script1 | /path/to/script2 | /path/to/script3)
As a bonus, I'd also love to say whether any of the previous commands (script1 or script2) failed on the subject line.
This has turned out to be... way more complicated than I expected.
Challenges:
Find a way for a pipeline member (coolmailer) to know the other
members of the pipeline.
There's an ingenious method for 1 here using lsof how-do-you-determine-the-actual-command-that-is-piping-into-you but it also sometimes finds commands started by the scripts in my pipeline (ie if script1 forks processes or does system calls, those show up too any time they take long enough to complete.) Ditto for the method using process groups at the same link.
Find a way for a pipeline member (coolmailer) to know the results of
other members of the pipeline. (I realize this may not be possible at all, but the lsof hack gives me hope.)
Any better way? Does the fact that I'm running from cron buy me anything? Part of me wants to combine the lsof strategy with grepping through crontab -l results, but that just seems too kludgy and prone to errors.
Caveats:
I can have changes made to my account, but I can't make changes that
would effect all users. I.e. if there's a way to change cron's
mailing format server-wide, that doesn't help.
I can't realistically update every script called to handle emailing its own results, even if that's probably the "right" way.
I know about the mail -s -E -S options, but would prefer to have a single place to change things. Also, I really want to find a way to get the pipeline.
Language used now for "coolmailer" is Perl, but I'll try anything
My first attempt:
(which works, except it often also shows other commands started inside my scripts, which means it doesn't work)
#!/usr/bin/perl -w
my $pgid=`ps -o pgid= -p $$`;
my $lsofout = `/usr/sbin/lsof -g $pgid`;
my #otherpids = `echo "$lsofout" | awk '\$5 == "1w" { print \$2 }'`;
my #longcmds;
my #shortcmds;
foreach my $pid (#otherpids) {
chomp($pid);
if (my $cmd = `ps -o cmd= -p $pid 2>/dev/null`) {
chomp($cmd);
push #longcmds, $cmd;
next;
}
}
my $cmdline = join (' | ',#longcmds);
foreach my $cmd (#longcmds) {
$cmd =~ s/(\/\S+\/)(\S+)/$2/g;
push #shortcmds, $cmd;
}
my $subj = join('|',#shortcmds);
print "SUBJ:$subj\n";
print "CMDLINE: $cmdline\n";
# and now do some mail stuff
And final version, based on suggestion by Jhnc
#!/usr/bin/perl -w
# cronmgr.pl -- understand cron emails for once
# usage: 0 */3 * * * cronmgr.pl cd blah\; /path/to/script1 \| /path/to/script2 \| /path/to/script3
# note that ; | & in any cronmgr.pl line must be backslashed to run!
use strict;
use IPC::Cmd qw[can_run run run_forked];
my $CMDLINE = join(' ',#ARGV);
my( $success, $error_message, $full_buf, $stdout_buf, $stderr_buf ) =
run( command => $CMDLINE, verbose => 0 ); # verbose = 0 means don't output normally, capture all output
my ($stdout, $stderr);
$stdout = join "", #$stdout_buf;
$stderr = join "", #$stderr_buf;
my $emailsubject;
if( $success ) {
if ($stdout eq '' && $stderr eq '') { # if there's no output, don't send any email!
exit;
}
} else {
print "CMD FAIL!\n$error_message\nSTDERR:\n$stderr";
$emailsubject = "FAIL:$error_message";
}
# etc etc
(Edited for clarity re goals and why options attempted so far aren't sufficient.)
determining the pipeline
If you always invoke coolmailer.pl with a unique argument then you can simply grep it from your list of cronjobs:
#!/usr/bin/perl -wT
$ENV{PATH} = '/sensible:/path';
my ($pipeline) = grep /\|\s+$0\s+$ARGV[0]/, `crontab -l`;
$pipeline ||= "oops";
# ... mung $pipeline ...
# ... do mail stuff ...
checking pipe failure
If you rewrite your cronjob entries from:
/path/to/script1 | /path/to/script2 | /path/to/script3 2>&1 | coolmailer.pl
to:
coolermailer /path/to/script1 \| /path/to/script2 \| /path/to/script3
then you could construct the pipeline manually and have control over pipe member status information. (This also gives you the pipeline directly, although you then have to construct it before it will run.)
For example, with a bash implementation, you might make use of eval and PIPESTATUS. With Perl, you might use results() from IPC::Run
I'm trying to configure crontab to execute at different times different lines of code inside a file. I basically have a bash script file that starts some java -jar. The problem is that each line should be executed at a different time. I can configure crontab to run the whole script at different times but no the lines to run. Now this is important that the bash file will stay only one file and not broken down to a few files.
Thanks!
One way of doing it (via command line arguments passed by cron)
some_script.sh:
if test $1 = 1 ; then
# echo "1 was entered"
java -jar some_file.jar
elif test $1 = 2 ; then
# echo "2 was entered"
java -jar another_file.jar
fi
crontab example:
* 1 * * * /bin/bash /home/username/some_script.sh 1
* 2 * * * /bin/bash /home/username/some_script.sh 2
Another approach (hour matching done in bash script)
some_script.sh:
hour=$(date +"%H");
if test $hour = 1 ; then
# echo "the hour is 1";
java -jar some_file.jar
elif test $hour = 2 ; then
# echo "the hour is 2";
java -jar another_file.jar
fi
crontab example:
* 1 * * * /bin/bash /home/username/some_script.sh
* 2 * * * /bin/bash /home/username/some_script.sh
My /etc/crontab looks like this:
* * * * * echo"Another 5 Minutes! " >> /tmp/5-minutes.txt
I figured this would append "Another 5 Minutes! " to /tmp/5-minutes.txt every minute. When I perform a 'less' command on the file, the changes are not there.
EDIT: If it helps, I have one other job in the crontab file, perhaps it's effecting the other job.
0 * * * * finger >> /tmp/hourly-finger.txt
* * * * * echo "Another 5 Minutes! " >> /tmp/5-minute.txt
Give a space after echo
echo "Another 5 Minutes! "
I figured it out. Because I'm editing the /etc/crontab directly I needed to specify a user for the crontab to run as. In my case I ran it as root so...
* * * * * root echo "Another 5 Minutes! : >> /tmp/5-minutes.txt
I have this code currently:
echo "20 0 * * * cd /var/www/test/ && ./prog >> /var/log/program.log" >> mycron
This works fine, but now I want to store it in a a timedated file each time in the format like this:
program_YYYYMMDD_HHMMSS.log
Can anyone tell me how I can do this? I think I need to use the date variable but im not really sure how to implement it.
Yes, you are right. You can use the date variable.
echo "20 0 * * * cd /var/www/test/ && ./prog >> /var/log/program_$(date "+%Y%m%d_%H%M%S").log" >> mycron
I'm trying to set a cron job, namely echoing "hi" every minute.
When I do * * * * * echo "hi" I get blado.kdb: command not found. Any ideas how I could fix this?
Run crontab -e in your shell. This opens a text editor
Only then type * * * * * echo "hi". Save the file the text editor just opened for you
Your Cron task is now set
PS: echo "hi" will print "hi" in a void, if you want to see some results, set a task such as * * * * * touch /tmp/foo, and you'll see the modification date being updated every minute (ls -l /tmp/foo)