I'm currently running jobs on a computing cluster that uses the slurm workload manager. I can view all of the jobs I currently have running with:
$ squeue -u <username>
JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)
3379570 sixhour job01 <username> PD 0:00 1 (Priority)
3409269 sixhour job02 <username> R 03:06:13 1 n387
So in this example, I have two jobs that have been submitted. job01 is pending, and job02 has been running for around 3 hours.
I want to set up a cron job that will only submit the job if it's not listed in the above view. Take the following cron file as an example:
MAILTO=""
* */1 * * * sbatch job01.sh
* */1 * * * sbatch job02.sh
* */1 * * * sbatch job03.sh
Using this cron file, each job would get submitted every hour. But because job01 and job02 are already listed under squeue -u <username>, I only want job03 to actually get submitted.
Is there a way I can add some conditional logic to the cron file?
You can try a script like this:
#!/bin/bash
jobnames=$(squeue -h --user <username> --format %j)
for jobname in job{01..03} ; do
grep "$jobname" <<< "$jobnames" >/dev/null || sbatch "$jobname.sh"
done
This script will collect all job names related to jobs submitted by <username> in $jobnames and then iterate over the ones it should expect. If one is not found, grep will return non-zero exit code and the || sbatch will be executed.
Replace the three lines in your cron file with a single one running the above script.
Related
can we check process id or process of scheduled cronjob script
let say, I have scheduled one script "abc.sh" which runs all time and I have scheduled it crontab like below -
* * * * 0-5 script.sh
Once I schedule, I know it will keep running but can I check process status whether "script.sh" is running or not
Use pgrep or pidof:
pgrep script.sh
I have to write a cron job to execute a java class every after 8 hours. To test, I wrote a simple cron using :
crontab -e
*/1 * * * * echo $PATH
When I do, crontab -u <user> -l, it prints the entire cron file with above job listed. But I want to ascertain that my job is getting executed as intended. I don't see output of "echo $PATH" every minute. How can I see this output...?
I have a series of cron jobs running at the command line calling the php interpreter all by the same user configured to run once a day
0 0 * * * /usr/bin/php -q /mydirectory/myphp.php
0 0 * * * /usr/bin/php -q /mydirectory/myphp2.php
0 0 * * * /usr/bin/php -q /mydirectory/myphp3.php
Do these all execute at once or do the execute in the order of entry in some cron table, complete and move on to the next cron job?
And to answer your question despite the off-topic:
They will get executed in parallel, not sequentially. If you need some order it would pay to add them all to one script, and execute them sequentially separated by &&, e.g.
#!/bin/bash
/usr/bin/php -q /mydirectory/myphp.php && /usr/bin/php -q /mydirectory/myphp2.php && /usr/bin/php -q /mydirectory/myphp3.php
I know this question has been asked many, many times and I've done a lot of research but still I'm not able to run this extremely simple cron:
$ crontab -l
* * * * * /bin/date
This should, ideally, print the date every minute.
There is no cron.allow or cron.deny file, and the cron daemon is working:
ps -e | grep cron
1119 ? 00:00:00 cron
17646 ? 00:00:00 cron
Any idea what might be wrong?
Cron processes run in a separate subprocess of their own, so the output of a cron job will not be visible to you in your shell.
Instead, you will have to capture the output of your cron commands and save them. So, set your cronjob like:
* * * * * /bin/date >> /home/user/date.log
And now, if you will tail this log file, you will start seeing the result.
I want to set a cron run after an other cron. For example: Cron A finishs at 01:00 PM, cron B will start at 01:01 PM. The problem is I don't know when cron A finishs.
I checked the crontab syntax. It doesn't provide any param for that purpose.
My actual situation is:
# This cron must run first.
? ? * * * /usr/local/bin/php -f /path/select_and_print_to_log_file.php
# two these crons runs at the same time.
0 13 * * * /usr/local/bin/php -f /path/update_user.php
0 13 * * * /usr/local/bin/php -f /path/update_image.php
# This cron runs right after two above cron completes.
? ? * * * /usr/local/bin/php -f /path/select_and_print_to_log_file.php
You can use the batch command inside the first cron to have the second thing being scheduled to run.
Your first job could produce a timestamp when finished.
Then you estimate - for example - that job A needs about 60 to 90 minutes. After 60 minutes, you start job B. Job b looks for the timestamp. If it is present, job B starts, else it waits for a minute and looks again.
After finishing, job B deletes the timestamp, or renames it, maybe from 'todo' to 'done'. You could insert the current date inside the file, to check, whether your estimation is still acceptable, or should be adjusted.
What I do in such cases (commonly a backup scenario where I don't want to thrash the disk by having concurrent backups) is to write a script that cron calls, and in the script have the actual tasks run serially.
Something like:
#!/bin/bash
/usr/local/bin/php -f /path/update_user.php
/usr/local/bin/someOtherTaskToRunSecond
YMMV.