I have an hourly job A that is configured and the script is placed under /etc/cron.hourly.
Now I have another job say B that also needs to be run hourly. However, one requirement is that the job B should run after job A since job B will consume some output by job A.
In this case how should I configure to run job B? Will placing it under /etc/cron.hourly work?
Use a single crontab entry that runs both jobs.
For example, you can add this script to your /etc/cron.hourly directory:
#!/bin/sh
A
B
Note that unless this is a system-level task, it's probably better to put it in a user crontab entry rather than messing with files under /etc:
0 * 0 0 0 A ; B
Related
I have a job (job a) that is changing my source database and should not run multiple times at once. Adding to that, I have jobs using it as their data source that do not change the src-db (jobs b). So I want every job a to stop all other jobs from running, but jobs b to be able to be run parallel.
If I add job a and b to a resource group, nothing will run parallel with job a, but I can also only have one job b at once. Is it possible to go around that problem somehow?
I thought about running a job c in a resource group with job a, that runs just to check before a job b - but when the check is over, another job a could be scheduled, so that does not work.
In a previous question I asked how to queue a job B to start after job A, which is done with
sbatch --dependency=after:123456:+5 jobB.slurm
where 123456 is the id for job A, and :+5 denotes that it will start five minutes after job A.
I now need to do this for several jobs. Job B should depend on job A, job C on B, job D on C.
sbatch jobA.slurm will return Submitted batch job 123456, and I will need to pass the job id to the call with dependency for all but the first job. As I am using a busy cluster, I can't rely on incrementing the job ids by one, as someone might queue a job between.
As such I want to write a script that takes the job scripts (*.slurm) I want to run as arguments, e.g.
./run_jobs.sh jobA.slurm jobB.slurm jobC.slurm jobD.slurm
The script should then run, for all jobs scripts passed to it,
sbatch jobA.slurm # Submitted batch job 123456
sbatch --dependency=after:123456:+5 jobB.slurm # Submitted batch job 123457
sbatch --dependency=after:123457:+5 jobC.slurm # Submitted batch job 123458
sbatch --dependency=after:123458:+5 jobD.slurm # Submitted batch job 123459
What is an optimal way to do this with bash?
You can use the --parsable option to get the jobid of the previously submitted job:
#!/bin/bash
ID=$(sbatch --parsable $1)
shift
for script in "$#"; do
ID=$(sbatch --parsable --dependency=after:${ID}:+5 $script)
done
I have to perform 2 jobs - A and B. Job 'A' is to be prformed at 9:00 am of every weekday. I dont know the duration for job 'A' though, duration may vary.
Also I want to perform job 'B' after 3mins of completion of job 'A'.
Can anyone suggest the cron expression for this please.
Assuming you are trying to run the second job three minutes after the first job completes, let's say you have Job A which involves calling /home/user/job_a.sh and then once that completes, you want to run /home/user/job_b.sh. Instead of trying to set up two different Cron jobs, you could just make a separate script, say job_c.sh. And all job_c.sh does is run Job A, wait three minutes and then run Job B.
Basically, rather than calling two Cron Jobs and trying to sort out timing for both of them, you can just establish one Cron Job which runs both jobs.
On the other hand, if you want to run the second job three minutes after the first one starts then you might as well create two Cron Jobs with three minutes between them which would look something like this:
00 9 * * 1-5 /home/user/job_a.sh
03 9 * * 1-5 /home/user/job_b.sh
I need help with cron job that sends output to file every day and overwrites this file every month my only problem is how to make it overwrite each month and I need this in one job so creating 2 jobs one that outputs to a file and other removing it every month is out of picture
You could run it every day but use date +%w to print the day number and act differently (call with > to clobber the file instead of >> to append) based on that.
Note that some cron daemons require % to be escaped, hence \%.
# Run every day at 00:30 but overwrite file on Mondays; append every other day.
# Note that this requires bash as your shell.
# May need to override with SHELL=/bin/bash
30 00 * * * if [ "$(date +\%w)" = "1" ]; then /your/command > /your/logfile; else /your/command >> /your/logfile; fi
Edit:
You mention in comments above that your actual goal is log rotation.
The norm for Linux systems is to use something like logrotate to manage logs like this. That also has the advantage that you can keep multiple previous log files and compress them if you like.
I would recommend making use of a logrotate config snippet to accomplish your goal instead of doing it in the cron job itself. To put this in the cron job is counter-intuitive if it's merely for log rotation.
Here's an example logrotate snippet, which may go in a location like /etc/logrotate.d/yourapp depending on which Linux distribution you're using.
/var/log/yourlog {
daily
missingok
# keep one year of logs
rotate 365
compress
# keep the first one uncompressed for ease of viewing
delaycompress
}
This will result in your log file being rotated daily, with the first iteration being like /var/log/yourlog.1 and then compressed iterations like /var/log/yourlog.2.gz, /var/log/yourlog.3.gz and so on.
In my opinion therefore, your question is not actually a cron question. The kind of cron trickery used above would only be appropriate in situations such as when you want a job to fire on the last Sunday of the month, or the last day of the month, or other criteria that can't be expressed in cron syntax.
I have a cron job which runs every minute. Sometimes, if the cron is running more than a minute then another cron job is instantiated to do the same task. Hence duplicate cron jobs are created which is NOT I want. I want to make a conditional check that if a cron for a specific task is running, wait till the cron job completes or skip creating new cron job till the existing cron completes.
Create a text file somewhere which will store a value. (for example 0 or 1) When the task execute, change the value to 1. In the cron job, add a check that if the value in the file is 1 then don't execute the job. When your task is complete, remember to switch the value back to the default (for example 0).
You can even create a file when the task starts, and delete the file when task end, and only execute the cron job if file doesn't exist.
You can even put the check in the task itself instead of cluttering your cron table