I am scheduling an Agenda Job as below:
await agenda.now("xyz");
But the above command makes my job running almost every 1 minute. But when I change it to
await agenda.every('5 minutes', "xyz");
The above works as expected i.e. it runs the job every 5 minutes.
But I don't want a recurring job. Rather run it once.
The issue was with the concurrency of the job definition. It was set to 10 because of which several instances of the same job were running in parallel.
Changing the concurrency to 1 solved the issue.
Related
I have around 20 cron jobs that run at various times over 3 computers, some of them run at the same time, some at even hours, some at odd hours. Some take 5 minute to complete, other up to 20 hours. The jobs i'm running are bash scripts.
I would like some of these jobs to run in a seamless fashion on any node of a set of computers, but always one instance at the time (for now i handle this "one at the time maximum" using /usr/bin/flock).
One constraint i have is that i want certain jobs to run only on the same node due to hardware and network limitations.
But i still want to have only one cron file to edit. For now i have to edit as many cron files that i have nodes, which is unpractical.
Is there a conventional way to achieve this ?
I am using NodeJS,MongoDB and node-cron npm module to schedule jobs. For 10K of jobs it is taking less time and less memory. But when i am scheduling 100k jobs it is taking more than 10 minutes to schedule jobs and taking nearly 1.5GB of RAM and some times out of memory. Is there any best way achieve this like using activemq or rabbitmq?
One strategy is that you only schedule the next job to run. When it runs, you query the database and find the next job and schedule it.
If you add a new job, you check if it wants to run sooner than the now current next job and, if so, you schedule it and deschedule the previous next job (it will get rescheduled later after this new job runs).
If you remove a job, you check if it is the current next job. If it is, you deschedule it and find the next job in the database and schedule it.
If your database is configured for efficiently querying by job run time, this can be very efficient, uses hardly any memory and scales to an infinitely large number of jobs.
Sometimes Crons is working sometimes getting missed. I have attached all setting and result. Anyone can check and revert.
It's completely normal behaviour. Some jobs are skipped caused the time frame is out of scheduled time for specified cron job. In your case the reindex process is scheduled every 1 minute. If there is more things to index (lot of changes on products, categories etc.) one minute is's not enough to complete. Also there is only one process per cron group, in your case index. Use Separate Process in cron configuration means that indexes process will run as separate process in relation to other cron groups.
I have a cron job that runs every 30 minutes, starting 10 minutes past a whole hour:
0+10/30+*+*+*+?
Now, this needs to be changed, so that in a specific time interval, it runs every 15 minutes instead. E.g. at 7.50, 8.05, 8.20 and 8.35. Then every 30 minutes again.
Is this possible with a single cron job and if so, how? Or do I need multiple jobs to accomplish this?
Thank you in advance.
not easy in a single cron, and that is also hard to read.
multiple jobs may work fine and show much clear
// This will start at 1:10am, and every 30minutes run once.
0+10/30+1-23/2+*+*+?
// This will start at 0:10am, and every 15minutes run once.
0+10/15+0-24/2+*+*+?
you may also consider to void the two job running at the same time.
As far as I've understood, this is not possible within a single cron job.
setup cron from morning to evening only points out that three different cron jobs are needed, so I am closing my question.
I have scheduled the K8s cron to run every 30 mins.
If the current job is still running and the next cron schedule has reached it shouldn't create a new job but rather wait for the next schedule.
And repeat the same process if the previous job is still in Running state.
set the following property to Forbid in CronJob yaml
.spec.concurrencyPolicy
https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#concurrency-policy
spec.concurrencyPolicy: Forbid will hold off starting a second job if there is still an old one running. However that job will be queued to start immediately after the old job finishes.
To skip running a new job entirely and instead wait until the next scheduled time, set .spec.startingDeadlineSeconds to be smaller than the cronjob interval (but larger than the max expected startup time of the job).
If you're running a job every 30 minutes and know the job will never take more than one minute to start, set .spec.startingDeadlineSeconds: 60