i implement cron job A(EVERY_DAY_AT_8AM) on the batch server
when cron job A executed it queues a job to bull queue
and the other server consumes the job from bull queue
on batch server other cronjob which is executed more frequently than cron job A exists
The problem is although cron job A executed, job doesn't be queued to bull queue
In shorts, the job which has to be consumed had gone somewhere.
I can't even guess why this situation happen..
any one have ideas why does this issue happen??
Related
I was recently looking at npm bull package to use for delayed jobs. I can see that for delayed jobs it is using redis sorted set and mapping the value of the job to a hashset in redis. But I fail to understand how does the process receives the delayed job on time without using any iterable function like a cron or a settimeout which might be used to constantly looks for the jobs, and pops them by the score? can any one help me point to the code which is responsible for getting the delayed jobs?
I have a cron scheduled to run on Thor cluster. Is there a way to monitor a cron running on HPCC Cluster and send a notification if the cron is not running due to a failure or system shutdown?
Akhilesh,
The only way I can think of to do that would be to make the CRON job periodically send a "ping" of some sort (an email, or update a semaphore file, or ... ) then have a separate process running on another box to alert someone if that "ping" doesn't arrive as scheduled (indicating the CRON job is no longer working).
HTH,
Richard
I setup JDBCJobStore for store jobs and schedule the jobs by Cron.
Sometimes, I will manually stop the Quartz Scheduler in order to bypass some scheduled job to be triggered for some specific purpose.
However, I face an issue after re-starting the Quartz Scheduler. All the jobs which was scheduled will be triggered at same time even through the next schedule time has been due. I check the database and find all the jobs has been scheduled and saved in QRTZ_FIRED_TRIGGERS table, but not can be delete. Cron only re-schedule jobs after run.
Is there any way to make Quartz to re-schedule job by Cron when I re-start the Quartz Server and without trigger these expired schedule?
Any help will be highly appreciated.
Best Regards,
Dean Huang
The job will be reschedule by Cron if I configure as RAMJobStore and setup job by xml, but not JDBCJobStore.
I want to run a python cron-type scheduler and schedule a job for one time execution at a particular date and time. Once the job is done, the job has to disappear, while the scheduler still continues to run. Also looking for examples of using scheduler and storing jobs in mongodb or redis.
You can use celery for this.Celery helps you run tasks in background and schedule cron jobs as well.
These docs state that you can
schedule tasks to execute at a specific time
other useful resources.
Sorry but can't find he configuration point a need. I schedule spark application, sometimes they may not succeed after 1 hour, in this case I want to automatically kill this task (because I am sure it will never succeed, and another scheduling may start).
I found a timeout configuration, but as I understand it, this is used to delay the start of a workflow.
So is there a kind of living' timeout ?
Oozie cannot kill a workflow that it triggered. However you can ensure that a single workflow is running at same time by setting Concurrency = 1 in the Coordinator.
Also you can have a second Oozie workflow monitoring the status of the Spark job.
Anyawy, you should investigate the root cause of Spark job not successful or being blocked.