I'm using camel framework to declare some scheduled jobs with quartz. In there, I want to execute my class in every two seconds.
So, I have mentioned this:
quartz2://quartzScheduler/Processor?cron=0/2+*+*+*+*+?
But its not executing.
the first six fields in a quartz cron expression are not optional, so your url should probably be:
quartz2://quartzScheduler/Processor?cron=0/2 * * * * *
Related
I want to trigger a shell script at 35th second, every minute. However, I see that airflow supports CRON from the minute level instead of the second level.
For example, the cron Expression
35 * * * *
triggers a job at every 35th minute in airflow.
I'm not familiar with Airflow, but cron only supports 1-minute resolution.
You could use cron to invoke a job every minute and let the job sleep for 35 seconds:
* * * * * sleep 35 ; do_something
Rewrite of the question:
Using airflow, I would like to schedule a process to run every two hours from 2 till 10 am and a single time at 22:30. The schedule_interval parameter accepts a cron-expression, but it is not possible to define a single cron-expression to achieve the above scheduling. Currently, I did:
dag = DAG(process_name, schedule_interval='30 2,4,6,8,10,12,14,16,18,20,22,23 * * *', default_args=default_args)
But this will execute the process every 30 minutes past the hour, and this every 2 hours from 2 till 23.
Is there a way I can combine two cron-schedules in Airflow?
0 2-10/2 * * *
30 22 * * *
Original question:
I have 2,4,6,10,12,14,16,18,20,22 00 * *
I need to have 23, 30 in my schedule, but I don't want 2-22 to be run at the 30 min interval.
So, I realized, it is not possible!
You can't use two cron expressions for the same DAG (Might change in the future if PR: Add support for multiple cron expressions in schedule_interval is accepted)
Starting Airflow >=2.2.0:
It is possible to get custom time based triggering using custom Timetable by customizing the DAG scheduling to match what you expect.
To do so you need to define the scheduling logic by implementing next_dagrun_info and infer_manual_data_interval functions - Airflow will leverage this logic to schedule your DAG.
You can view an example can be found here.
I wonder if it is possible to write an cron expression with several conditions:
Job should be run with given interval in minutes. For example with interval 42 minutes the fire times would be 10:00, 10:42, 11:24, 12:06 and etc.
If the current minute does not end with 0 (e.g. 10:28,10:29), then cron first fire time should be 10:30. So it means that first fire time should have "round" minutes.
I hope that you understand these conditions. Is it possible to describe them with quartz cron?
You can use job trigger like described below in Quartz.net 3.0:
var jobTrigger = TriggerBuilder.Create()
.StartNow()
.WithSimpleSchedule(s => s
.WithIntervalInMinutes(42)
.RepeatForever())
.Build();
And you can restart app at first round time, so it will fire first time at the same time only.
I usually use http://www.cronmaker.com/ to generate my cron expressions. And if you try the every 42 mins option you'll get the following expression: " 0 0/42 * 1/1 * ? *". As for the "round" minutes thing, you can try this when building your trigger:
ITrigger trigger = TriggerBuilder.Create()
.WithIdentity(JobTrigger, JobGroup)
.WithCronSchedule(CroneExpression)
.StartAt(new DateTimeOffset(DateTime.Now,
TimeSpan.FromMinutes(DateTime.Now.Minute % 10)))
.Build();
It is not possible, see for explanation and similar issue: Quartz.net - Repeat on day n, of every m months?
it is also not possible by Cron expressions. To do this, you would need to apply some complex logic, use some operator that is not present in evaluators. Why do you need this? Would you like to combine those 2 requirements and create single complex pattern?
I'm trying to run my Spark application on Bluemix by schedule. For now I'm using scheduling of spark-submit.sh script locally on my machine. But I'd like to use Bluemix for this purpose. Is there any way to set scheduling directly inside Bluemix infrastructure for running Spark notebooks or Spark applications?
The Bluemix OpenWhisk offering provides an easy way to schedule actions run on a periodic schedule similar to cron jobs.
Overview of OpenWhisk-based solution
OpenWhisk provides a programming model based actions, triggers, and rules. For this use case, you would
Create an action that kicks off your spark job.
Use the /whisk.system/alarms package to arrange for triggers to arrive periodically according to your schedule.
Create a rule that declares that your action should fire whenever a trigger event occurs.
Your action can be coded in javascript if it's easy to kick off your job from a javascript function. If not, and you'd like your action to be implemented by a shell script, you can use whisk docker actions to manage your shell script as an action.
Using the whisk.system/alarms package to generate events on a schedule.
This page in the whisk docs includes a detailed description of how to accomplish this. Briefly:
The /whisk.system/alarms/alarm feed configures the Alarm service to fire a trigger event at a specified frequency. The parameters are as follows:
cron: A string, based on the Unix crontab syntax, that indicates when to fire the trigger in Coordinated Universal Time (UTC). The string is a sequence of six fields separated by spaces: X X X X X X. For more details on using cron syntax, see: https://github.com/ncb000gt/node-cron. Here are some examples of the frequency indicated by the string:
* * * * * *: every second.
0 * * * * *: top of every minute.
* 0 * * * *: top of every hour.
0 0 9 8 * *: at 9:00:00AM (UTC) on the eighth day of every month
trigger_payload: The value of this parameter becomes the content of the trigger every time the trigger is fired.
maxTriggers: Stop firing triggers when this limit is reached. Defaults to 1000.
Here is an example of creating a trigger that will be fired once every 20 seconds with name and place values in the trigger event.
$ wsk trigger create periodic --feed /whisk.system/alarms/alarm --param cron '*/20 * * * * *' --param trigger_payload '{"name":"Odin","place":"Asgard"}'
Each generated event will include as parameters the properties specified in the trigger_payload value. In this case, each trigger event will have parameters name=Odin and place=Asgard.
I want to schedule a job to run every 5 mins during morning (4-8)AM and evening (7-11)PM and rest of the day every 30 minutes, how can i write a cron expression for it?
I can do this by using two cron expressions for scheduling, but can i do the task using one expression?
No, I believe you'll need 2 separate CRON expressions.
The timings (or intensity) are different, so you need different expressions.
So something like the following is needed:
*/5 4-7,19-22 * * * your/command
*/30 0-3,8-18,23 * * * your/command