I have a job which need to be scheduled daily at 8am, 9am and every 15min from 10am to 4pm how can I schedule this job on azure logic apps?
You will not be able to schedule individual time in logic app.
In order to work through your requirement :
You could have the logic app triggered (scheduled) for every 15 minutes. In the logic app flow you could extract the time.
formatDateTime(utcNow(),"HH:mm")
You could compare with your schedule and run the job if the conditions met.
Pseudo Logic
if time is 8:00 AM or if time is 9:00 AM
Yes, set the flag (a boolean variable) to True
No, check if the time is between 10 AM to 4 PM
Yes , Set the flag to true
No ,Set the flag to false
if the flag is true run the job else, do nothing.
Related
Lets assume a scenario where pipeline A runs every day and pipeline B runs once in every month and it is dependent on pipeline A (pipeline B should trigger after successful completion of pipeline A).
Using scheduled trigger, we cannot have hard dependencies between 2 pipelines, where as with tumbling window, we cannot exactly specify the day which the pipeline B should run(it has only two options, minutes and hours where as scheduled trigger has months and weeks also)
Both the triggers has its disadvantages with respect to this scenario.
What could be the best possible solution for this scenario?
You can Run Pipeline A everyday, and have an IF check that checks if its a specific date today, then run Pipeline B if TRUE and nothing if FALSE.
For the settings of If Condition, you can use this as variable, if you want to run it every 1st of every month:
#Contains('01',Substring(formatDateTime(utcnow()),8,2))
I have a pipelines in Azure DataFactory which is scheduled to run hourly.
Since every schedule task will have start time and end time (e.g. 1am - 2am) to copy files within this interval. I would like to know if old task overrun like finishing at 2:15am, what will be behaviour of next task?
(a) running task with start time and end time 2am-4am
(b) running task with start time and end time 3am-4am
My aim is to make sure no missing copying files.
I have tested this in my ADF.
Conclusion:
The previous pipeline's status won't affect the next task start time. So in your case, if you the previous pipeline started at 1am and finished at 2:15am, your next task will still start at 2am.
My test:
I create a Schedule trigger which runs every 3 min. My pipeline runs about 6 min.
Monitor pipeline runs and trigger runs:
My first task ends at 3/4/21, 3:32:41 PM, and the next task starts at 3/4/21, 3:30:00 PM. So if old task overrun, it won't affect the next task start time.
With an Azure Data Factory "Tumbling Window" trigger, is it possible to limit the hours of each day that it triggers during (adding a window you might say)?
For example I have a Tumbling Window trigger that runs a pipeline every 15 minutes. This is currently running 24/7 but I'd like it to only run during business hours (0700-1900) to reduce costs.
Edit:
I played around with this, and found another option which isn't ideal from a monitoring perspective, but it appears to work:
Create a new pipeline with a single "If Condition" step with a dynamic Expression like this:
#and(greater(int(formatDateTime(utcnow(),'HH')),6),less(int(formatDateTime(utcnow(),'HH')),20))
In the true case activity, add an Execute Pipeline step executing your original pipeline (with "Wait on completion" ticked)
In the false case activity, add a wait step which sleeps for X minutes
The longer you sleep for, the longer you can possibly encroach on your window, so adjust that to match.
I need to give it a couple of days before I check the billing on the portal to see if it has reduced costs. At the moment I'm assuming a job which just sleeps for 15 minutes won't incur the costs that one running and processing data would.
there is no easy way but you can create two deployment pipelines for the same job in Azure devops and as soon as your winodw 0700 to 1900 expires you replace that job with a dummy job using azure dev ops pipeline.
When I go to configure a Schedule in the Azure management console, I'm only given the option of scheduling with an absolute end date/time (or never ending) and an interval.
So I can't, from this UI, schedule a job to every 30 minutes run every day from 8:00 AM to 6:00 PM only (i.e. don't run from 6:01 PM to 7:59 AM). Windows Task Manager and all other schedulers (cron, quartz) I've used before support the behaviour I want.
Is type of schedule supported at all in Azure, e.g. through the API or a hackish use of the Portal HTTP/JSON interfaces?
You can use the built-in scheduling which is more flexible than the Azure one.
You can learn more about how that works from this blog post http://blog.amitapple.com/post/2015/06/scheduling-azure-webjobs/
The summary: create a file called settings.job that contains the following piece of json
{"schedule": "cron expression for the schedule"}
in your case the cron expression for "every 30 minutes from 8am to 6pm" would be 0,30 8-18 * * *
so the JSON you want is
{"schedule": "0,30 8-18 * * *"}
Keep in mind that this uses the timezone of the machine, which is UTC by default.
This is something you need to implement in your WebJob. I have a similar issue in that I have WebJobs with complex schedules. Fortunately it isn't hard to implement.
This snippit gets your local time (Eastern from what I can tell) from UTC which everything is Azure is set to. It then checks if it is Saturday or Sunday and if it is exits out (not sure if you need this). It then checks whether it is before 8AM or after 6PM and if it is exits out. If it passes both those conditions the WebJob runs.
//Get current time, adjust 4 hours to convert UTC to Eastern Time
DateTime dt = DateTime.Now.AddHours(-4);
//This job should only run Monday - Friday from 8am to 6pm Eastern Time.
if (dt.DayOfWeek == DayOfWeek.Saturday || dt.DayOfWeek == DayOfWeek.Sunday) return;
if (dt.Hour < 8 || dt.Hour > 16) return;
//Go run WebJob
Hope this helps.
I am dealing with a workflow where I need to start three processes. I have the first process which is to be scheduled at the beginning of every hour and the rest two at 45th minute of every hour and the 52nd minute of every hour.
But Instead of making the client schedule two different jobs on their server what I would rather want is to have just one job configured to run in the beginning of every hour which does a bunch of stuff and then starts these cron jobs at their respective times. i.e. 45th minute and 52nd minute of the hour.
Is there any way to do this.
I don't have any experience with shell scripting and always schedule cron jobs manually on cron-tab.
Thanks!