I have Created python azure webjob which I want to trigger twice - 1st is daily #2 am and second is every Saturday # 4 am.
How can I achieve this in a single cron expression?
Below is my current cron expression file looks like:
{"schedule":"0 0 14 * * *"}
As #ThiagoCustodio Said is correct that it is not possible to allow two schedules in one CRON expression and you have to create two Web jobs for achieving your requirement.
It can be a WebJobs or Functions in the Azure Function, two instances required for 2 different schedules.
In this SO #57211407, the user #GeorgeChen given 3 ways for implementing multiple CRON expressions in Azure Web Jobs with the practical code examples. Those ways are Multiple Static Web Job Methods in Azure Timer Triggered Function, Custom Schedule Class file, and abstract class custom scheduler in Azure Web Jobs.
Related
I have azure webjob which is basically timer trigger type. I want to run this webjob different time slots for different days.
For example, if I want to run azure webjob for Monday-Friday from 09:00pm to 06:00am following day and
if I want to run same webjob for saturday and sunday for entire day.
so, can you please let me know the CRON expression for above requirement?
In your requirement, you need to write 2 CRON expressions. CRON expressions cannot be combined, so I think creating two webjobs, each with 2 expressions, can achieve your needs.
I'm currently running 2 scripts on a weekly schedule on a raspberry pi with the following configuration:
Cron executes a python script at a fixed time weekly. This python script waits between 0 and 50 hours then runs python script A. It waits about 16 hours and runs script A again 3 more times every 8 hours (The script takes about 4x longer to run the first time). 8 hours after the 4th run it runs script B.
I would like to move my scripts to Google Cloud VM for improved reliability but running the VM 24/7 just to run 30 hours worth of computations over a 100 hour period is inefficient and expensive.
I know I can use Google Scheduler as my cron to initiate the VM weekly but I still risk letting it run up to 50 hours waiting for script A to run. I understand cron supports adding a random sleep interval as listed in the example here:
30 8-21/* * * * sleep ${RANDOM:0:2}m ; /path/to/script.php
However, from what I've discovered, Google Cloud Scheduler is limited to 60 minutes and rightfully so. In this case what are my options? Does Google Cloud Task support delayed triggering of VM (up to 50 hours)? Is this something Pub Sub would support instead?
My scripts use a python library that I don't think is compatible with Google App Engine so I would further need to figure how to trigger a specific script in the VM on trigger.
You can use Cloud Scheduler and Pub/Sub to trigger a Cloud Function that will start your VM and execute your script. If you do not want your Compute Engine instances to be running 24/7, at the end of your script you can have your Cloud Function stop your VM.
You can find how to schedule compute instances with Cloud Scheduler here and how to use HTTP functions in Cloud Functions to start and stop your Compute Engine instance [1].
Most importantly, here is the documentation on how to use Cloud Scheduler and Pub/Sub to trigger a Cloud Function [2].
[1] https://cloud.google.com/scheduler/docs/start-and-stop-compute-engine-instances-on-a-schedule
[2] https://cloud.google.com/scheduler/docs/tut-pub-sub
[3] Cloud Functions: https://cloud.google.com/functions/docs/concepts/overview
I have two pipelines (also called "build definitions") in azure pipelines, one is executing system tests and one is executing performance tests. Both are using the same test environment. I have to make sure that the performance pipeline is not triggered when the system test pipeline is running and vice versa.
What I've tried so far: I can access the Azure DevOps REST-API to check whether a build is running for a certain definition. So it would be possible for me to implement a job executing a script before the actual pipeline runs. The script then just checks for the build status of the other pipeline by checking the REST-API each second and times out after e.g. 1 hour.
However, this seems quite hacky to me. Is there a better way to block a build pipeline while another one is running?
If your project is private, the Microsoft-hosted CI/CD parallel job limit is one free parallel job that can run for up to 60 minutes each time, until you've used 1,800 minutes (30 hours) per month.
The self-hosted CI/CD parallel job limit is one self-hosted parallel job. Additionally, for each active Visual Studio Enterprise subscriber who is a member of your organization, you get one additional self-hosted parallel job.
And now, there isn't such setting to control different agent pool parallel job limit.But there is a similar problem on the community, and the answer has been marked. I recommend you can check if the answer is helpful for you. Here is the link.
I have an Azure Batch service set up with a job schedule that runs every minute.
The job manager task creates 3-10 tasks within the same job.
Sometimes, one of these tasks within the job may take extremely long to complete but usually are very fast.
In the event that one of the tasks takes long to apply, the next iteration of the job manager task does not begin in that case. It basically waits till all the tasks from the previous iteration have completed.
Is there a way to ensure that the job schedule keeps creating a version of the job every minute even if all the tasks from its previous iteration have not been completed?
I know one option is to make the job manager task create additional jobs instead of tasks. But preferably, I was hoping there is some configuration at the job schedule level that I can turn on that will allow the schedule to create tasks without the dependency of completion on the previous job.
This seems like more towards design question, AFAIK, No, the duplicate active job names should not be doable from az batch perspective. (I will get corrected if at all this is doable somehow)
Although in order to further think this you can read through various design recommendations via Azure batch technical overview page or posts like:
How to use Azure Batch in an event based design and terminate/cleanup finished jobs or
Add Tasks to a running Azure batch job and manually control termination
I think simplicity will be better like handling each iteration with unique job name or some thing of other sort but you will know your scenario better. Hope this helps.
Currently, a Job Schedule can have at most one active Job under it at any given time (link) so the behavour you're seeing is expected.
We don't have any simple feature you can just "turn on" to achieve concurrent jobs from a single job schedule - but I do have a suggestion:
Instead of using the JobSchedule to run all the processing directly, use it to create "worker" jobs that do the processing.
E.g.
At 10:03 am, your job schedule triggers to create job processing-20191031-1003.
At 10:04 am, your job schedule triggers to create job processing-20191031-1004.
At 10:05 am, your job schedule triggers to create job processing-20191031-1005.
and so on
Because the only thing your job schedule does is create another job, it will finish very quickly, ensuring the next job is created on time.
Since your existing jobs already create a variable number of tasks (you said 3-10 tasks, above), I'm hoping this won't be a very complex change for your code.
Note that you will need to ensure your concurrent worker jobs don't step on each others toes by trying to do the same work multiple times.
I am using spring cloud and have various microservices for an online shopping vendor. Everything is working as expected.
But, I got a requirement where I need to run a cron job over customer's records, get the customer's who's statement date matches the current date and calculate the rate of interest to be paid. This needs to be run every day.
I am confused about how to accommodate this cron job with MS architecture. Do I need to have another server having just this cron job?
Depending on the platform (eg: cf, k8s..) that you're orchestrating the batch-jobs in SCDF, you could write a simple Quartz based Boot Application that can interact with SCDF's REST endpoints to schedule the Task definitions defined in SCDF.
There are several online literatures on Quartz + Boot solution.
We are also working on a native scheduler integration for Cloud Foundry (via PCF Scheduler). Once it ready, you'd be able to schedule (i.e., cron-expressions) for Tasks from SCDF's Dashboard natively.
As I understand you should have one centralized supervisor of jobs, because multiplied instances can potentially run the same job at the same time.
This supervisor can be a microservice, which delegates job execution to other services via rest call or message queue, and wait for result.
It means that job supervisor becomes part of infrastructure, like message queue or database.