In our requirement we need to connect to ARGO instance and take necessary actions on the jobs.
So we are consuming ARGO java apis to connect to argo instance and performing the actions like suspend,resume,retry....
But i am not able to find apis for stopping/terminating cron jobs.
So how can i stop/terminate the cron jobs ?
There is no direct API for stopping/terminating cron jobs in ARGO. However, you can achieve the same using the ARGO API for getting the list of jobs and then deleting the job using the API for deleting a job.
Related
I have this .NET long running API process/function that usually runs 30 mins in one execution that is hosted in AKS. This API is usually executed from the users coming from the front end of the app.
Due to concurrent executions from users, this is causing exhaustion of the app so I'm planning to implement a some sort of a queueing mechanism with the help of a scheduler(s).
What possibly is applicable Azure service that can execute my API in AKS on a scheduled basis (let's say every minute) and possibly check the database for some flagging values.
I need a way to check the table for some flagging value if there a currently running process or its been completed so it can process the next one, otherwise ignore the call until current on is complete.
I was looking into Azure Web Apps, Web Jobs or Batch Jobs but kinda confused which is applicable with my case.
Please advise thank you in advance.
There are a couple of options here.
Hangfire
Hangfire is an open-source library that can run background jobs in queues. In your case, you can enqueue each request from the client in a queue. Then Hangfire server will process them one by one (even with retry if the job fails). Hangfire supports SQL Server or Redis. You can query the storage to see the status of the queued jobs.
Hangfire can also run scheduled jobs, which will take care of that only one job run at a time.
Azure Service Bus
A more expensive option is to use Azure Service Bus for your queueing capability. For scheduled jobs, you can use AKS CronJobs but you will
implement the check yourself to see if there is a job already running.
Overall, I would recommend Hangfire, which can meet your requirements and is cheaper.
When deploying a dataflow job with terraform it does not drain the old job.
Is there a possibility to automatically drain it and the deploy?
I think it's not the responsability of Terraform to deploy custom Dataflow jobs (not Dataflow templates).
It's more the responsability of the deployment task proposed by a CI CD pipeline.
In this case there is no possibility to automatically drain and deploy a new version of the job.
You have to develop your own script (Shell for example) to do that and apply your strategy.
For example, a Dataflow job can be drained by a gcloud command :
gcloud dataflow jobs drain JOB_ID
We have a requirement for generating reports based on a schedule. These schedules can be configured by the users for a report. Initially We were planning to use Quartz in a worker role and Create jobs based on the user input.
I'm not sure whether the Azure Scheduler can be used for this because I feel any jobs that could run at the application level could be configured in Azure but not the user jobs.
Please validate and let me know if there are any resources which I could look at.
How about using Azure Web Jobs?
(https://learn.microsoft.com/en-us/azure/app-service/web-sites-create-web-jobs)
Can you please explain more about the nature of reports you're running?
If I want to schedule a task to run say once every 30 minutes. I could do this with a basic timeout or use a node module like node-schedule.
But If I deploy my app to the cloud, such as Amazon AWS or Azure, and scale the instances to say 10, will this task then be scheduled to run 10 times, one for each instance? How can I avoid this, or am I thinking about how cloud instanced work in the wrong way.
If you want to use an Azure component, you could use the Azure scheduler:
Create jobs that run on your schedule
Azure Scheduler lets you create
jobs in the cloud that reliably invoke services inside and outside of
Azure—such as calling HTTP/S endpoints or posting messages to Azure
Storage queues. You can choose to run jobs right away, on a recurring
schedule, or at some point in the future.
Azure Scheduler
It entirely depends on your code but I'd imagine that yes it will run on each instance, probably slightly out of sync too.
This article describes quite nicely the problems with scheduled tasks:
http://dejanglozic.com/2014/07/21/node-js-apps-and-periodic-tasks/
My advice would be to try and avoid scheduled tasks as much as possible.
Is there a way to use the windows scheduled task to kick off a url or a exe on a schedule?
Can I write a program as an exe then create a Azure VM then RDP into the Azure VM and hook it up to windows task scheduler?
Azure does have a scheduler now.
It allows invoking a Web Service over HTTP/s and post a message to a Windows Azure Storage Queue. It's very new but it can be free if you do not need the scheduler to be executed often. Otherwise it's a small monthly fee which come with scheduled task that can be up to every minute.
Things got much easier lately, please see this link https://azure.microsoft.com/en-us/services/scheduler/ to create a scheduled job in the new Azure Scheduler. There is a basic Free tier as well as some paid options but I think this is exactly what many of us were looking for. It is an excellent solution for triggering URLs with GET,POST,PUT,DELETE requests.
Just follow the simple instructions. Starting by selecting "Scheduler" from the Azure dashboard app menu:
Today the scheduler has been Azure Logic Apps:
https://learn.microsoft.com/en-us/azure/logic-apps/logic-apps-overview
If you are looking for something like a cron job (which is a job, that is being run at specific time again and again), then check out Azure Functions:
https://learn.microsoft.com/en-us/azure/azure-functions/functions-overview
Google Azure Storage Queues. They allow you to schedule jobs that will run at a later date. You can even specify when the job should run.