How to schedule automatically execute runbooks based on minutes instead of hours in Microsoft azure automation account?
hm, I thought this was possible, but it appears not. you can create multiple hourly schedules for this (if you want it to start every 10 minutes - create 6 hourly schedules), you can create an azure function timer triggered and just start your runbook with that (more hacky, I guess) or create an external event that would start the runbook using the Azure API or, perhaps, via a webhook
Related
I need to setup an alert system if my Azure Datafactory pipeline runs for more than 20 minutes. The alert should come while the pipeline is running and the duration passes 20mins, not after the completion of pipeline. How can I do this? I think this can be done using Azure function but I am not familiar with it so I'm in search for a script for the same.
Yes, azure function is a solution to acheive your requirement.
For example, if you are using Python. You need an azure function that runs periodically to monitor the status of the pipeline. The key is the duration time of the pipeline. pipeline is based on activities. You can monitor every activity.
In Python, This is how to get the activity you want:
https://learn.microsoft.com/en-us/python/api/azure-mgmt-datafactory/azure.mgmt.datafactory.operations.activityrunsoperations?view=azure-python#query-by-pipeline-run-resource-group-name--factory-name--run-id--filter-parameters--custom-headers-none--raw-false----operation-config-
The below is to get the duration time of azure datafactory activity:
https://learn.microsoft.com/en-us/python/api/azure-mgmt-datafactory/azure.mgmt.datafactory.models.activityrun?view=azure-python#variables
(There is a variable named duration_in_ms, you can use this to get the duration time of the activity run.)
This is use Python to monitor pipeline:
https://learn.microsoft.com/en-us/azure/data-factory/monitor-programmatically#python
You can create a azure function app with a timetrigger to monitor the azure datafactory activity. This is the document of azure function timetrigger:
https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-timer?tabs=python
The basic idea is put the code that monitor the pipeline whether run more than N minutes in the logic body of azure function timetrigger. And then use the status of the azure function to reflect whether the pipeline running time of azure datafactory exceeds N hours.
Then use the alarm event of the azure function. The alarm events supported by azure for the azure function are as follows: (You can set an output binding of your azure function.)
In azure portal, you can find the alert in this place:
(Select Email/SMS message as the action type and give it your email address.)
We are using Azure SQL in our development environment and to reduce cost, we have created an Azure Automation runbook to scale down/up the Azure SQL instances. During working hours the databases are running with P2 and outside working hours they fall back to standard. This works fine, but there is a catch...
After a few days the runbook stopped working, because "the quota for the monthly total job run time has been reached". After some investigation it seems that the runbook takes about 30-45 minutes to complete.
I suspect that the runbook waits until the database is actually in the desired state. When I run the Set-AzureRmSqlDatabase from the command-line, then it returns immediately. How can I reproduce this with Azure Automation?
I am using the Scale an Azure SQL DB to a specified edition and tier from the Automation Gallery. Maybe I could create a custom script, but I would rather use standard scripts...
I have created an azure service which is responsible for below task:
(1) Access the blob containers and download the files from there.
(2) Extract some data from downloaded files
(3) Stored the extracted data to an Azure SQL Server
I want to run this processing after every 7 days. Is there a way to achieve this? or can I use any other option than cloud service to achieve the above goal?
I would recommend you to use Azure Function as its Timer-based processing (Timer trigger) feature is able to fulfill your requirements.
Timer triggers call functions based on a schedule, one time or
recurring.
Reference: Azure Functions timer trigger, Azure Functions Pricing
Another great advantage of using Azure Function for your scenario is its pricing model.
Azure Functions consumption plan is billed based on resource
consumption and executions.
Consumption plan pricing includes a
monthly free grant of 1 million requests and 400,000 GB-s of resource
consumption per month.
Certainly not natively with the Cloud Service itself. I mean, you can obviously code it so it performs some task(s) and sleeps for 7 days, but you will pay for all of that time, that makes no sense
You can use Azure WebJobs, Functions and Scheduler for this purpose, or you can create a PowerShell\Cli or something else cron task\task scheduler to turn on your Azure Cloud Service, wait for it to finish processing and turn it off. But that seems like a lot of extra effort, I'd rather go with Scheduler or Functions.
Is there a way we can specify a Complex Schedule for an Azure Automation Runbook?
Currently we can only configure if the Runbook needs to run Hourly,Daily,Weekly / Monthly. I want to set up something like "Run from Tuesday to Saturday, Daily at 1 PM".
Only way I can think of right now is to code it inside the RunBook.
Please let me know if there's any other way.
Thanks
Automation offers weekly and monthly schedules in the new Azure portal. This service announcement talks a bit about how you can use weekly/monthly schedules to do what you are looking for. You can also use the New-AzureRMAutomationSchedule cmdlet to create weekly schedules and specify the days (use the ByWeekly parameter set).
You have to write custom powershell script within your runbook and schedule your runbook to run every hour or probably everyday at 1PM.
The custom powershell script will have code which checks if a day is NOT Sunday, Monday and Saturday!
Is there a way to use the windows scheduled task to kick off a url or a exe on a schedule?
Can I write a program as an exe then create a Azure VM then RDP into the Azure VM and hook it up to windows task scheduler?
Azure does have a scheduler now.
It allows invoking a Web Service over HTTP/s and post a message to a Windows Azure Storage Queue. It's very new but it can be free if you do not need the scheduler to be executed often. Otherwise it's a small monthly fee which come with scheduled task that can be up to every minute.
Things got much easier lately, please see this link https://azure.microsoft.com/en-us/services/scheduler/ to create a scheduled job in the new Azure Scheduler. There is a basic Free tier as well as some paid options but I think this is exactly what many of us were looking for. It is an excellent solution for triggering URLs with GET,POST,PUT,DELETE requests.
Just follow the simple instructions. Starting by selecting "Scheduler" from the Azure dashboard app menu:
Today the scheduler has been Azure Logic Apps:
https://learn.microsoft.com/en-us/azure/logic-apps/logic-apps-overview
If you are looking for something like a cron job (which is a job, that is being run at specific time again and again), then check out Azure Functions:
https://learn.microsoft.com/en-us/azure/azure-functions/functions-overview
Google Azure Storage Queues. They allow you to schedule jobs that will run at a later date. You can even specify when the job should run.