I've got the Azure SQL procedure for migrating data from table A to table B. And I need to run this procedure at 15 min intervals.
I thought I could use Azure webjob for it, but I don't know how to call my azure sql procedure.
Are there any available schedulers in azure sql or how is it possible to make add Azure sql procedure to the Azure scheduler?
One option for scheduling Azure SQL Database stored procedures would be to leverage Azure Automation. There is a blog post which explains how to run TSQL with Azure Automation here. Azure Automation has a scheduler built in, however if it doesn't meet your needs, you could leverage Azure Scheduler to schedule the Automation Runbook instead.
Related
I have an Azure SQL database with some fact tables. I have an SSAS Tabular Cube on an Azure Analysis Services database running on the same subscription. The cube's source database is the Azure SQL database.
I have an elastic job which contains steps to calculate the fact tables in Azure SQL. It does this daily. I would like to have an additional step that would invoke the SSAS Tabular Cube to process with the latest information.
In an on-premise version - I could use SQL Agent to call a powershell script. Azure SQL does not have an agent - only elastic jobs. So it needs to be something that I can call using a T-SQL script. i.e. the T-SQL could be a script that calls a powershell script but I'm not sure how that would work given that the script would need to be saved somewhere and there is nowhere to store scripts in Azure SQL.
Does anyone know if I can invoke the tabular cube processing command from Azure SQL using a T-SQL script? Or if that isn't possible, would I be able to schedule the Azure SSAS Cube to process at a certain time every day? Or is there some other Azure method I could use?
NOTE: Please do not suggest I switch to a virtual machine or managed instance - we need to use Azure SQL. I am willing to use other Azure technology to achieve the same result but I can't change the source database from Azure SQL.
Any and all help appreciated.
No. Elastic Database Jobs just run TSQL. Use Azure Automation instead. You can schedule the Azure Automation job, and to coordinate the Elastic Database Job and the AAS processing you can kick-off an Elastic Database Job by calling a stored procedure.
Or manage the whole process in an Azure Data Factory Pipeline.
Is it possible to trigger Azure Functions or AppService webapp whenever an insert operation is performed against a table on Azure SQL MI?
if not, is there a way to trigger applications outside Azure SQL rather than using LogicApp? I want to avoid LogicApp because it requries using one more application, and it is still using polling.
Link below said it is not for Azure functions
https://feedback.azure.com/forums/355860-azure-functions/suggestions/16711846-sql-azure-trigger-support
Link below suggests using LogicApp.
Trigger Azure Function by inserting (adding) new row into table, SQL Server Database
Today, in Azure SQL, there is no such possibility. The closest option is to create a Timer Trigger Azure Function that checks if there has been any changes in the table you want to monitor (using Change Tracking, for example).
If you are using Azure SQL MI instead, you could create a SQLCLR procedure that calls an Azure Function via an HTTP request or, another option, via Azure Event Hubs or Azure Event Grid
There have been several feature requests for triggering Azure functions based on changes to at Azure SQL database. For example:
https://github.com/Azure/azure-functions-python-worker/issues/365
It seems that they are not prioritizing it, since it is possible to implement this functionality using logic apps.
Stored procedure in on-premises SQL Server is used to call another app through 'xp_cmdshell'.
Now I am trying to do the same on Azure SQL Database. Of course 'xp_cmdshell' is not supported there, so I need workaround.
I was thinking of using Azure Functions as a bridge between Azure SQL Database and app on VM.
Is it possible to call Azure Function from Azure SQL Database using
stored procedure?
Is it possible to call app on Azure VM from Azure
Function?
Not at the moment. The workaround is to create a Timer-based Azure Function that polls the data every "x" minutes or seconds.
As long the app on Azure VM expose an HTTP endpoint, and you configure the network and the firewall correctly, yes.
The best practice is to use Azure Automate to run tasks on the VM then using a pipeline in Azure Data Factory you can invoke the runbook using a webhook
I have some T-sql scripts which generate some data and we manually update them into the excel spreedsheet, we need a way to push this into azure sql database, from a job so that we can access them from there and remove the manual process of uploading the information to the azure sql database every time. What is the best way to do this?
I assume you are trying to move data from an on prem server to Azure. The simplest method may be Azure Data Sync.
You could load your data from your queries into an on prem table which syncs to Azure.
On all your SQL Server instances, you can create a Linked Server to one Azure SQL Database. Once the linked server is created you can directly insert on Azure SQL Database from your on-premises SQL Server instances.
Here is how you create the Linked Server.
Below image shows how you insert data on Azure SQL Database using the linked server.
For detailed steps, you can visit this tutorial.
I think you can think about Azure Data Factory.
Azure Data Factory Copy Active can help you use T-sql scripts to move data to another Azure SQL database.
For more details, please the Azure tutorial:Copy multiple tables in bulk by using Azure Data Factory.
When the pipeline created, you can trigger and monitor the pipeline runs.
Trigger the pipeline on a schedule:
You can create a scheduler trigger to schedule the pipeline to run periodically (hourly, daily, and so on). In this procedure, you create a trigger to run every minute until the end date and time that you specify.
Please see: Trigger the pipeline on a schedule.
This can help you push the data to Azure SQL Database automatically.
Hope this helps.
you can try the SSIS package? which automates the process of data upload data into azure sql database.... i have not used ssis for Azure but to sink data from csv/xls/xlsx into ms sql server database,,I refered this article which can be helpful in anyway
I need to upload WEB API response files into Azure Data Lake.
Then I have to dump those files into Azure SQL tables.
Above both processes must be scheduled to execute on hourly basis.
Should I use Azure Web Jobs or Azure Function.
Azure Data Factory is probably the better mechanism to drive this recurring hourly pipeline. More details here.
https://learn.microsoft.com/en-us/azure/data-factory/data-factory-scheduling-and-execution
If you are running Azure Functions on Consumption plan, the function call must complete within 5 minutes, which might be not enough for big data sets.
For the rest Functions and Web Jobs are similar for your scenario. Functions are actually running on top of Web Jobs. And if you don't need any advanced features of Functions (e.g. bindings), I would go for a Job.