I want to run Azure Durable Functions on premises. I have set up a durable task MSSQL provider already.
Now, I want a timer-triggered durable client and it is achievable with the [TimerTrigger] attribute. I also can run it locally by using "AzureWebJobsStorage": "UseDevelopmentStorage=true".
I have a question: what are the options for web jobs storage for production besides Azure?
If you have to run it on-prem one neat option is to use an existing on-prem SQL Server and you can use this package to integrate the durable task with SQL Server
https://github.com/microsoft/durabletask-mssql
Related
I am trying to create a scheduled nightly SSIS package that does the following:
Execute an Azure app insights query
Dump the data into an on-prem database server
Is there a way to do this directly in SSIS? I have already installed the Azure Feature Pack for Integration Services (SSIS). I see a bunch of connection managers like AzureSubscription, AzureStorage, etc.
At the company that i work, they deploy applications to azure using Azure App Services.
This involves creating the App service manually, and setting up the pipelines manually.
My question is if there is another way to host and run applications in Azure without using App Services?
The reason for this is that i don't like the manually work when setting up a app service and all it's configuration.
Any suggestions?
What you actually need is setting up CI/CD pipelines for your application to create resource(AppService in this case) and deploy on them.
If you are new to Azure Devops, i would highly recommend to explore Devops starter service to deploy your application with few clicks and see how it creates resources and pipelines automatically
On the 2nd question, there are many compute options available on Azure such as Virtual Machines , AKS (Containers Orchestration) , Container instances etc.
You can explore those compute options using the decision tree here
I have a console application connecting to database and executing DML transactions. This console application is currently scheduled using windows task scheduler. I am planning to migrate this to Azure.
Which is the recommended strategy ?
Should this be moved as Azure webjobs or function apps ?
Since you already have a console project, then it's more easier to use Azure Webjobs to achieve that.
To create a webjob, just create a .zip file which includes the .exe / .dll and other necessary files, then upload to azure. For schedule, please refer to Create a scheduled WebJob. For more details, you can refer to this doc.
Note: there're some limitation of azure webjobs / azure function, see Azure-Web-App-sandbox. But if you only need to connecting to database and executing DML transactions, you can ignore the limits.
And yes, you can also do this via azure function, but since you already have a console project, it's easier to use webjobs.
IN Azure Data Factory, is it possible to use one Integration Run time to connect two different On-Premise data sources?
Scenario:
I have created one self hosted Integration Runtime installed in Virtual Machine for DB2 Database which is On-Premise DB.
I wanted to add one more On-Premises DB which is SQL Server.
Is it possible to use the existing Self Hosted Integration Runtime for SQL Server On Prem DB?
I have tried connecting to existing Self Hosted Integration Runtime in Linked Service. The test connection is getting failed.
I know, some where access privileges required for SQL Server DB either from VM or from the SQL Server to make the connectivity possible via existing Integration Runtime.
Connectivity to SQL Server DB is getting failed, while I use the existing IR, which is already used for DB2.
Yes, you can.
You can find this in this document Considerations for using a self-hosted IR:
A single self-hosted integration runtime can be used for multiple on-premises data sources. A single self-hosted integration runtime can be shared with another data factory within the same Azure Active Directory tenant. For more information, see Sharing a self-hosted integration runtime.
When you want add a another on premise DB, you can try like this:
New link service:
Add another on premise DB:
Hope this helps.
Yes you can reuse self-hosted IR.
Probably issue with connectivity lies somewhere else.
You can test this by logging into that VM via RDP and running tests either with SSMS to test connectivity or run simple PowerShell command to test network
Test-NetConnection "<server_address>" -port 1433
Yes, you can. Note that adding more nodes as part of self-hosted IR (integration runtime) is part of highly available and making sure that there is no SPOF (single point of failure) with one on-premise data gateway.
This is no relation with the number of on-premise data sources which can be connected from services launched in Azure.
I have already tried to deploy SSIS using AzureVM and it's working fine for us. Just want to explore other options.
Is it possible to deploy SSIS on Azure Sql Service without using AzureVM? If yes, then provide some guidance.
How to connect local (on-premises) database (For example. Oracle) from Azure SQL using SSIS without using AzureVM?
No, this requires an Azure VM or an on premise installation of SSIS- SSIS as a Service is not an Azure offering at this time.
Azure SQL Database won't allow for Linked Servers and, elastic query may not fit your use case; It may not be possible to do from Azure SQL. However, you could try defining one data source for SQL Azure and one for your on-prem database (or even Oracle) within SSIS and run your report on the data that way. More about establishing data sources/connections.
Now you can deploy your SSIS package on Azure by creating Integration Run-time in Data factory which will create a SSISDB and under SSISDB, Integration Service Catalog will be available.
SSIS Project deployment is available as of now on Integration Service Catalog.
For more details and steps, click below link-
deploy-sql-server-integration-service-packages-to-azure-using-azure-data-factoryv2