I am trying to copy a file data from Azure Blob storage to Azure SQL DB just for my learning. I cannot able to create the linked service for Azure SQL db destination as it is giving the error. I can able to connect fine from my local SSMS to the Azure SQL server but not from AZURE data factory. I turned on Allow access to Azure services. I am using the default integration runtime (AutoResolveIntegrationRuntime). I also did Add client IP by adding my current IP address to the rule list
Try using an Azure Integration Runtime with the same region as the SQL server. Sometimes the auto resolve cannot reach the sql server.
Hope this helped!
Related
A client of mine needs to join tables from his Azure SQL financial data mart with external tables built upon a Data Lakehouse (Parquet files) in Azure Synapse Analytics.
I was wondering if it's possible to create a database link within a Azure SQL database accessing a Azure Synapse Analytics Serverless (on-demand) SQL Pool.
Yes, it’s possible open the Integrate hub, and select and add a link connection.
Select test connection and make sure to check whether SQL firewall rules are properly configured or not.
Reference:
Get started with Azure Synapse Link for Azure SQL Database (Preview) - Azure Synapse Analytics | Microsoft Docs
I am trying to create an ADF Linked Service connection to a Synapse Link Serverless SQL Pool connected to ADSL Storage. I can successfully get a connection but when I try and use a dataset to access the data I get a permission issue.
I can successfully access the data via Synapse studio :
This is the error I get when I use the data set in ADF.
I can also look at the schemas in SSMS , where they appear as External tables. But get a similar credential error at the same point.
Has anyone come across this issue please ?
There are a few pieces of information you haven’t supplied in your question but I believe I know what happened. The external table worked in Synapse Studio because you were connected to the Serverless SQL pool with your AAD account and it passed through your AAD credentials to the data lake and succeeded.
However when you setup the linked service to the Serverless SQL Pool Im guessing you used a SQL auth account for the credentials. With SQL auth it doesn’t know how to authenticate with the data lake so looked for a server scoped credential but couldn’t find one.
The same happened when you connected from SSMS with a SQL auth account I’m guessing.
You have several options. If it’s important to be able to access the external table with SQL auth you can execute the following to tell it how to access the data lake. This assumes the Synapse Workspace Managed Service Identity has Storage Blob Data Reader or Storage Blob Data Contributor role on the data lake.
CREATE CREDENTIAL [https://<YourDataLakeName>.dfs.core.windows.net]
WITH IDENTITY = 'Managed Identity';
Or you could change the authentication on the linked service to use the Managed Service Identity.
Stored procedure in on-premises SQL Server is used to call another app through 'xp_cmdshell'.
Now I am trying to do the same on Azure SQL Database. Of course 'xp_cmdshell' is not supported there, so I need workaround.
I was thinking of using Azure Functions as a bridge between Azure SQL Database and app on VM.
Is it possible to call Azure Function from Azure SQL Database using
stored procedure?
Is it possible to call app on Azure VM from Azure
Function?
Not at the moment. The workaround is to create a Timer-based Azure Function that polls the data every "x" minutes or seconds.
As long the app on Azure VM expose an HTTP endpoint, and you configure the network and the firewall correctly, yes.
The best practice is to use Azure Automate to run tasks on the VM then using a pipeline in Azure Data Factory you can invoke the runbook using a webhook
Not sure if this is an option? Ideally looking at something like an openrowset query from a stored procedure within SQL server or a web job.
If so, are there any resources available to help learn how to make SQL Server and Cosmos play nicely together?
If your SQL Server in Azure is managed by you on a VM (IaaS) you should be able to add the Cosmos DB as "SQL Server Linked Server" e.g. by adding as ODBC Source.
Then it's also optional if you link the Cosmos DB PaaS service to your Vnet or access it via public endpoints.
Not 100% sure if this will also succeed with Azure managed instances (which is quite new in GA).
Generating output to a Azure SQL database is supported, but I was shocked when I found that the portal does not allow to specify a SQL Server database running on a VM. Is not this supported?
We need to store lots of data coming through the ASA jobs, and use SQL Jobs, that's why we were planning to use a SQL Server VM.
Thanks!
You cannot configure the SQL Database running on VM as an output to the ASA job.
However, Azure provides SQL services with 2 variants
Microsoft Azure SQL Database (Azure SQL Database) as PaaS
where lower stack is managed by Microsoft Azure and billed as pay-as-you-go model.
and
SQL Server in Azure Virtual Machine (VM) as IaaS where user owns the VM and make any changes, including licences for the SQL database.
the Microsoft Azure SQL Database provided as PaaS is configurable as
ASA output.
one idea might be to create and Event Hub output for the ASA and then consume it from there using any sort of application to write into an IaaS SQL DB. The application that consumes the data can also be hosted as a Web App as well.
Hope this helps.