I have successfully setup a Linked Service in Azure Data Factory that uses a Key Vault for the connection string which includes the user/pwd and connects to the Azure SQL DB as desired. However, I can only do this when I use the "admin" account. The string below works.
Server=tcp:database1.database.windows.net,1433;Initial Catalog=DB;Persist Security Info=False;User ID=Admin;Password=Pa$$w0rd;MultipleActiveResultSets=False;Encrypt=True;TrustServerCertificate=False;Connection Timeout=30;
I created a new login/user and granted the necessary permissions. I know because I can connect using the new login via remote SSMS or by adding the credentials directly in the linked service in Azure. (e.g. hard coding the user/pwd in the connection string in the linked service)
Unfortunately, when I switch to using the key vault connection string, I get the generic SQLErrorNumber 18456 for the newly created user. I know the credentials are correct, I know I can connect via the Key Vault (when using the elevated admin account), I just cannot use the Key Vault connection string when using the new user.
Server=tcp:database1.database.windows.net,1433;Initial Catalog=DB;Persist Security Info=False;User ID=Username;Password=Pa$$w0rd;MultipleActiveResultSets=False;Encrypt=True;TrustServerCertificate=False;Connection Timeout=30;
If using Azure SQL with Data Factory look at using Managed Service Identity That way you'd add the name of the Data Factory directly to the SQL DB and no need for a username and password. Plus can assign permissions directly on the user.
Only downside is if DataFactory get's wiped out and redeployed the user will need to be dropped and recreated since it is using a thumbprint to recognize the Identity, this isn't the case with all resources and MSI auth.
For your specific case check to make sure the SQL server is allowing Azure Services and resource to Access the sever by going to "Firewalls and virtual networks" and make sure it is turned on:
Related
I have followed the following for How to reference Key Vault connection strings from an Azure App Service.
https://davecallan.com/how-to-reference-key-vault-connection-strings-from-an-azure-app-
service/
The connection string has a name of "sqlconnection" in the App Service Configuration. I then deleted this entry from the Configuration Json file and restarted the Web app.
I then try to connect but it gives a database error with no details.
Any ideas on this ?
I tried to reproduce the same in my environment like below:
I created a key vault with Azure role-based access control (RBAC) like below:
Make sure to add Key Vault Administrator role in Access control like below:
Generate a key vault secret under setting. While generating secret of type as Manual name and set the value to the current SQL Database connection string stored in the Application web.config
Try to use Secret value connection string of your sql database like below:
Then you can able to see Secret Identifier like below:
In your web app -> under setting click configuration -> click new connection string
Type the name of the connection string and set the value #Microsoft.KeyVault(SecretUri=VALUE), where the VALUE
In Identity under setting -> make sure to add system assigned status as on like below:
Click on the "Role Assignments" button and then Add keyvault secrets user role like below:
As you are facing database error with no details you can check the Log4net errors under Portals_default\Logs.
Reference:
Connection strings and app settings not working properly on Azure App Service · Issue #4227 · dnnsoftware/Dnn.Platform · GitHub
I have an Azure SQL server and database which have MFA login and I am the admin. But when I try to establish a connection via a new linked service from ADF to this database using System Managed Identity option, it throws error -
"Cannot connect to SQL Database. Please contact SQL server team for further support. Server: 'Server details', Database: 'database name', User: ''. Check the linked service configuration is correct, and make sure the SQL Database firewall allows the integration runtime to access.
I have already given contributor role access to ADF in SQL database using system managed Identity. Also, I have tried to access this database using Autoresolve runtime and azure runtime. But still the error is coming.
It sounds like you are missing the user creation and role assignment within the SQL database:
Connect to the database with your account and create an account for the data factory:
CREATE USER [<datafactory-name>] FROM EXTERNAL PROVIDER;
Then grant it the required role for your task:
ALTER ROLE [<roleName>] ADD MEMBER [<datafactory-name>]
Some available role names are:
db_accessadmin
db_backupoperator
db_datareader
db_datawriter
db_ddladmin
db_denydatareader
db_denydatawriter
db_owner
db_securityadmin
public
I created Azure SQL database in portal and created linked service in azure data factory with managed identity authentication I got below error:
I followed below procedure to resolve this:
I turned on the managed identity of data factory
I set admin for azure SQL database:
Login with Admin to sql database Create User username as data factory name using below code:
CREATE USER [DATAFACTORY NAME] FROM eXTERNAL PROVIDER
Added rules to the user using below code:
ALTER ROLE db_datareader ADD MEMBER [DATA FACTORY NAME];
I tested linked service again, tested successfully
It worked for me, once check from your end.
I have created a simple SSIS project and in this project, I have a package that will delete a particular file in Downloads folder.
I deployed this project to Azure. And when I am trying to execute this package using Azure Data Factory then the pipeline fails with an empty error (I am attaching the screenshot here).
enter image description here
What I have done to fix this error is:
I have added self-hosted IR to Azure-SSIS IR as the proxy to access the data on-premise.
Set the ConnectByProxy as True.
Converted the project to Project Deployment Model.
Please help me out to fix this error and if you need more details then just leave a comment.
Windows Authentication :
To access data stores such as SQL servers/file shares on-premises or Azure Files, check the Windows authentication check box.
If this check box is selected, fill in the Domain, Username, and Password fields with the values for your package execution credentials. The domain is Azure, the username is storage account name>, and the password is storage account key> to access Azure Files, for example.
Using the secrets stored in your Azure Key Vault
As a substitute, you can leverage secrets from your Azure Key Vault as values. Select the AZURE KEY VAULT check box next to them to do so. Create a new key vault connected service or choose or update an existing one. Then choose your value's secret name and version. You can pick or update an existing key vault or create a new one when creating or editing your key vault connected service. If you haven't previously done so, allow Data Factory managed identity access to your key vault. You may also directly input your secret in the format key vault linked service name>/secret name>/secret version>.
Note : If you are using Windows Authentication, there are four methods to
access data stores with Windows authentication from SSIS packages
running on your Azure-SSIS IR: Access data stores and file shares with
Windows authentication from SSIS packages in Azure | Docs
Make Sure it Falls under one of such methods, else it could potentially fail at the Run Time.
When I develop for Azure I usually start copying in some keyvault client code so only keyvault urls will be in my settings file, no secrets can ever end up my git repositories.
After starting to make Azure functions I realized that it was not possible to do this for the trigger connection string for e.g. service bus or blob storage.
The recommended approach seems to connect the app to keyvault directly in Azure when deployed, and just manage secrets locally in Secret Manager, like suggested in
this article
I am not developing alone, so while I am not adverse to using a tool like Secret Manager, I need to still have my offline secrets connected to the Azure keyvault! If others change anything.
Question: How do I manage secrets offline in a way that is synchronized with Azure keyvault?
it was not possible to do this for the trigger connection string for e.g. service bus or blob storage.
In short, it's possible.
Here are steps you could follow and refer to the detailed article.
1.Add a System Assigned Managed Identity to the Azure Function.
2.Go to the Access Control section of your Key Vault and click on Add a role assignment blade.
3.Go to your Key Vault and click on Access Policies and then click on Add service principal with secret GET permission.
4.When you use ServiceBusTrigger, you set ServiceBusConnectionString in Function ->Configuration ->Application settings.
public static void Run([ServiceBusTrigger(_topicName, _subscriptionName, Connection = "ServiceBusConnectionString")] string mySbMsg, ILogger log)
{ ....
}
5.Now you change the value of ServiceBusConnectionString to the Azure Key Vault reference with #Microsoft.KeyVault(SecretUri=Secret URI with version). Then you could run your function successfully with Key Vault.
I've been trying to get to through this issue for some time now, and unfortunately, I'm hitting a wall.
We have Azure AD configured with Azure Domain Services. VM's are joined to this domain. We also have Azure SQL Database that we're attempting to connect to from the IIS application running on the VM.
I've followed the official MS docs on this (https://learn.microsoft.com/en-us/azure/sql-database/sql-database-aad-authentication-configure) that helped verify Azure AD identities can connect to the Azure SQL DB (I can connect via SSMS to Azure SQL using an Azure AD Admin user. I can then use that ID to generate other contained users, which I have done).
The problem exists when I attempt to authenticate with the database with Active Directory Integration in my connection string. I get two errors, which i'm going to include below.
Now, I have not been able to get past the last one (stating that federation is needed), and in fact, have read numerous things (albeit nothing directly from MS) that says this is simply not supported and that I need to create a federation to get this working, which just seems completely unreasonable given that this is an Azure-only environment.
Connection string included below as well.
I've tried various iterations of that connection string to no avail. If anyone can provide some insight into this, I'd appreciate it greatly!
One last thing, the IIS application pool is running as a user that is both part of the AAD Domain group on Azure AD, and a contained user created on the DB.
Thanks!
I've tried various iterations of the connection string that include removing much of the additional params on the connection string, and the quotes.
I've tried different users on the application pool
End goal is to remove all plain-text PW's from appearing within the connection string to Azure SQL.
First error:
Unable to load adalsql.dll (Authentication=ActiveDirectoryPassword). Error code: 0x2.
Second Error:
[AdalException: Integrated Windows authentication supported only in federation flow.]
ADALNativeWrapper.ADALGetAccessToken(String username, IntPtr password, String stsURL, String servicePrincipalName, ValueType correlationId, String clientId, Boolean* fWindowsIntegrated, Int64& fileTime) +829
System.Data.SqlClient.<>c__DisplayClass2_0.<AcquireTokenAsync>b__0() +132
System.Threading.Tasks.Task`1.InnerInvoke() +121
System.Threading.Tasks.Task.Execute() +47
Connection String:
name="LocalSqlServer" connectionString="Server=tcp:XXXXX;Initial Catalog=XXXXX;Persist Security Info=False;MultipleActiveResultSets=False;Encrypt=True;TrustServerCertificate=False;Authentication='Active Directory Integrated';" />