I have few linked services used to copy data from on premise server to Azure BLOB and then loading that data into Azure DB using ADF pipeline, these linked services are using a user account to authenticate, I want to replace this user account with a Service Principal name. Is it possible? If yes then how.
Thanks.
Yes,it is possible.I checked blob storage linked service and azure sql db linked service documents,then all support service principal authenticate ways. You could choose that way when you create linked services:
Related
Stuff in Azure are secured with Service Accounts. In order for me to see stuff I need to download the Service Account certificate and then log in via the Azure CLI using the extracted certificate and the Service Account Application Id. So now I can see everything the Service Account can see, great. But it is a pain in the neck and slow. So my question: Can I use the same certificate and credentials to log into the Azure Portal website so I can browse around using the web browser instead?
Using a Service Principal for interactive logins to the Azure Portal is not possible - which is by design. In order to be able to see the same resources as the Service Principal through the Azure Portal, you would require a user account that holds the Azure RBAC Reader role against those resources that are in scope of the Service Principal role assignments.
As you mentioned performance being an issue with using the Service Principal login, you could try Azure Resource Graph queries. These are supported by Azure CLI, Azure PowerShell as well as all the major Azure SDK's. Obviously, this won't bring you the visual experience like the Azure Portal but might resolve the performance piece maybe.
However, requesting/creating a user account that has the corresponding RBAC roles assigned would be the only way to allow you to see the resources through the Azure Portal.
So, I was provisioned an Azure SQL server into my subscription of Azure, and I am using Azure Data Factory. Well, starting to use it
The problem I ran into is the following.
I am able to connect
to Azure SQL Server database via
- SQL Management Studio,
- via Visual Studio Code with SQL Tools extension,
- and with Azure data studio.
I was able to create tables and stored procedures.
All this I am doing by logging in using Authentication type
"Azure Active Directory - Universal With MFA"
.. and providing my "Active Directory Admin". (The "Server Admin" does not work as I do not know what the password was when it was created and our sys admin said "it was blank" ??)
However, when I try to set up a linked service on Azure Data Factory and connect to the same SQL Server - I do not see that option (Azure Active Directory - Universal With MFA) in the drop down for Authentication type.
Those available
- SQL Authentication
- System Assigned Managed Identity
- Service Principal
- User Assigned Managed Identity
I tried them all, and I can't connect using either.
My understanding is that there is more to be done on the server management side, but I do not understand what.
You can use service principal method to create a success linked service to your SQL database using Azure active directory authentication. You need to follow the steps below:
First create an app registration in Azure active directory. Go to azure active directory and select App registrations.
Create a new app registration with certain name. As you can see in the above image, I have created it with the name for_sql_db.
Since you already have set your SQL server with Azure Active directory admin, create a user using the name of the app registration created above. Assign this user any required role. I have used the following queries in my query editor of SQL database.
CREATE USER [for_sql_db] FROM EXTERNAL PROVIDER;
ALTER ROLE [db_owner] ADD MEMBER [for_sql_db];
Once the user is created in your SQL database, go to data factory to create linked service.
Select the SQL server and database you want to create linked service to. Choose the Authentication as Service Principal.
Then you can see Tenant , Service principal ID and Service principal key. Go to the app registration you have created. Here you can see Directory tenant ID and application client ID. Give them as following:
linked service Tenant: Directory Tenant ID
Service principal ID: application client ID
For Service principal key, you need to create a secret in your app registration (for_sql_db).
After creating the client secret, copy its value and use it as Service principal key in data factory linked service.
You can see that, I have followed the above procedure and successfully created linked service to my SQL database.
In Azure DevOps, I have created a service connection (type: Azure Resource Manager) to be able to upload files to Azure Blob Storage.
Then I have added the Storage Blob Data Contributor role for this service principal under Access Control (IAM) in my Azure Storage account by searching for the service principal's name under Select.
I have noticed that each time I create a new DevOps pipeline that uses the (same) service connection, I need to add the Storage Blob Data Contributor role again because under Select, there are then multiple items with the same (service principal's) name. It's not clear why there are multiple items and it's also unclear which one is the newest, such that I am just adding all items as a workaround.
Is there anything that I am missing to avoid ending up with dozens of items to select when assigning roles for a new pipeline that uses the same service connection?
As design, one service connection map to one single service principal.
You issue mostly like you did not ever assign the actual service principal id to that service connection while you configure it. When the system finds there is no principal there, it will automatically create one for it in azure.
Please give the full parameters value there, including service principal id and secret, when you create the service connection.
Then you can just grant the permission to the currently used service principal.
I'm trying to create linked service to azure data lake analytics and it is showing the error like this
[{"code":9512,"message":"Failed to connect to ADLA account 'miniprojectanalytics'
with error 'The client '8899dd6a-45b1-4955-a3b9-7b3a95dd6dfa' with object id
'8899dd6a-45b1-4955-a3b9-7b3a95dd6dfa' does not have authorization to perform action
'Microsoft.Authorization/permissions/read' over scope '
subscriptions/8e9519a3-7b76-40fe-9b64-752ed2d0d4dc/resourceGroups/harigrp/providers/Microsoft.DataLakeAnalytics/accounts/miniprojectanalytics/providers/Microsoft.Authorization'.
Trace: 0941cde4-bff5-42b4-8186-1bd08f04944b Time: 2018-10-05T00:27:23.0661884-07:00'."}]
could anyone please assist me how to resolve this
I believe you need to give your service account access on the ADLA via the Add User Wizard.
From here
Service principal authentication
The Azure Data Lake Analytics linked service requires a service principal authentication to connect to the Azure Data Lake Analytics service. To use service principal authentication, register an application entity in Azure Active Directory (Azure AD) and grant it the access to both the Data Lake Analytics and the Data Lake Store it uses. For detailed steps, see Service-to-service authentication. Make note of the following values, which you use to define the linked service:
Application ID
Application key
Tenant ID
Grant service principal permission to your Azure Data Lake Anatlyics using the Add User Wizard.
Is it possible to list out the cloud services for azure storage account with out using certification thumbprint . And also get the deployment ID for particular cloud service?
I have connected storage account(based on account key and account name) using azure storage client library and list out the tables and containers. My question is, displays the cloud services for particular storage account with out using certification.
Note: I saw rest api to list out the storage account and services using subscription id with certification.
I am waiting your response.
Is it possible to list out the cloud services for azure storage
account with out using certification thumbprint . And also get the
deployment ID for particular cloud service?
To achieve this, you would need to use Azure Service Management API and API calls need to be authenticated. Using X509 Certificate is one of them (which you don't want to use). Other way to achieve this would be to use Azure Active Directory. You can read more about it here: http://msdn.microsoft.com/en-us/library/azure/ee460782.aspx#bk_ad however authenticating API requests using Azure AD is more complicated than using a certificate IMHO.