Is there any way to create Azure Data Factory v2 client without create AD app in azure?
Yes. You could do this with SDK or restful API
Data factory service identity is generated as follows:
When creating data factory through Azure portal or PowerShell,
service identity will always be created automatically since ADF V2
public preview.
When creating data factory through SDK, service
identity will be created only if you specify "Identity = new
FactoryIdentity()" in the factory object for creation. See example
in .NET quickstart - create data factory.
When creating data factory through REST API, service identity will be created only if you specify "identity" section in request body. See example in REST quickstart - create data factory.
Please reference this doc.
Related
I am new to azure cloud infrastructure, I am trying to create a azure data factory, which I did now I am trying to create a linked service to another SaaS provider "salesforce". I am not seeing any place to create one.
I have consulted the following links, but could not find anything yet.
I cannot see management hub
https://learn.microsoft.com/en-us/azure/data-factory/author-management-hub
or in azure portal
https://learn.microsoft.com/en-us/azure/data-factory/concepts-linked-services
Thank you
Please ref this tutorial: Copy data from and to Salesforce by using Azure Data Factory
This article outlines how to use Copy Activity in Azure Data Factory
to copy data from and to Salesforce. It builds on the Copy Activity
overview article that presents a general overview of the copy
activity.
You could create the Salesforce linked service from here Data Factory UI on Portal:
Manage-->linked services-->new-->Salesforce:
Configure the Salesforce:
I have few linked services used to copy data from on premise server to Azure BLOB and then loading that data into Azure DB using ADF pipeline, these linked services are using a user account to authenticate, I want to replace this user account with a Service Principal name. Is it possible? If yes then how.
Thanks.
Yes,it is possible.I checked blob storage linked service and azure sql db linked service documents,then all support service principal authenticate ways. You could choose that way when you create linked services:
I created a Web task in Azure Data Factory to start/stop SSIS Integration Runtime. However, when I try to run the pipeline, I keep getting this error - Activity Start My IR failed:
{"error":{"code":"AuthorizationFailed","message":"The client
'2b36c922-34cf-43d8-be05-5d757017e2ed' with object id
'2b36c922-34cf-43d8-be05-5d757017e2ed' does not have authorization to
perform action
'Microsoft.DataFactory/factories/integrationruntimes/start/action'
over scope
'/subscriptions/e8e76bdc-4591-4936-9899-31bffef9c5cd/resourcegroups/XXXXXBI-RG/providers/Microsoft.DataFactory/factories/XXXXXBI-DF/integrationruntimes/XXXXXSSIS-IR'."}}
I followed this instruction - https://learn.microsoft.com/en-us/azure/data-factory/how-to-schedule-azure-ssis-integration-runtime
For Authentication, select MSI to use the managed identity for your ADF, see Managed identity for Data Factory article for more info.
What is the issue?
According to the error, you may not add the service principal to the ADF as a role.
You need to assign the managed identity for your ADF a Contributor role to itself, so Web activities in its pipelines can call REST API to start/stop Azure-SSIS IRs provisioned in it.
Follow the step 5 in this link:
I'm trying to create linked service to azure data lake analytics and it is showing the error like this
[{"code":9512,"message":"Failed to connect to ADLA account 'miniprojectanalytics'
with error 'The client '8899dd6a-45b1-4955-a3b9-7b3a95dd6dfa' with object id
'8899dd6a-45b1-4955-a3b9-7b3a95dd6dfa' does not have authorization to perform action
'Microsoft.Authorization/permissions/read' over scope '
subscriptions/8e9519a3-7b76-40fe-9b64-752ed2d0d4dc/resourceGroups/harigrp/providers/Microsoft.DataLakeAnalytics/accounts/miniprojectanalytics/providers/Microsoft.Authorization'.
Trace: 0941cde4-bff5-42b4-8186-1bd08f04944b Time: 2018-10-05T00:27:23.0661884-07:00'."}]
could anyone please assist me how to resolve this
I believe you need to give your service account access on the ADLA via the Add User Wizard.
From here
Service principal authentication
The Azure Data Lake Analytics linked service requires a service principal authentication to connect to the Azure Data Lake Analytics service. To use service principal authentication, register an application entity in Azure Active Directory (Azure AD) and grant it the access to both the Data Lake Analytics and the Data Lake Store it uses. For detailed steps, see Service-to-service authentication. Make note of the following values, which you use to define the linked service:
Application ID
Application key
Tenant ID
Grant service principal permission to your Azure Data Lake Anatlyics using the Add User Wizard.
Creating linked service(Azure Data Factory V2) for Azure Sql supports SQL Authentication. I want to know if it also supports Azure Active Directory Integrated Authentication.
As of today (Feb '18),
ADFV2 can connect to some sources using Managed Service Identity (MSI). In short, the ADFV2 instance is given an identity in the Active Directory as an Active Directory Application. See the docs here. Then the ADFV2 can connect to data sources as that identity.
Although Azure SQL supports Managed Service Identity, accessing Azure SQL Server through MSI is not available for ADFV2 yet. From the docs:
ConnectionString: Specify information needed to connect to the Azure
SQL Database instance for the connectionString property. Only basic
authentication is supported. Mark this field as a SecureString to
store it securely in Data Factory, or reference a secret stored in
Azure Key Vault.
As the docs state, you can use the ADFV2 Managed Service Identity to connect to KeyVault and use keys and secrets stored there, which is probably your best best for limiting security information in configuration.
Keep in mind that the UI for ADFV2 is still quite far behind the API, so you may need to use PowerShell or Azure command line to set it up properly.