I am using a Classic Web Service with a non-default endpoint for a Update Resource activity on the Azure Data Factory. This is the error I get:
Screenshot of Error
I didn't find any info on the web and couldn't figure it out myself. This website shows an example that I used by just filling in my values for mlEndpoint, apiKey and updateRessourceEndpoint:
{
"name": "updatableScoringEndpoint2",
"properties": {
"type": "AzureML",
"typeProperties": {
"mlEndpoint": "https://ussouthcentral.services.azureml.net/workspaces/xxx/services/--scoring experiment--/jobs",
"apiKey": "endpoint2Key",
"updateResourceEndpoint": "https://management.azureml.net/workspaces/xxx/webservices/--scoring experiment--/endpoints/endpoint2"
}
}
}
There is no mention of a token that needs to be passed...
this error is basically saying the apiKey you provided is invalid to perform the update resource operation. Here is some posts for your reference: https://social.msdn.microsoft.com/Forums/azure/en-US/3bb77e37-8860-43c6-bcaa-d6ebd70617b8/retrain-predictive-web-service-programmatically-when-do-not-have-access-to-managementazuremlnet?forum=MachineLearning
Please also be noted that if you modified your linked service in ADF, remember to re-deploy the pipeline as well to reflect your change in time.
Related
So, I have a Power Automate Flow, which creates a Sharepoint Site using the Sharepoint REST API via "Send HTTP Request to Sharepoint" connector. I`ve following body:
{
"request": {
"Title": "#{variables('strSPName')}",
"Url":"#{variables('strSPAddress')}",
"Lcid": 1031,
"ShareByEmailEnabled":true,
"Description":"-",
"WebTemplate":"SITEPAGEPUBLISHING#0",
"SiteDesignId":"-",
"Owner": "#{outputs('Get_my_profile_(V2)')?['body/mail']}"
}
}
So right now I have only me as owner, but I need an additional owner.
I`ve tried to set two owners in an array like this:
"Owner": ["#{outputs('Get_my_profile_(V2)')?['body/mail']}", "email#test.com"]
Unfortunately, I`m getting the following error:
An unexpected 'StartArray' node was found when reading from the JSON reader. A 'PrimitiveValue' node was expected.
Any suggestions?
Thanks in advance.
As far as I am aware you can only set one owner when creating a site via the SPSiteManager endpoint. However, a workaround could be to add the second user to the Owners group directly after creation with a second POST request.
Below is an example
The principal id of the sitegroup should be 3.
Uri
_api/web/SiteGroups(3)/users
Body
{
"__metadata": { "type": "SP.User" },
"LoginName":"i:0#.f|membership|jane#contoso.onmicrosoft.com"
}
I was asked by Azure Support to post this question, just to see if anyone had a useful opinion.
I am stepping through MS Azure training courses. I created the usual free account to go through these. I've gone through a few dozen of them, and am now at this one:
https://learn.microsoft.com/en-us/learn/modules/secure-and-isolate-with-nsg-and-service-endpoints/3-exercise-network-security-groups?source=learn
This attempts to use the Azure PowerShell service. I had some trouble getting to the PowerShell page. It appears that if I'm not already logged into the portal, it goes into a semi-infinite loop, trying to get to the shell page, then trying to login, then the shell page, and finally it gives up and says "We couldn't sign you in. Please try again.".
However, I was able to work around this. If in a separate tab, I log into the Azure Portal, and then go back and follow the link to Azure Cloud Shell, it passes the login gate and sends me to the page where I choose Bash or PowerShell. The course specifies using Bash. When I select that, it then asks me to create a Storage object. When I confirm that, it gives me the following error (subscription id elided):
{
"error": {
"code": "RequestDisallowedByPolicy",
"target": "cs733f82532facdx4f04x95b",
"message": "Resource 'cs733f82532facdx4f04x95b' was disallowed by policy. Policy identifiers: '[{\"policyAssignment\":{\"name\":\"Enforce tag on resource\",\"id\":\"/subscriptions/xxxxx/providers/Microsoft.Authorization/policyAssignments/740514d625684aad84ef8ca0\"},\"policyDefinition\":{\"name\":\"Enforce tag on resource\",\"id\":\"/subscriptions/xxxxx/providers/Microsoft.Authorization/policyDefinitions/be3862a6-ca1e-40b0-a024-0c0c7d1e8b3e\"}}]'.",
"additionalInfo": [
{
"type": "PolicyViolation",
"info": {
"policyDefinitionDisplayName": "Enforce tag on resource",
"evaluationDetails": {
"evaluatedExpressions": [
{
"result": "True",
"expressionKind": "Field",
"expression": "tags[Department]",
"path": "tags[Department]",
"targetValue": "false",
"operator": "Exists"
}
]
},
"policyDefinitionId": "/subscriptions/xxxxx/providers/Microsoft.Authorization/policyDefinitions/be3862a6-ca1e-40b0-a024-0c0c7d1e8b3e",
"policyDefinitionName": "be3862a6-ca1e-40b0-a024-0c0c7d1e8b3e",
"policyDefinitionEffect": "deny",
"policyAssignmentId": "/subscriptions/xxxxx/providers/Microsoft.Authorization/policyAssignments/740514d625684aad84ef8ca0",
"policyAssignmentName": "740514d625684aad84ef8ca0",
"policyAssignmentDisplayName": "Enforce tag on resource",
"policyAssignmentScope": "/subscriptions/xxxxx",
"policyAssignmentParameters": {
"tagName": {
"value": "Department"
}
}
}
}
]
}
}
I think the simple conclusion from this is that my free account doesn't have enough rights to do what is needed here. The documentation I've read seems to imply that I have to get additional rights on the account in order to do this. However, I'm just using a free account that I created to go through the Azure training courses. It doesn't really make sense to ask me to do this. I've seen other Azure courses create a temporary sandbox supposedly because they have particular objects pre-created in the sandbox, but I'm also thinking that the sandbox has particular permissions that are not available in the free account. It seems to me that the only reasonable fix for this problem is for that course to be refactored to use a temporary sandbox with the correct permissions.
I'm just looking for any opinions on this, and confirmations that this is what should be done.
It doesn't look like you are creating resource, cloudshell storage, on your free subscription. Except if you added to a Work/Corporate tenant.
From the information you provide, subscription you are trying to use has a policy to enforce tags Department, mean any resource created should have a tag with Department information.
Currently, I'm trying to set up a dynamic key vault linked service.
Unfortunately, whatever I try I'm not able to successfully test the connection.
{
"name": "AzureKeyVault1",
"properties": {
"type": "AzureKeyVault",
"typeProperties": {
"baseUrl": {
"value": "#concat('https://vault.azure.net','/')",
"type": "Expression"
}
}
}
}
The above code using concat is not a real use case but just a way to test if dynamic json is possible for linked service.
I was expecting (based on the documentation) that I could make the baseUrl property dynamic. Am I using the right formatting?
I get the following error:
Error: Error: Can't get property concat
Wim,based on the official document,parameterize linked services only supports below services,not including Azure Key Vault.
You could submit feedback here to push some improvements of azure data factory you want.
#Wim
To Answer your question, this is the correct formatting.
I have been able to parameterize a pipeline parameter, and then pass it as the baseurl in a dynamic expression.
I want to set up CI/CD (ARM template) with StreamAnalytics Job with output set to DataLake Store.
https://learn.microsoft.com/en-us/azure/templates/microsoft.streamanalytics/streamingjobs/outputs#microsoftdatalakeaccounts
The issue comes with refreshToken:
"It is recommended to put a dummy string value here when creating the data source
and then going to the Azure Portal to authenticate the data source
which will update this property with a valid refresh token"
Furthermore after 90-days refresh token is outdated and you need to do "Renvew Authorization"
https://learn.microsoft.com/en-us/azure/stream-analytics/stream-analytics-data-lake-output#renew-data-lake-store-authorization
I tried to authorize ServicePrincipal.
How to do automatic deployment for ASA with DataLake?
How to handle issue with this 90-days token validitiy?
Maybe you wiped the trail :)
at this time it is not yet possible. Sorry for the inconvenience. However we know this is very important and we will add Service Principal auth in the near future (we are looking at the exact ETA).
In the meantime you need to renew manually the token. This can be done without losing any data by (1) stopping the job, (2) changing the token, then (3) restarting the job last time it was stopped.
Let me know if you have any further question.
As far as I know quite soon MSI-based authentication will be in preview.
But if you need an immediate solution (to e.g. be able to have a VSTS pipeline running through without error) you can do the following:
Create template (e.g. with the CICD NuGet Package [1])
Manipulate the ARM Template <jobName>.JobTemplate.json
Add the output datasource object for the ADLS output object
If you work with Visual Studio you can get the values quite easily from the ADLS output JSON
It is important to set refreshToken to some fake value
Like the following:
"outputs": [
{
"name": "xxx",
"properties": {
"serialization": {
"type": "Json",
"properties": {
"encoding": "UTF8",
"format": "LineSeparated"
}
},
"datasource": {
"type": "Microsoft.DataLake/Accounts",
"properties": {
"accountName": "xxx",
"tenantId": "xxx-xxx-xxx-xxx-xxx",
"tokenUserPrincipalName": "xxx#xxx.com",
"tokenUserDisplayName": "xxx, xxx",
"filePathPrefix": "xxx/{date}/{time}",
"dateFormat": "yyyy/MM/dd",
"timeFormat": "HH",
"refreshToken": "faketoken"
}
}
}
},
...
Deploy the ARM Template
The job will start successfully but it is necessary to renew the token, therefore
Stop the job
Renew the authentication of the ADLS output
Start the job
Resources
[1] CICD NuGet Package
I am trying to automate creating an API Connection for a storage account in Azure using Resource Manager templates.
I am using the listKeys method in ARM to retrieve the access key of the storage account. I went through this question and it is not working for me.
When I use the method in the outputs section of the template, it is working fine and successfully retrieving and displaying the access key.
"outputs": {
"listKeysOutput": {
"type": "string",
"value": "[listKeys(resourceId('Microsoft.Storage/storageAccounts', parameters('storagename')), providers('Microsoft.Storage', 'storageAccounts').apiVersions[0]).keys[0].value]"
}
}
However, when I try to use the same function inside a connection resource (as shown below), the template executes without any error. But on accessing the API Connection from the Azure portal, it says 'parameter is missing'.
"parameterValues": {
"accesskey": "[listKeys(resourceId('Microsoft.Storage/storageAccounts', parameters('storagename')), providers('Microsoft.Storage', 'storageAccounts').apiVersions[0]).keys[0].value]",
"accountName": "[parameters('storagename')]"
}
Am I missing something here? Or the output of listKeys is not being accepted by the 'accesskey' property?
I had a similar experience a few months ago, and resolved it by using a connection string directly in my code and then passing the connection string into the connections. The value looked like this:
[concat('DefaultEndpointsProtocol=https;AccountName=', variables('storageConfigs')[0].name,';AccountKey=', listKeys(resourceId('Microsoft.Storage/storageAccounts/', variables('storageConfigs')[0].name), variables('defaultStorageApiVersion')).key1)]
I used a storage config object as an input, so that's why it looks like above you could replace variables('storageConfigs')[0].name with whatever name or variable function you use in your code. Looks like above it may be storagename
Two things that might be causing the issue:
Ensure the API Connection has a dependency on the storage account
Capitalise the key in "accessKey" (some things in templates are case sensitive)
#Naren, I recommend you can use this API function to get your Storage Key
POST
https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Storage/storageAccounts/{accountName}/listKeys?api-version={api-version}
You could get the same result as the template.
{
“keys”: [
{
“keyName”: “key1”,
“value”: "key1Value”,
“permissions”: “FULL”
},
{
“keyName”: “key2”,
“value”: "key2Value”,
“permissions”: “FULL”
},
]
}
Just for your reference:
https://msdn.microsoft.com/en-us/library/mt163589.aspx
Dependency is indeed a requirement so that the storage account is already created before the deployment of the api connection is initiated.
The problem with the OP template code is the use of accesskey while the correct parameter name is accessKey (Note the capital K) for an Azure Blob api connection resource.
For anyone who struggles with the lack of documentation for the required parameters of API Connection resources - initiate this API Call:
https://management.azure.com/subscriptions/<YOUR SUBSCRIPTION ID>/providers/Microsoft.Web/locations/<YOUR LOCATION>/managedApis/<API TYPE>?api-version=2016-06-01
The <API TYPE> should be the api type of the connection to check for example azureblob, azurequeues or documentdb.
A description of all the expected parameters is returned along side other descriptive information for that resource.