I trying to run below PowerShell script to add azure data factory data sets. But im getting motioned error.
Json File
{
"name": "DSNAME",
"properties": {
"linkedServiceName": {
"referenceName": "REGNAME1",
"type": "LinkedServiceReference"
},
"annotations": [],
"type": "AzureDataExplorerTable",
"schema": [],
"typeProperties": {
"table": "TABLE_TEST"
}
},
"type": "Microsoft.DataFactory/factories/datasets"
}
Powershell
az config set extension.use_dynamic_install=yes_without_prompt
Get-ChildItem "ADF_DATASETS/" -Filter *.json |
Foreach-Object {
$content = Get-Content $_.FullName
az datafactory dataset create --properties $content --name "DATASETNAME" --factory-name "ADFNAME" --resource-group "RG_TEST"
}
Error:
Error detail: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)
Please provide a valid JSON file path or JSON string.
The provided JSON string may have been parsed by the shell. See https://docs.microsoft.com/cli/azure/use-cli-effectively#use-quotation-marks-in-arguments
ERROR: Failed to parse string as JSON:
What kind of text --properties expect?
It is expecting JSON string in double quotation marks. So you have to put the value of $content inside the double quotation mark.
az datafactory dataset create --properties "{\"type\":\"AzureBlob\",\"linkedServiceName\":{\"type\":\"LinkedServiceReference\",\"referenceName\":\"exampleLinkedService\"},\"parameters\":{\"MyFileName\":{\"type\":\"String\"},\"MyFolderPath\":{\"type\":\"String\"}},\"typeProperties\":{\"format\":{\"type\":\"TextFormat\"},\"fileName\":{\"type\":\"Expression\",\"value\":\"#dataset().MyFileName\"},\"folderPath\":{\"type\":\"Expression\",\"value\":\"#dataset().MyFolderPath\"}}}" --name "exampleDataset" --factory-name "exampleFactoryName" --resource-group "exampleResourceGroup"
Related
I am referencing the below codes with command but getting error for both
az datafactory linked-service create --resource-group $resourcegroup --factory-name $factoryname --linked-service-name ls_AzureKeyVault_storage --properties #ls_AzureKeyVault_storage.json > \dev\null
{
"name": "ls_AzureKeyVault_storage",
"properties": {
"annotations": [],
"type": "AzureKeyVault",
"typeProperties": {
"baseUrl": "https://kvadfconnections.vault.azure.net/"
}
}
}
az datafactory linked-service create --resource-group $resourcegroup --factory-name $factoryname --linked-service-name AzureStorageLinkedService --properties #AzureStorageLinkedService.json > \dev\null
{
"name": "AzureStorageLinkedService",
"properties": {
"annotations": [],
"type": "AzureBlobStorage",
"typeProperties": {
"connectionString": {
"type": "AzureKeyVaultSecret",
"store": {
"referenceName": "ls_AzureKeyVault_storage",
"type": "LinkedServiceReference"
},
"secretName": "sec-stforadfcli-connection"
}
}
},
"type": "Microsoft.DataFactory/factories/linkedservices"
}
We have tested this in our local environment, Below statements are based on our analysis.
While creating the linked service through AzureCLI cmdlet using az data factory linked-service create ,you need to pass the json file to the --properties flag .
az datafactory linked-service create --factory-name
--linked-service-name
--properties
--resource-group
[--if-match]
If you declare the properties{} list on top of the typeproperties{} in your json file then while creating the linked service to the data factory it will fail with the error that you have shared as shown in the below
You Need pass only typeProperties in json file to create a linked service with the data factory as shown in the below.
In the below example ,we are trying to create a keyvault linked service with our existing ADF.
Here is our keyvault.json file which has type properties.
{
"type": "AzureKeyVault",
"typeProperties":{
"baseUrl": "<keyvault>"
},
"annotations":[<requiredannotations],
"description":"<requireddescription>",
"parameters": {
"test":{
"type":"String",
"defaultValue":"test"
}
}
}
Here is the sample output for reference :
You can use the above keyvault.json file as reference & make the changes as per your requirement.
You can also refer this documentation, for more information about what all the properties that we can pass to AzurekeyVaultLinkedService & there respective datatypes.
i have azure function with LinuxFxVersion set to DOTNET:
"siteProperties": {
"metadata": null,
"properties": [
{
"name": "LinuxFxVersion",
"value": "DOTNET|3.1"
},
{
"name": "WindowsFxVersion",
"value": null
}
],
"appSettings": null
},
I want to set it to Python:
"siteProperties": {
"metadata": null,
"properties": [
{
"name": "LinuxFxVersion",
"value": "Python|3.9"
},
{
"name": "WindowsFxVersion",
"value": null
}
],
"appSettings": null
},
According to msdn source, I need to use Power shell to change it:
az functionapp config set --name <func_name> --resource-group <rg> --linux-fx-version 'Python|3.9'
but im getting error:
'3.9' is not recognized as an internal or external command,
operable program or batch file.
When im typing just 'Python' i get response:
Operation returned an invalid status 'Bad Request'
How to change linux fx version in Azure Function from .NET to Python?
The way you can solve this error in Powershell is to wrap up the string containing the pipe character with quotes.
Here are multiple examples:
az functionapp config set --name <func_name> --resource-group <rg> --linux-fx-version '"Python|3.9"'
az functionapp config set --name <func_name> --resource-group <rg> --linux-fx-version 'Python"|"3.9'
If you are running the above command in bash use : instead of |
az functionapp config set --name <func_name> --resource-group <rg> --linux-fx-version "Python:3.9"
https://octopus.com/blog/powershell-pipe-escaping
https://github.com/Azure/azure-cli/issues/7874
I have extracted the ARM template belonging to the preview version of Azure App Configuration, and am setting it into our IaC repository - so far so good.
Our next logical step is to include insertion of the AppConfiguration.PrimaryKey into our Key Vault. However I do not know the name of this property, and I can not find any information on the subject online. Also I can not see the AppConfiguration/configurationStores type listed in resources.azure.com (assuming its because its still in public preview).
Does anyone know how to reference the primary key (and possibly the read-only primary key), so i can reference them through a "outputs" variable in my arm template?
Then I can let Az Cli/Az Powershell insert the secret into our Key Vault, and we obtain full automation of our IaC
I was not able to figure this out.
However by using az cli commands in a IaC script (which anyways invokes the arm template residing in a azure blob store) I circumvented the problem:
$connStrings = az appconfig credential list -n $configName| ConvertFrom-Json
$readOnlyConnString = ($connStrings | Where {$_.name -eq "Primary Read Only"}).connectionString
$primaryConnString = ($connStrings | Where {$_.name -eq "Primary"}).connectionString
#then
az keyvault secret set --vault-name $kvName --name $keyNameRO --value $readOnlyConnString
az keyvault secret set --vault-name $kvName --name $keyNamePrimary --value $primaryConnString
For an ARM template I did the following. The listkeys function returns a full list of all the values that have to do with the keys. This was hard to figure out. I hope it helps.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"configurationStores_instance_name": {
"defaultValue": "ac-instance",
"type": "String"
}
},
"variables": {
"apiVersionVar": "[providers('Microsoft.AppConfiguration', 'configurationStores').apiVersions[0]]",
"resourceId": "[resourceId('Microsoft.AppConfiguration/configurationStores', parameters('configurationStores_instance_name'))]",
},
"resources": [
{
"type": "Microsoft.AppConfiguration/configurationStores",
"apiVersion": "2019-10-01",
"name": "[parameters('configurationStores_instance_name')]",
"location": "northcentralus",
"sku": {
"name": "standard"
},
"properties": {}
}
],
"outputs": {
"AppConfigEndpoint": {
"type": "string",
"value": "[reference(parameters('configurationStores_instance_name')).endpoint]"
},
"AppConfigKeys": {
"type": "Array",
"value": "[listkeys(variables('resourceId'), variables('apiVersionVar')).value]"
}
}
}
hope this helps!
I'm trying to create a VM via the CLI and add an artifact from the public repo in a dev test lab.
Here is my script:
az lab vm create --resource-group TestRG --lab-name TestLab --name 'ScriptVM' --image "Windows 10 Pro, Version 1809" --image-type gallery --size 'Standard_B2s' --admin-username 'user' --admin-password 'pw' --artifacts '{ "/artifactsources/publicrepo/artifacts/linux-java" }'
I don't know the format of the artifacts switch though as it just says a JSON encoded array. How does this work? I just get the error
Expecting property name enclosed in double quotes: line 1 column 3 (char 2)
So the answer was I didn't have all the required elements in the JSON. I also fell foul of escaping quotation marks in the PS script. The final switch ended up looking like this:--artifacts '[{"""artifactId""": """artifacts/windows-notepadplusplus"""}]'
The following example shows the sections that make up the basic structure of a definition file:
{
"$schema": "https://raw.githubusercontent.com/Azure/azure-devtestlab/master/schemas/2016-11-28/dtlArtifacts.json",
"title": "",
"description": "",
"iconUri": "",
"targetOsType": "",
"parameters": {
"<parameterName>": {
"type": "",
"displayName": "",
"description": ""
}
},
"runCommand": {
"commandToExecute": ""
}
}
It resides in your Git repo and is referenced in the AZ command. A video of the entire process can be found in our documentation.
https://learn.microsoft.com/en-us/azure/lab-services/devtest-lab-artifact-author
I had been using the line below to grab the value of the internalIpAddress property from an ILB App Service Environment in Azure:
az resource show `
--ids "/subscriptions/$subscription_id/resourceGroups/$ilbase_rg_name/providers/Microsoft.Web/hostingEnvironments/$ilbase_name/capacities/virtualip" `
--query "internalIpAddress"
The format of the virtualip resource was:
{
"internalIpAddress": "10.30.0.139",
"outboundIpAddresses": [
"13.72.76.135"
],
"serviceIpAddress": "13.72.76.135",
"vipMappings": []
}
Seems like in the past day or so, the format of the virtualip resource has now changed to this:
{
"additionalProperties": {
"internalIpAddress": "10.30.0.139",
"outboundIpAddresses": [
"13.72.76.135"
],
"serviceIpAddress": "13.72.76.135",
"vipMappings": []
},
"id": null,
"identity": null,
"kind": null,
"location": null,
"managedBy": null,
"name": null,
"plan": null,
"properties": null,
"sku": null,
"tags": null,
"type": null
}
And now my command no longer works...it returns nothing. I can modify my command to get the entire additionalProperties object but I then don't know how to parse thru it to get just the value of the internalIpAddress property.
Another interesting note on this is, if you go to the Azure Resource Explorer and navigate to the virtualip resource, it still shows it in the same old format. If you try the PowerShell code the Azure Resource Explorer gives you to query the resource, it returns nothing.
Here is the PowerShell the Azure Resource Explorer said to use:
Get-AzureRmResource -ResourceGroupName MyRG -ResourceType Microsoft.Web/hostingEnvironments/capacities -ResourceName "myilbase/virtualip" -ApiVersion 2018-02-01
Looking for some help on how to parse the nested internalIpAddress property from the additionalProperties object
just traverse the object like you normally would:
--query "additionalProperties.internalIpAddress"