Groovy: Executing azure CLI Command with JSON - Parsing Issue? - azure

Currently I fail to run an azure CLI command from Groovy because of the JSON Part in the Command.
There is an azure command to run custom scripts on a virtual machine. The CommandToExecute on the Machine is passes as JSON.
WORKING Example:
REQUEST-CALL in Console:az vm extension set -g demo --vm-name demo-cfg01 --name CustomScript --publisher Microsoft.Azure.Extensions --settings '{"commandToExecute":"ls"}'
RESPONSE: {
"autoUpgradeMinorVersion": true,
"forceUpdateTag": null,
"id": "/subscriptions/xxxxxxxxxx-xxxxxx-xxxx-xxxx-xxxxxxxxxxxxx/resourceGroups/demo/providers/Microsoft.Compute/virtualMachines/demo-cfg01/extensions/CustomScript",
"instanceView": null,
"location": "germanycentral",
"name": "CustomScript",
"protectedSettings": null,
"provisioningState": "Succeeded",
"publisher": "Microsoft.Azure.Extensions",
"resourceGroup": "demo",
"settings": {
"commandToExecute": "ls"
},
"tags": null,
"type": "Microsoft.Compute/virtualMachines/extensions",
"typeHandlerVersion": "2.0",
"virtualMachineExtensionType": "CustomScript"
}
This script works fine.
"Same" Command executed with Groovy leads to following:
def process
StopWatch.withTimeRecording("EXECUTING COMMAND '" + cargs + "'",_logger, Level.ALL) {
process = (cargs).execute(null,null);
process.waitForProcessOutput(sout, serr)
}
Please notice the StopWatch which logs the StringArray containing the params:
EXECUTING COMMAND '[az, vm, extension, set, -g, demo, --vm-name,
demo-cfg01, --name, CustomScript, --publisher,
Microsoft.Azure.Extensions, --settings, '{"commandToExecute":"ls"}']'
The Params looks the same as in the console
The Response from Azure is:
VM has reported a failure when processing extension 'CustomScript'.
Error message: "Enable failed: failed to get configuration: error
reading extension configuration: error parsing settings file: error
parsing json: json: cannot unmarshal string into Go value of type
map[string]interface {}
I think groovy somehow escapes the characters before execution, i cannot figure out what went wrong. Any suggestion?

when you call execute on array groovy (actually java) doublequotes each parameter.
just build your command line as you need in a string
string in groovy has the same execute method as an array...
def cmd = """az vm extension set -g demo --vm-name demo-cfg01 --name CustomScript --publisher Microsoft.Azure.Extensions --settings '{"commandToExecute":"ls"}' """
def process = cmd.execute()
when you use execute on string groovy will execute the exact command you've provided

Found a "workaround". The az command also accept an *.json file as settings parameter. Therefor i first create the command in a temporary json file and passes the json file as parameter. Works!

You are quoting for an .execute() call. You dont need to quote there, because no shell or command parser is involved here.
Your command there gets '{"commandToExecute":"ls"}', which is a valid JSON String (no Map) and this is also what the error message states:
error parsing json: json: cannot unmarshal string into Go value of type map[string]interface
Just use {"commandToExecute": "ls"} (no surrounding ') as argument there.

Related

How to assign json file's content to a variable using powershell

I trying to run below PowerShell script to add azure data factory data sets. But im getting motioned error.
Json File
{
"name": "DSNAME",
"properties": {
"linkedServiceName": {
"referenceName": "REGNAME1",
"type": "LinkedServiceReference"
},
"annotations": [],
"type": "AzureDataExplorerTable",
"schema": [],
"typeProperties": {
"table": "TABLE_TEST"
}
},
"type": "Microsoft.DataFactory/factories/datasets"
}
Powershell
az config set extension.use_dynamic_install=yes_without_prompt
Get-ChildItem "ADF_DATASETS/" -Filter *.json |
Foreach-Object {
$content = Get-Content $_.FullName
az datafactory dataset create --properties $content --name "DATASETNAME" --factory-name "ADFNAME" --resource-group "RG_TEST"
}
Error:
Error detail: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)
Please provide a valid JSON file path or JSON string.
The provided JSON string may have been parsed by the shell. See https://docs.microsoft.com/cli/azure/use-cli-effectively#use-quotation-marks-in-arguments
ERROR: Failed to parse string as JSON:
What kind of text --properties expect?
It is expecting JSON string in double quotation marks. So you have to put the value of $content inside the double quotation mark.
az datafactory dataset create --properties "{\"type\":\"AzureBlob\",\"linkedServiceName\":{\"type\":\"LinkedServiceReference\",\"referenceName\":\"exampleLinkedService\"},\"parameters\":{\"MyFileName\":{\"type\":\"String\"},\"MyFolderPath\":{\"type\":\"String\"}},\"typeProperties\":{\"format\":{\"type\":\"TextFormat\"},\"fileName\":{\"type\":\"Expression\",\"value\":\"#dataset().MyFileName\"},\"folderPath\":{\"type\":\"Expression\",\"value\":\"#dataset().MyFolderPath\"}}}" --name "exampleDataset" --factory-name "exampleFactoryName" --resource-group "exampleResourceGroup"

Azure ARM template for Automation Account Variable - cant use paramets as value

I am trying to create an ARM template for my runbook with additional variables ab packages.
I want to have the values of Automation Account Variables as parameters of ARM template.
When I am using the documentation syntax I am getting the variables value as "[parameters(parameterName)] instead the value of the parameter
when I am not using the syntax code I just get this error:
Invalid JSON - Kindly check the value of the variable
This is the ARM template resource code:
{
"apiVersion": "2020-01-13-preview",
"type": "Microsoft.Automation/automationAccounts/variables",
"name": "[concat(parameters('AutomationAccount'), '/blobContainerName')]",
"location": "[parameters('automationRegion')]",
"properties": {
"description": "output container name",
"isEncrypted": false,
"value": "\"[parameters('blobContainerName')]\""
}
}
how its looks like in the variables after deployment:
Try the following:
"properties": {
"value": "[concat('\"', parameters('blobContainerName'), '\"')]"
},
You need to use concat to join the strings

Azure VM creation failed using downloaded template (The entity name 'vmImageName' is invalid according to its validation rule)

I want to create a vm on azure using az cli.
So I created interactively a VM and downloaded the template and its parameters.
I added some missing parameters, the final script is
az deployment group create --resource-group my-rg --template-file ./azure-32Go.template.json --parameters #./azure-32Go.template.parameters --parameters publicIpAddressName=my-ip networkInterfaceName=my-nic adminPublicKey="$(cat ~/.ssh/id_rsa.pub)" imageName=MsOps-Demo1-image-20210212-175858 --debug
I get an error
msrestazure.azure_exceptions.CloudError: Azure Error: DeploymentFailed
Message: At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.
Exception Details:
Error Code: BadRequest
Message: {'error': {'code': 'InvalidParameter', 'message': "The entity name 'vmImageName' is invalid according to its validation rule: ^[^_\\W][\\w-._]{0,79}(?<![-.])$.", 'target': 'vmImageName'}}
Googling did not point me to someone to get the same issue.
I looked in azur portal Resource Group / Activity log to look at more information, but I only see the same above issue with no more information
How can I fix my code ?
I auto answer this question.
I have redone the whole process, and starting the vm with the template works fine.
As I wanted to set a specific image name. I added a parameter imageName
Now I updated the template.json by adding a parameter
"parameters": {
"location": {
"type": "string"
},
"imageName": {
"type": "string"
},
"networkInterfaceName": {
Updated where it is used
"storageProfile": {
"osDisk": {
"createOption": "fromImage",
"managedDisk": {
"storageAccountType": "[parameters('osDiskType')]"
}
},
"imageReference": {
"id": "/subscriptions/14b6e880-753e-4a5f-8618-c0786702aa1c/resourceGroups/MsOps-Horizon-rg/providers/Microsoft.Compute/images/[parameters('imageName')]"
}
},
But when doing the creation process fails with the same reason
Deployment failed. Correlation ID: 39b18e50-ff0d-4ecd-a3e8-e40b988a341b. {
"error": {
"code": "InvalidParameter",
"message": "The entity name 'vmImageName' is invalid according to its validation rule: ^[^_\\W][\\w-._]{0,79}(?<![-.])$.",
"target": "vmImageName"
}
}
Creation of vm failed
My workaround is therefore to do a sed in the template.json to insert the image name I want :(
My guess is that the [parameter('imageName')] is not expanded early enough which makes the process fails as the name does not match the matching rule

Create Azure VM with artifact switch

I'm trying to create a VM via the CLI and add an artifact from the public repo in a dev test lab.
Here is my script:
az lab vm create --resource-group TestRG --lab-name TestLab --name 'ScriptVM' --image "Windows 10 Pro, Version 1809" --image-type gallery --size 'Standard_B2s' --admin-username 'user' --admin-password 'pw' --artifacts '{ "/artifactsources/publicrepo/artifacts/linux-java" }'
I don't know the format of the artifacts switch though as it just says a JSON encoded array. How does this work? I just get the error
Expecting property name enclosed in double quotes: line 1 column 3 (char 2)
So the answer was I didn't have all the required elements in the JSON. I also fell foul of escaping quotation marks in the PS script. The final switch ended up looking like this:--artifacts '[{"""artifactId""": """artifacts/windows-notepadplusplus"""}]'
The following example shows the sections that make up the basic structure of a definition file:
{
"$schema": "https://raw.githubusercontent.com/Azure/azure-devtestlab/master/schemas/2016-11-28/dtlArtifacts.json",
"title": "",
"description": "",
"iconUri": "",
"targetOsType": "",
"parameters": {
"<parameterName>": {
"type": "",
"displayName": "",
"description": ""
}
},
"runCommand": {
"commandToExecute": ""
}
}
It resides in your Git repo and is referenced in the AZ command. A video of the entire process can be found in our documentation.
https://learn.microsoft.com/en-us/azure/lab-services/devtest-lab-artifact-author

Properly using Terraform External Data Source

I am using Terraform from the bash cloud shell in Azure. I am trying to add an external data source to my Terraform configuration file that will use az cli to query for the virtualip object on a Microsoft.Web/hostingEnvironment the template deploys.
AZ CLI command line:
az resource show --ids /subscriptions/<subscription Id>/resourceGroups/my-ilbase-rg/providers/Microsoft.Web/hos
tingEnvironments/my-ilbase/capacities/virtualip
Output when run from command line:
{
"additionalProperties": {
"internalIpAddress": "10.10.1.11",
"outboundIpAddresses": [
"52.224.70.119"
],
"serviceIpAddress": "52.224.70.119",
"vipMappings": []
},
"id": null,
"identity": null,
"kind": null,
"location": null,
"managedBy": null,
"name": null,
"plan": null,
"properties": null,
"sku": null,
"tags": null,
"type": null
}
In my Terraform config I create a variable for the --ids value:
variable ilbase_resourceId {
default = "/subscriptions/<subscription Id>/resourceGroups/my-ilbase-rg/providers/Microsoft.Web/hostingEnvironments/my-ilbase/capacities/virtualip"
}
I then have the data source structured this way:
data "external" "aseVip" {
program = ["az", "resource", "show", "--ids", "${var.ilbase_resourceId}"]
}
When I execute my configuration, I get the error below:
data.external.aseVip: data.external.aseVip: command "az" produced invalid JSON: json: cannot unmarshal object into Go value of type string
Any ideas what I am doing wrong?
I discovered the problem was that the Terraform External Data Source is not yet able to handle the complex structure of what gets returned by the command. I was able to get around this by adding an AZ CLI command block at the beginning of the script I use to deploy the Application Gateway that grabs the IP address and passes it into the Terraform config as a variable. Below is the script block I am using:
ilbase_virtual_ip=$(
az resource show \
--ids "/subscriptions/$subscription_id/resourceGroups/$ilbase_rg_name/providers/Microsoft.Web/hostingEnvironments/$ilbase_name/capacities/virtualip" \
--query "additionalProperties.internalIpAddress"
)
That command will be successful when you are working in a session. I guess that when you run it from your shell, you already have done az login. When terraform executes your command, it is not using your existing session. You would need to create a PS1 script where you would be propmted for login, or where you provide your credentials so your request can be successful.
Whatever your choice is, take into account that the ONLY output that script should have is a JSON. If any other command add something to the output (for example, when you do a login, you have an output with information about your subscription) then you will have the same error as the output is not a proper JSON. You will need to pipeline that kind of extra ouputs to Out-Null making them "silent" and just write to the output the JSON you are receiving from your request.
I hope this can help.
While there is an accepted response, that is actually a good workaround.
The error is because terraform expect an one level json map like { "a" = "b", "c" = "d" } and your az command returns a multi level map. ( a map of maps )
You can improve your az command to limit the return only one map by adding --query
data "external" "aseVip" {
program = ["az", "resource", "show", "--ids", "${var.ilbase_resourceId}" , "--query additionalProperties" ]
}
output "internalIpAddress" {
value = data.external.aseVip.internalIpAddress
}
output "outboundIpAddresses" {
value = data.external.aseVip.outboundIpAddresses
}
I hope this may help other people.

Resources