I'm trying to create a VM via the CLI and add an artifact from the public repo in a dev test lab.
Here is my script:
az lab vm create --resource-group TestRG --lab-name TestLab --name 'ScriptVM' --image "Windows 10 Pro, Version 1809" --image-type gallery --size 'Standard_B2s' --admin-username 'user' --admin-password 'pw' --artifacts '{ "/artifactsources/publicrepo/artifacts/linux-java" }'
I don't know the format of the artifacts switch though as it just says a JSON encoded array. How does this work? I just get the error
Expecting property name enclosed in double quotes: line 1 column 3 (char 2)
So the answer was I didn't have all the required elements in the JSON. I also fell foul of escaping quotation marks in the PS script. The final switch ended up looking like this:--artifacts '[{"""artifactId""": """artifacts/windows-notepadplusplus"""}]'
The following example shows the sections that make up the basic structure of a definition file:
{
"$schema": "https://raw.githubusercontent.com/Azure/azure-devtestlab/master/schemas/2016-11-28/dtlArtifacts.json",
"title": "",
"description": "",
"iconUri": "",
"targetOsType": "",
"parameters": {
"<parameterName>": {
"type": "",
"displayName": "",
"description": ""
}
},
"runCommand": {
"commandToExecute": ""
}
}
It resides in your Git repo and is referenced in the AZ command. A video of the entire process can be found in our documentation.
https://learn.microsoft.com/en-us/azure/lab-services/devtest-lab-artifact-author
Related
currently I want to create simple automation that will tag my Azure resources based on content of json file. To parse json into tags that can be used inside az tag update I'm using jq with command jq -rj '. | to_entries[] |" \"\(.key)\"=\"\(.value)\""'.
This is returning tags in a format that is consumed by AZ CLI i.e:
"primary_owner"="Owner Name" "primary_email"="owner#owner.com" "deplyment_version"="1.0.0" "project_epic"="" "project_feature"="" "project_name"="DIF" "project_billing_name"="DIF" "cost_center"="" "owner_email"="" "incident_management_application_id"="ServiceNow" "environment"="PROD NEW" "opco"="PROD NEW" "department_name"="DATA and ANALYTICS"
Problem that I'm facing right now is that when I copy-paste those tags into a command
az tag update --operation Replace --resource-id /subscriptions/$SUBSCRIPTON/resourceGroups/$RESOURCE --tags "primary_owner"="Owner Name" "primary_email"="owner#owner.com" "deplyment_version"="1.0.0" "project_epic"="" "project_feature"="" "project_name"="DIF" "project_billing_name"="DIF" "cost_center"="" "owner_email"="" "incident_management_application_id"="ServiceNow" "environment"="PROD NEW" "opco"="PROD NEW" "department_name"="DATA and ANALYTICS"
everything is working fine with output looking like
"properties": {
"tags": {
"cost_center": "",
"department_name": "DATA and ANALYTICS",
"deplyment_version": "1.0.0",
"environment": "PROD NEW",
"incident_management_application_id": "ServiceNow",
"opco": "PROD NEW",
"owner_email": "",
"primary_email": "owner#owner.com",
"primary_owner": "Owner Name",
"project_billing_name": "DIF",
"project_epic": "",
"project_feature": "",
"project_name": "DIF"
}
However whenever I try to get them using any command i.e
az tag update --operation Replace --resource-id /subscriptions/$SUBSCRIPTON/resourceGroups/$RESOURCE --tags $(jq -rj '. | to_entries[] |" \"\(.key)\"=\"\(.value)\""'`)
or by running
jq -rj '. | to_entries[] |" \"\(.key)\"=\"\(.value)\""'` > file
az tag update --operation Replace --resource-id /subscriptions/$SUBSCRIPTON/resourceGroups/$RESOURCE --tags $(less file)
my command is completely messed up and shows every escaped " character and output looks like
"properties": {
"tags": {
"\"cost_center\"": "\"\"",
"\"department_name\"": "\"DATA",
"\"deplyment_version\"": "\"1.0.0\"",
"\"environment\"": "\"PROD",
"\"incident_management_application_id\"": "\"ServiceNow\"",
"\"opco\"": "\"PROD",
"\"owner_email\"": "\"\"",
"\"primary_email\"": "\"owner#owner.com\"",
"\"primary_owner\"": "\"Owner",
"\"project_billing_name\"": "\"DIF\"",
"\"project_epic\"": "\"\"",
"\"project_feature\"": "\"\"",
"\"project_name\"": "\"DIF\"",
"ANALYTICS\"": "",
"NEW\"": "",
"Name\"": "",
"and": ""
}
Any idea what am I doing wrong?
tag my Azure resources based on content of json file.
I have reproduced in my environment and got expected results as below and I followed Microsoft-Document:
$res= Get-Content "Path"
$answer=az tag update --operation Replace --resource-id "XX" --tags $res
$answer | ft
Path- Path of the file or files Location
XX- Resource Id
y.json:
"primary_email"= "owner#owner.com"
"primary_owner"= "Owner-Name"
"project_billing_name"= "DIF"
"project_epic"= " "
"project_feature"= " "
"project_name"= "DIF"
Output:
In Azure Portal(Just to check) Output:
i have azure function with LinuxFxVersion set to DOTNET:
"siteProperties": {
"metadata": null,
"properties": [
{
"name": "LinuxFxVersion",
"value": "DOTNET|3.1"
},
{
"name": "WindowsFxVersion",
"value": null
}
],
"appSettings": null
},
I want to set it to Python:
"siteProperties": {
"metadata": null,
"properties": [
{
"name": "LinuxFxVersion",
"value": "Python|3.9"
},
{
"name": "WindowsFxVersion",
"value": null
}
],
"appSettings": null
},
According to msdn source, I need to use Power shell to change it:
az functionapp config set --name <func_name> --resource-group <rg> --linux-fx-version 'Python|3.9'
but im getting error:
'3.9' is not recognized as an internal or external command,
operable program or batch file.
When im typing just 'Python' i get response:
Operation returned an invalid status 'Bad Request'
How to change linux fx version in Azure Function from .NET to Python?
The way you can solve this error in Powershell is to wrap up the string containing the pipe character with quotes.
Here are multiple examples:
az functionapp config set --name <func_name> --resource-group <rg> --linux-fx-version '"Python|3.9"'
az functionapp config set --name <func_name> --resource-group <rg> --linux-fx-version 'Python"|"3.9'
If you are running the above command in bash use : instead of |
az functionapp config set --name <func_name> --resource-group <rg> --linux-fx-version "Python:3.9"
https://octopus.com/blog/powershell-pipe-escaping
https://github.com/Azure/azure-cli/issues/7874
Using Azure Machine Learning CLI extension, how do we get the Model ID for the latest version of a Model (with known model name)?
To get the entire list of Model Details with a given name the command is
az ml model list --model-name [Model_Name] --resource-group [RGP_NAME] --subscription-id [SUB_ID] --workspace-name [WS_NAME]
Running this will give a list of all the models:
[
{
"createdTime": "2021-03-19T07:02:03.814172+00:00",
"framework": "Custom",
"frameworkVersion": null,
"id": "model:2"
"name": "model",
"version": 3
},
{
"createdTime": "2021-03-19T06:46:34.301054+00:00",
"framework": "Custom",
"frameworkVersion": null,
"id": "model:2",
"name": "model",
"version": 2
},
{
"createdTime": "2021-03-19T06:38:56.558385+00:00",
"framework": "Custom",
"frameworkVersion": null,
"id": "model:1",
"name": "model",
"version": 1
}
]
The Microsoft Documentation mentions, we can use a -l parameter to get the latest version details:
az ml model list --model-name [Model_Name] --resource-group [RGP_NAME] --subscription-id [SUB_ID] --workspace-name [WS_NAME] -l
However, running this gives the following error:
ERROR: UnrecognizedArgumentError: unrecognized arguments: -l
What is the syntax to use this -l flag?
I've just spent a fun few hours wrestling with this. Running something like
az ml model list -w your_workspace_name -g your_resource_group_name -l -n name_of_your_registered_model
will get you back some JSON that looks like this:
[
{
"createdTime": "2022-03-04T16:05:47.103407+00:00",
"framework": "Custom",
"frameworkVersion": null,
"id": "name_of_your_registered_model:3",
"name": "name_of_your_registered_model",
"version": 3
}
]
This is well and good for a human, but not much use to a machine. The azure cli supports something called JMESPath, which allows you to write a query against the result of a CLI command. Running
az ml model list -w your_workspace_name -g your_resource_group_name -l -n name_of_your_registered_model --query "[0].{id:id}" -o tsv
should get you back
"name_of_your_registered_model:3"
Which you can then use in an env var or whatever other use case you have.
If we wish to obtain the model-id for the latest model, instead of using az ml model list with -l flag, using az model show will return the details for the latest model. The syntax to get a string for model-id will be:
az ml model show --model-id $(TRN_MODEL_ID) --resource-group $(AML_TRN_RG) --subscription-id $(AML_TRN_SUB_ID) --workspace-name $(AML_TRN_WS) --query name -o tsv
Currently I fail to run an azure CLI command from Groovy because of the JSON Part in the Command.
There is an azure command to run custom scripts on a virtual machine. The CommandToExecute on the Machine is passes as JSON.
WORKING Example:
REQUEST-CALL in Console:az vm extension set -g demo --vm-name demo-cfg01 --name CustomScript --publisher Microsoft.Azure.Extensions --settings '{"commandToExecute":"ls"}'
RESPONSE: {
"autoUpgradeMinorVersion": true,
"forceUpdateTag": null,
"id": "/subscriptions/xxxxxxxxxx-xxxxxx-xxxx-xxxx-xxxxxxxxxxxxx/resourceGroups/demo/providers/Microsoft.Compute/virtualMachines/demo-cfg01/extensions/CustomScript",
"instanceView": null,
"location": "germanycentral",
"name": "CustomScript",
"protectedSettings": null,
"provisioningState": "Succeeded",
"publisher": "Microsoft.Azure.Extensions",
"resourceGroup": "demo",
"settings": {
"commandToExecute": "ls"
},
"tags": null,
"type": "Microsoft.Compute/virtualMachines/extensions",
"typeHandlerVersion": "2.0",
"virtualMachineExtensionType": "CustomScript"
}
This script works fine.
"Same" Command executed with Groovy leads to following:
def process
StopWatch.withTimeRecording("EXECUTING COMMAND '" + cargs + "'",_logger, Level.ALL) {
process = (cargs).execute(null,null);
process.waitForProcessOutput(sout, serr)
}
Please notice the StopWatch which logs the StringArray containing the params:
EXECUTING COMMAND '[az, vm, extension, set, -g, demo, --vm-name,
demo-cfg01, --name, CustomScript, --publisher,
Microsoft.Azure.Extensions, --settings, '{"commandToExecute":"ls"}']'
The Params looks the same as in the console
The Response from Azure is:
VM has reported a failure when processing extension 'CustomScript'.
Error message: "Enable failed: failed to get configuration: error
reading extension configuration: error parsing settings file: error
parsing json: json: cannot unmarshal string into Go value of type
map[string]interface {}
I think groovy somehow escapes the characters before execution, i cannot figure out what went wrong. Any suggestion?
when you call execute on array groovy (actually java) doublequotes each parameter.
just build your command line as you need in a string
string in groovy has the same execute method as an array...
def cmd = """az vm extension set -g demo --vm-name demo-cfg01 --name CustomScript --publisher Microsoft.Azure.Extensions --settings '{"commandToExecute":"ls"}' """
def process = cmd.execute()
when you use execute on string groovy will execute the exact command you've provided
Found a "workaround". The az command also accept an *.json file as settings parameter. Therefor i first create the command in a temporary json file and passes the json file as parameter. Works!
You are quoting for an .execute() call. You dont need to quote there, because no shell or command parser is involved here.
Your command there gets '{"commandToExecute":"ls"}', which is a valid JSON String (no Map) and this is also what the error message states:
error parsing json: json: cannot unmarshal string into Go value of type map[string]interface
Just use {"commandToExecute": "ls"} (no surrounding ') as argument there.
I am using Terraform from the bash cloud shell in Azure. I am trying to add an external data source to my Terraform configuration file that will use az cli to query for the virtualip object on a Microsoft.Web/hostingEnvironment the template deploys.
AZ CLI command line:
az resource show --ids /subscriptions/<subscription Id>/resourceGroups/my-ilbase-rg/providers/Microsoft.Web/hos
tingEnvironments/my-ilbase/capacities/virtualip
Output when run from command line:
{
"additionalProperties": {
"internalIpAddress": "10.10.1.11",
"outboundIpAddresses": [
"52.224.70.119"
],
"serviceIpAddress": "52.224.70.119",
"vipMappings": []
},
"id": null,
"identity": null,
"kind": null,
"location": null,
"managedBy": null,
"name": null,
"plan": null,
"properties": null,
"sku": null,
"tags": null,
"type": null
}
In my Terraform config I create a variable for the --ids value:
variable ilbase_resourceId {
default = "/subscriptions/<subscription Id>/resourceGroups/my-ilbase-rg/providers/Microsoft.Web/hostingEnvironments/my-ilbase/capacities/virtualip"
}
I then have the data source structured this way:
data "external" "aseVip" {
program = ["az", "resource", "show", "--ids", "${var.ilbase_resourceId}"]
}
When I execute my configuration, I get the error below:
data.external.aseVip: data.external.aseVip: command "az" produced invalid JSON: json: cannot unmarshal object into Go value of type string
Any ideas what I am doing wrong?
I discovered the problem was that the Terraform External Data Source is not yet able to handle the complex structure of what gets returned by the command. I was able to get around this by adding an AZ CLI command block at the beginning of the script I use to deploy the Application Gateway that grabs the IP address and passes it into the Terraform config as a variable. Below is the script block I am using:
ilbase_virtual_ip=$(
az resource show \
--ids "/subscriptions/$subscription_id/resourceGroups/$ilbase_rg_name/providers/Microsoft.Web/hostingEnvironments/$ilbase_name/capacities/virtualip" \
--query "additionalProperties.internalIpAddress"
)
That command will be successful when you are working in a session. I guess that when you run it from your shell, you already have done az login. When terraform executes your command, it is not using your existing session. You would need to create a PS1 script where you would be propmted for login, or where you provide your credentials so your request can be successful.
Whatever your choice is, take into account that the ONLY output that script should have is a JSON. If any other command add something to the output (for example, when you do a login, you have an output with information about your subscription) then you will have the same error as the output is not a proper JSON. You will need to pipeline that kind of extra ouputs to Out-Null making them "silent" and just write to the output the JSON you are receiving from your request.
I hope this can help.
While there is an accepted response, that is actually a good workaround.
The error is because terraform expect an one level json map like { "a" = "b", "c" = "d" } and your az command returns a multi level map. ( a map of maps )
You can improve your az command to limit the return only one map by adding --query
data "external" "aseVip" {
program = ["az", "resource", "show", "--ids", "${var.ilbase_resourceId}" , "--query additionalProperties" ]
}
output "internalIpAddress" {
value = data.external.aseVip.internalIpAddress
}
output "outboundIpAddresses" {
value = data.external.aseVip.outboundIpAddresses
}
I hope this may help other people.