Read/Save Azure DevOps pipeline variable in shell script - azure

Question

I am using Azure DevOps for some application deployment. I need to have a saved variable build number that would update every time I do a build successfully and send the apk/ipa to the store.
Right now, from what I read in the Azure documentation and other post on StackOverflow about this, I setup my scripts this way.
This is my pipeline variable
This is my current script
the output is:
So, it seems to update my local variable but not my pipeline variable. I am unsure why since this is the example provided EVERYWHERE.
Sources:
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/set-variables-scripts?view=azure-devops&tabs=bash
how to increment and save variable in azure devops pipeline
Thank you for the help!
Edit 1: Ok, so it seems that there is a variable/function called counter. I haven't figured out how to use it yet, but looking into it.
Edit 2:
Updated my azure-pipelines.yml
variables:
major: 1
minor: 0
patch: 0
build: $[counter(variables['patch'], 1)]
On my pipeline it looks like this
and my fastlane (ruby script ) lane looks like this
lane :tf do
`echo $major`
`echo $minor`
`echo $patch`
`echo $build` # Nothing
`echo $MAJOR`
`echo $MINOR`
`echo $PATCH`
`echo $BUILD` # Nothing
`echo $(major)` # fails
end
those show nothing.
This azure DevOps is very depressing. It says here I can do a bash call to this variable.

Answer 1

Variables used with macro syntax are expanded in runtime before task is executed, that is why there is value '1' in last log, even when value previously was set in previous step to '2'.
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#runtime-expression-syntax
Try to use runtime expression instead, it is expanded in runtime

Answer 2

Two solutions for you.
1. Use counter expression.
counter(<prefix>, <feed>)
counter expression is for pipeline, and it has two parameters, prefix and seed. Seed is based on the prefix.
When the prefix is been set and run for the first time, the counter result will start from the feed value. But for the later run based on the same prefix, the counter result will ignore the feed value, it will be 'last time counter result + 1'
2, Change the pipeline definition directly.
For example, I can use the below python code to get and change the variable of classic pipeline:
import json
import requests
org_name = "xxx"
project_name = "xxx"
pipeline_definition_id = "xxx"
personal_access_token = "xxx"
key = 'variables'
var_name = 'BUILDNUMBER'
url = "https://dev.azure.com/"+org_name+"/"+project_name+"/_apis/build/definitions/"+pipeline_definition_id+"?api-version=6.0"
payload={}
headers = {
'Authorization': 'Basic '+personal_access_token
}
response = requests.request("GET", url, headers=headers, data=payload)
print(response.text)
json_content = response.text
def get_content_of_json(json_content, key, var_name):
data = json.loads(json_content)
return data[key][var_name].get('value')
def change_content_of_json(json_content, key, var_name):
data = json.loads(json_content)
data[key][var_name]['value'] = str(int(get_content_of_json(json_content,key,var_name)) + 1)
return data
json_data = change_content_of_json(json_content, key, var_name)
url2 = "https://dev.azure.com/"+org_name+"/"+project_name+"/_apis/build/definitions/"+pipeline_definition_id+"?api-version=6.0"
payload2 = json.dumps(json_data)
headers2 = {
'Authorization': 'Basic '+personal_access_token,
'Content-Type': 'application/json'
}
response2 = requests.request("PUT", url2, headers=headers2, data=payload2)
Write a JSON demo for you, you can import it in your DevOps and design yourself classic pipeline based on this:
{
"options": [
{
"enabled": false,
"definition": {
"id": "5d58cc01-7c75-450c-be18-a388ddb129ec"
},
"inputs": {
"branchFilters": "[\"+refs/heads/*\"]",
"additionalFields": "{}"
}
},
{
"enabled": false,
"definition": {
"id": "a9db38f9-9fdc-478c-b0f9-464221e58316"
},
"inputs": {
"workItemType": "Bug",
"assignToRequestor": "true",
"additionalFields": "{}"
}
}
],
"variables": {
"BUILDNUMBER": {
"value": "7"
},
"system.debug": {
"value": "false",
"allowOverride": true
}
},
"properties": {},
"tags": [],
"_links": {
"self": {
"href": "https://dev.azure.com/BowmanCP/c6358b04-e91a-4bd1-a894-1adb543134d6/_apis/build/Definitions/359?revision=11"
},
"web": {
"href": "https://dev.azure.com/BowmanCP/c6358b04-e91a-4bd1-a894-1adb543134d6/_build/definition?definitionId=359"
},
"editor": {
"href": "https://dev.azure.com/BowmanCP/c6358b04-e91a-4bd1-a894-1adb543134d6/_build/designer?id=359&_a=edit-build-definition"
},
"badge": {
"href": "https://dev.azure.com/BowmanCP/c6358b04-e91a-4bd1-a894-1adb543134d6/_apis/build/status/359"
}
},
"jobAuthorizationScope": 2,
"jobTimeoutInMinutes": 60,
"jobCancelTimeoutInMinutes": 5,
"process": {
"phases": [
{
"steps": [
{
"environment": {},
"enabled": true,
"continueOnError": false,
"alwaysRun": false,
"displayName": "PowerShell Script",
"timeoutInMinutes": 0,
"retryCountOnTaskFailure": 0,
"condition": "succeeded()",
"task": {
"id": "e213ff0f-5d5c-4791-802d-52ea3e7be1f1",
"versionSpec": "2.*",
"definitionType": "task"
},
"inputs": {
"targetType": "inline",
"filePath": "",
"arguments": "",
"script": "# Write your PowerShell commands here.\n\nWrite-Host \"Hello World\"\n\npip install requests\n",
"errorActionPreference": "stop",
"warningPreference": "default",
"informationPreference": "default",
"verbosePreference": "default",
"debugPreference": "default",
"progressPreference": "silentlyContinue",
"failOnStderr": "false",
"showWarnings": "false",
"ignoreLASTEXITCODE": "false",
"pwsh": "false",
"workingDirectory": "",
"runScriptInSeparateScope": "false"
}
},
{
"environment": {},
"enabled": true,
"continueOnError": false,
"alwaysRun": false,
"displayName": "Run a Python script",
"timeoutInMinutes": 0,
"retryCountOnTaskFailure": 0,
"condition": "succeeded()",
"task": {
"id": "6392f95f-7e76-4a18-b3c7-7f078d2f7700",
"versionSpec": "0.*",
"definitionType": "task"
},
"inputs": {
"scriptSource": "inline",
"scriptPath": "",
"script": "import json\nimport requests\n\n\norg_name = \"BowmanCP\"\nproject_name = \"BowmanCP\"\npipeline_definition_id = \"359\"\npersonal_access_token = \"OnhlbXFzd29hdXJrdGhvNGJhemJza3hhenZldnRhbXhhZTVhNDMycXZoNzRicmo3YTZjc3E=\"\n\nkey = 'variables'\nvar_name = 'BUILDNUMBER'\n\nurl = \"https://dev.azure.com/\"+org_name+\"/\"+project_name+\"/_apis/build/definitions/\"+pipeline_definition_id+\"?api-version=6.0\"\n\npayload={}\nheaders = {\n 'Authorization': 'Basic '+personal_access_token\n}\n\nresponse = requests.request(\"GET\", url, headers=headers, data=payload)\nprint(response.text)\njson_content = response.text\ndef get_content_of_json(json_content, key, var_name):\n data = json.loads(json_content)\n return data[key][var_name].get('value')\n\ndef change_content_of_json(json_content, key, var_name):\n data = json.loads(json_content)\n data[key][var_name]['value'] = str(int(get_content_of_json(json_content,key,var_name)) + 1)\n return data\n\njson_data = change_content_of_json(json_content, key, var_name)\n\n\nurl2 = \"https://dev.azure.com/\"+org_name+\"/\"+project_name+\"/_apis/build/definitions/\"+pipeline_definition_id+\"?api-version=6.0\"\n\npayload2 = json.dumps(json_data)\nheaders2 = {\n 'Authorization': 'Basic '+personal_access_token,\n 'Content-Type': 'application/json'\n}\n\nresponse2 = requests.request(\"PUT\", url2, headers=headers2, data=payload2)\n",
"arguments": "",
"pythonInterpreter": "",
"workingDirectory": "",
"failOnStderr": "false"
}
}
],
"name": "Agent job 1",
"refName": "Job_1",
"condition": "succeeded()",
"target": {
"executionOptions": {
"type": 0
},
"allowScriptsAuthAccessOption": false,
"type": 1
},
"jobAuthorizationScope": 2
}
],
"target": {
"agentSpecification": {
"identifier": "windows-2019"
}
},
"type": 1
},
"repository": {
"properties": {
"cleanOptions": "0",
"labelSources": "0",
"labelSourcesFormat": "$(build.buildNumber)",
"reportBuildStatus": "true",
"fetchDepth": "1",
"gitLfsSupport": "false",
"skipSyncSource": "false",
"checkoutNestedSubmodules": "false"
},
"id": "421488b2-be68-4b8e-8faf-8302da314071",
"type": "TfsGit",
"name": "XunitTest_Auto",
"url": "https://dev.azure.com/BowmanCP/BowmanCP/_git/XunitTest_Auto",
"defaultBranch": "refs/heads/main",
"clean": "false",
"checkoutSubmodules": false
},
"processParameters": {},
"quality": 1,
"authoredBy": {
"displayName": "Bowman Zhu",
"url": "https://spsprodsea2.vssps.visualstudio.com/A64545e3d-c12d-4c81-b77f-4de83783d9bd/_apis/Identities/af91e22a-cc35-4c8e-8af3-f49c4a1b9b6a",
"_links": {
"avatar": {
"href": "https://dev.azure.com/BowmanCP/_apis/GraphProfile/MemberAvatars/aad.ZGU3N2NiY2YtZTgzYy03ZDkwLWI0YTYtOTk3Nzg3NDczMzBl"
}
},
"id": "af91e22a-cc35-4c8e-8af3-f49c4a1b9b6a",
"uniqueName": "xxx#xxx.com",
"imageUrl": "https://dev.azure.com/BowmanCP/_apis/GraphProfile/MemberAvatars/aad.ZGU3N2NiY2YtZTgzYy03ZDkwLWI0YTYtOTk3Nzg3NDczMzBl",
"descriptor": "aad.ZGU3N2NiY2YtZTgzYy03ZDkwLWI0YTYtOTk3Nzg3NDczMzBl"
},
"drafts": [],
"queue": {
"_links": {
"self": {
"href": "https://dev.azure.com/BowmanCP/_apis/build/Queues/18"
}
},
"id": 18,
"name": "Azure Pipelines",
"url": "https://dev.azure.com/BowmanCP/_apis/build/Queues/18",
"pool": {
"id": 9,
"name": "Azure Pipelines",
"isHosted": true
}
},
"id": 359,
"name": "ChangeVariable",
"url": "https://dev.azure.com/BowmanCP/c6358b04-e91a-4bd1-a894-1adb543134d6/_apis/build/Definitions/359?revision=11",
"uri": "vstfs:///Build/Definition/359",
"path": "\\",
"type": 2,
"queueStatus": 0,
"revision": 11,
"createdDate": "2022-11-07T10:14:18.003Z",
"project": {
"id": "c6358b04-e91a-4bd1-a894-1adb543134d6",
"name": "BowmanCP",
"url": "https://dev.azure.com/BowmanCP/_apis/projects/c6358b04-e91a-4bd1-a894-1adb543134d6",
"state": 1,
"revision": 178,
"visibility": 0,
"lastUpdateTime": "2022-09-05T06:02:02.693Z"
}
}

Answer 3

it seems to update my local variable but not my pipeline variable
Refer to this doc: Set variables in scripts
A script in your pipeline can define a variable so that it can be consumed by one of the subsequent steps in the pipeline.Set variables in scripts
The method you used to update the Pipeline Variable value is correct.
It will not work on the current task, but the updated value can be used in the next tasks.
For example:
steps:
- bash: |
echo $(TestVariable)
echo "##vso[task.setvariable variable=TestVariable;]2"
displayName: 'Bash Script'
- bash: |
echo $(TestVariable)
For the requirement about using counter expression, you can refer to my another ticket: Azure DevOps: release version

Answer 4

I eventually used this formula that works without having to hack the NSA and FBI to get a build number updated. It's not exactly what I wanted, but whatever, I will deal with this atm.
then using the ENV['BUILDNUMBER'] in my ruby script, and it reads the env variable with the counter. It is the type of solution I needed, although it doesn't do exactly what I want.