How to change pipeline variables for usage in the next build in Azure DevOps - azure

The case I have is as follow:
I've created an Azure DevOps Pipeline with a Pipeline variable, let's say 'variable A'. The value of 'variable A' is 1. During the build, I change the value of 'variable A' to 2.
When the build runs for the second time I want the value of these 'variable A' but this is back 1 but I want that the value is 2 because on the previous build I set the value of 'variable A' to 2.
These are the methods I tried without success:
Method 1:
Write-Host "##vso[task.setvariable variable=A;]2"
Method 2:
$env:A = 2
The only thing that works but I don't think this is the way to go is to get the whole build definition via the rest api and put it back with the value of the variable changed.
Get
Update
Is there any other solution to this problem?

If you're specifically looking at an increasing number, then you can also use counters. These only woork in YAML based build definitions.
The format is as follows:
You can use any of the supported expressions for setting a variable. Here is an example of setting a variable to act as a counter that starts at 100, gets incremented by 1 for every run, and gets reset to 100 every day.
yaml
jobs:
- job:
variables:
a: $[counter(format('{0:yyyyMMdd}', pipeline.startTime), 100)]
steps:
- bash: echo $(a)
For more information about counters and other expressions, see expressions.
The counter is stored for the pipeline and is based on the prefix you provide in the counterr expression. The above expression uses the yyyymmdd to generate a prefix which is unique every day.
For UI driven build definitions, then indeed using the REST api to update the whole definiton would work, though it's really hard to work around all possibilities concerning paralelism.

How to change pipeline variables for usage in the next build in Azure DevOps
I am afraid you have to use the rest api to change the value of that pipeline variables.
That because when you use the script `"##vso[task.setvariable variable=testvar;]testvalue" to overwrite it, the overwrite value only work for current build pipeline.
When you use the execute the build again, it will still pull the value from pipeline variable value.
So, we have to update the value of that variables on the web portal. Then we need use the need the REST API (Definitions - Update) to update the value of the build pipeline definition variable from a build task:
Similar thread: How to modify Azure DevOps release definition variable from a release task?
Note:Change the API to the build definitions:
PUT https://dev.azure.com/{organization}/{project}/_apis/build/definitions/{definitionId}?api-version=5.0
Hope this helps.

I have found the easiest way to update variable values during a pipeline execution is to use the Azure CLI having also tried other methods with little or no success.
In a YAML pipeline, this may look something like so:
jobs:
- job: Update_Version
steps:
- task: AzureCLI#2
inputs:
azureSubscription: [your_subscription_id]
scriptType: 'pscore'
scriptLocation: 'inlineScript'
inlineScript: |
# set environment variable for current process
$env:AZURE_DEVOPS_EXT_PAT = $env:SYSTEM_ACCESSTOKEN
$oldVersionNumber = $(version-number)
$newVersionNumber = $oldVersionNumber + 1
az pipelines variable-group variable update --group-id [your_variable_group_id] --name version-number --organization [your_organization_url] --project [your_project_name] --value $newVersionNumber
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
The pipeline build service may also need permission to execute this command. To check, go to Pipelines -> Library -> Variable groups then edit the variable group containing your variable. Click on the Security button and make sure the user Project Collection Build Service has the Administrator role.
More information on the Azure CLI command can be found here. There is also another form of the command used to update variables that are not in a variable group, as described here.

Related

Return a value from node.js script in azure pipeline?

In Azure Pipelines, I see that you can access the environment variables from scripts in node.js during a pipeline run. However, I want to actually return a value and then capture/use it.
Does anyone know how to do this? I can't find any references on how to do this in documentation.
For consistency's sake it'd be nice to use node scripts for everything and not go back and forth between node and bash.
Thanks
Okay I finally figured this out. Azure documentation is a bit confusing on the topic, but my approach was what follows. In this example, I'm going to make a rather pointless simple script that sets a variable whose value is the name of the source branch, but all lower case.
1) Define your variable
Defining a variable can be done simply (though there is a lot of depth to how variables are used and I suggest consulting Azure documentation on variable creation for more). However, at the top of your pipeline yaml file you can define it as such:
variables
lowerCaseBranchName: ''
This creates an empty variable for use across your jobs. We'll use this variable as our example.
2) Create your script
"Returning a value" from your script simply means outputting it via node's stdout, the output of which will be consumed by the task to set it as a pipeline variable.
An important thing to remember is that any environment variables from the pipeline can be used within node, they are just reformatted and moved under node's process.env global. For instance, the commonly used Build.SourceBranchName environment variable in azure pipelines is accessible in your node script via its alias process.env.BUILD_SOURCEBRANCHNAME. This uppercase name transformation should be uniform across all environment variables.
Here's an example node.js script:
const lowerCaseBranchName = process.env.BUILD_SOURCEBRANCHNAME.toLowerCase();
process.stdout.write(lowerCaseBranchName);
3) Consume the output in the relevant step in azure pipelines
To employ that script in a job step, call it with a script task. Remember that a script task is, in this case, a bash script (though you can use others) that runs node as a command as it sets the value of our variable:
- script: |
echo "##vso[task.setvariable variable=lowerCaseBranchName]$(node path/to/your/script)"
displayName: 'Get lower case branch name'
Breaking down the syntax
Using variable definition syntax is, in my opinion extremely ugly, but pretty easy to use once you understand it. The basic syntax for setting a variable in a script is the following:
##vso[task.setvariable variable=SOME_VARIABLE_NAME]SOME_VARIABLE_VALUE
Above, SOME_VARIABLE_NAME is the name of our variable (lowerCaseBranchName) as defined in our azure pipeline configuration at the beginning. Likewise, SOME_VARIABLE_VALUE is the value we want to set that variable to.
You could do an additional line above this line to create a variable independently that you can then use to set the env variable with, however I chose to just inline the script call as you can see in the example above usign the $() syntax.
That's it. In following tasks, the environment variable lowerCaseBranchName can be utilized using any of the variable syntaxes such as $(lowerCaseBranchName),
Final result
Defining our variable in our yaml file:
variables
lowerCaseBranchName: ''
Our nodejs script:
const lowerCaseBranchName = process.env.BUILD_SOURCEBRANCHNAME.toLowerCase();
process.stdout.write(lowerCaseBranchName);
Our pipeline task implementation/execution of said script:
- script: |
echo "##vso[task.setvariable variable=lowerCaseBranchName]$(node path/to/your/script)"
displayName: 'Get lower case branch name'
A following task using its output:
- script: |
echo "$(lowerCaseBranchName)"
displayName: 'Output lower case branch name'
This will print the lower-cased branch name to the pipline console when it runs.
Hope this helps somebody! Happy devops-ing!

Azure Pipelines - Setting Custom Variable to Build Number

In my Azure Build Pipeline (classic, not YAML) I set my build number to be the branch name and then a revision number variable. This was my process for that:
Pipelines -> Pipelines -> {my pipeline} -> Edit -> Options -> Build Number Format
$(SourceBranchName)$(Rev:.r)
In my testing, that works great.
Now, in my Release Pipeline, the first script I run is a PowerShell script that takes the build number, and applies it to a local variable (MyBuild) I created. The script is as follows:
Write-Host "Pipeline = $($pipeline | ConvertTo-Json -Depth 100)"
$buildNumber = $Env:BUILD_BUILDNUMBER
$pipeline.variables.MyBuild.value = $buildNumber
This variable is used later in the pipeline to create a folder that houses my release files.
$(BuildDirectory)/$(MyBuild)/Debug
For some reason, my variable is always one build behind. For example, if my build number is master.5, the folder that is created by my Release Pipeline is master.4. I have tried changing the order my scripts are in the pipeline, but that doesn't solve anything. It is weird because my Build Pipeline is correct (always named properly, ex. master.1, master.2, master.3, etc.) but my Release Pipeline variable is always one revision behind.
Powershell script to update the custom build number
- powershell: |
[string]$version="$(Build.Repository.Name)_SomeCustomData_$(Build.BuildId)"
Write-Output "##vso[build.updatebuildnumber]$(Version)"
displayName: Set Build Number
I tested it and it works well. Below is my reproduction, you can refer to:
In release pipeline :
Write-Host '##vso[task.setvariable variable=MyBuild]$(Build.BuildNumber)'
md $(Agent.ReleaseDirectory)/$env:MyBuild/Debug
Select build source as release artifact, default version select Latest, enable Continuous deployment trigger. This creates a release every time a new build is available.
Test reuslt:
In addition, the point I am confused about is how do you use the $(BuildDirectory) in the release pipeline? Agent.BuildDirectory:
The local path on the agent where all folders for a given build pipeline are created. This predefined variable should not be available in the release pipeline, we should use Agent.ReleaseDirectory.You can refer to predefined variable.

Declaration and usage of Output Variable in Azure Devops

I'm creating a Continuous Integration pipeline that uses Bash script tasks in order to create the initial variables for runtime.
I have a variable that I call: datebuild, which is formatted accordingly : $(date +%Y%m%d_%H%M%S).
Currently I'm using the pipeline variable that's how I'm declaring it
When using the datebuild variable under Bash#3 task, it successfully formatting it.
Afterwards I want to take the formatted output in order to use it on different tasks inside one agent job.
On the second task I need to copy file to the Artifact Staging Directory:
20200423_141808 is the file and Artifact Staging Directory is the Destination Directory, for example.
I've been reading that it can be used with feature called Output Variables.
Created the reference variable named: ref1, and on the task I want to take the output variable I'm using the ref1.datebuild in order to access the variable
Used the following documentation in order to use the output variable it doesn't seem to work.
here's the task inside the pipeline:
Trying to understand What I'm missing.
You can take the formatted date and set it as a variable for the next steps in the job.
For example, in YAML pipeline:
variables:
datebuild: '$(date +%Y%m%d_%H%M%S)'
steps:
- bash: |
formated="$(datebuild)"
echo "##vso[task.setvariable variable=formatedDate]$formated"
- bash: |
echo $(formatedDate)
In the editor:
The second bash task output is:

Azure DevOps Release Pipeline - How to set and pass values in variables in pipeline with Azure Powershell

With Azure Release Pipeline, in a task using the PowerShell Script, I am able to set values of variables and pass to next task using the command
Write-Host '##vso[task.setvariable variable=varResourceExists;isOutput=true;something'
However, when I put this similar command in a task that uses Azure PowerShell, this command is no longer allowed, the task produces a warning:
2019-10-22T00:23:14.3080614Z ##[warning]'##vso[task.setvariable
variable=varResourceExists;isOutput=true;something' contains logging
command keyword '##vso', but it's not a legal command. Please see the
list of accepted commands:
https://go.microsoft.com/fwlink/?LinkId=817296
As a result, the variable varResourceExists cannot be set by my task. I have also tried a conventional PowerShell set value by doing
$varResourceExists = 'something'; # this also does not work
Is there a way I can set this value in Azure Powershell script so that the next task can reference it?
##vso[task.setvariable variable=varResourceExists;isOutput=true;something is not correct syntax. You're missing a closing ].
It should be ##vso[task.setvariable variable=varResourceExists;isOutput=true;]something
Here is how I solved my topic. In the pipeline Azure PowerShell task, I can have code such as Write-Host '##vso[task.setvariable variable=varResourceExists;isOutput=true;]False';
In the Output Variable option, I set a Reference Name "step1":
Output Variable
Then in the next step, I can do a conditional check using a Custom Condition:
Custom Condition
I can also reference the variable in my code such as Write-Host "The step1.varResourceExists says: $(step1.varResourceExists)";

Merge inner parameter struct when using template - azure pipelines

I have a lot of default parameters in my template. I want to categorize them.
# template.yml
parameters:
azure:
name: cargo_test # Default job name
displayName: Cargo test # Default displayName
condition: true # Job condition
strategy: # Default strategy to test on Windows, MacOs and Linux.
matrix:
Linux:
vmImage: ubuntu-16.04
MacOS:
vmImage: macOS-10.13
Windows:
vmImage: vs2017-win2016
name: job_name
default_parameter1: default1
default_parameter2: defualt2
# rest of code
- job:A
template: template.yml
parameters:
azure:
name: test_name
This cause parameter.azure contains only one field name. I want to overwrite parameters.azure.name not all parameters.azure struct. Is it possible in azure pipelines?
I want to overwrite azure.name not all azure struct.
It seems that you are worrying if you just overwrite one parameter in .yml file which is using other template.yml file, it will affect all azure struct, right?
If so, you don't need worry about this. As what you defined in template.yml file, it has lots of parameters. After you use it in other .yml file: name: test_name , it only overwrite the value of parameter name with no effect on other parameters, and also this overwrite only available on current job.
For example, if in your use-template.yml:
- job:A
template: template.yml
parameters:
azure:
name: test_name
- job:B
template: template.yml
parameters:
azure:
condition: failed()
The overwriting of name, will only affect this parameter(name) value in Job A. After Job A finished, the value of name will reback to cargo_test in Job B.
In one word, the configuration in template.yml is fixed, the used in other yml will have any affect to the template.yml. So, you don't need to worry about how to categorize parameters which we does not support it until now.
You can check this simple example in official doc: Job templates.If have any misunderstanding about your idea, just feel free to correct me.
Updated:
Since we can get the value with parameters.azure.name, the Azure Devops should support these parameters categorize. And also, after tested, I got the same result with you. If overwrite parameters.azure.name, the rest parameters which in the same level with parameters.azure.name are all empty. I think this should be a issue which need our Product Group to fix it.
I have raise this issue report on our official Developer Community: When I overwrite the template parameter, the value be empty. You can follow this ticket, so that you can get the notification once it has any updated.
In addition, it seems no other work around to achieve parameters categorize. Just patience for this issue fixed. Once the fixed script release, our engineer would inform it in that ticket.

Resources