In my Azure Build Pipeline (classic, not YAML) I set my build number to be the branch name and then a revision number variable. This was my process for that:
Pipelines -> Pipelines -> {my pipeline} -> Edit -> Options -> Build Number Format
$(SourceBranchName)$(Rev:.r)
In my testing, that works great.
Now, in my Release Pipeline, the first script I run is a PowerShell script that takes the build number, and applies it to a local variable (MyBuild) I created. The script is as follows:
Write-Host "Pipeline = $($pipeline | ConvertTo-Json -Depth 100)"
$buildNumber = $Env:BUILD_BUILDNUMBER
$pipeline.variables.MyBuild.value = $buildNumber
This variable is used later in the pipeline to create a folder that houses my release files.
$(BuildDirectory)/$(MyBuild)/Debug
For some reason, my variable is always one build behind. For example, if my build number is master.5, the folder that is created by my Release Pipeline is master.4. I have tried changing the order my scripts are in the pipeline, but that doesn't solve anything. It is weird because my Build Pipeline is correct (always named properly, ex. master.1, master.2, master.3, etc.) but my Release Pipeline variable is always one revision behind.
Powershell script to update the custom build number
- powershell: |
[string]$version="$(Build.Repository.Name)_SomeCustomData_$(Build.BuildId)"
Write-Output "##vso[build.updatebuildnumber]$(Version)"
displayName: Set Build Number
I tested it and it works well. Below is my reproduction, you can refer to:
In release pipeline :
Write-Host '##vso[task.setvariable variable=MyBuild]$(Build.BuildNumber)'
md $(Agent.ReleaseDirectory)/$env:MyBuild/Debug
Select build source as release artifact, default version select Latest, enable Continuous deployment trigger. This creates a release every time a new build is available.
Test reuslt:
In addition, the point I am confused about is how do you use the $(BuildDirectory) in the release pipeline? Agent.BuildDirectory:
The local path on the agent where all folders for a given build pipeline are created. This predefined variable should not be available in the release pipeline, we should use Agent.ReleaseDirectory.You can refer to predefined variable.
Related
I have a GitLab CI pipeline with a 'migration' job which I want to be added only if certain files changed between current commit and master branch, but in my current project I'm forced to use GitLab CI pipelines for push event which complicates things.
Docs on rules:changes clearly states that it will glitch and will not work properly without MR (my case of push event), so that's out of question.
Docs on rules:if states that it only works with env variables. But docs on passing CI/CD variables to another job clearly states that
These variables cannot be used as CI/CD variables to configure a
pipeline, but they can be used in job scripts.
So, now I'm stuck. I can just skip running the job in question overriding the script and checking for file changes, but what I want is not adding the job in question to pipeline in first place.
While you can't add a job alone to a pipeline based on the output of a script, you can add a child pipeline dynamically based on the script output. The method of using rules: with dynamic variables won't work because rules: are evaluated at the time the pipeline is created, as you found in the docs.
However, you can achieve the same effect using dynamic child-pipelines feature. The idea is you dynamically create the YAML for the desired pipeline in a job. That YAML created by your job will be used to create a child pipeline, which your pipeline can depend on.
Sadly, to add/remove a Gitlab job based on variables created from a previous job is not possible for a given pipeline
A way to achieve this is if your break your current pipeline to an upstream and downstream
The upstream will have 2 jobs
The first one will use your script to define a variable
This job will trigger the downstream, passing this variable
Upstream
check_val:
...
script:
... Script imposes the logic with the needed checks
... If true
- echo "MY_CONDITIONAL_VAR=true" >> var.env
... If false
- echo "MY_CONDITIONAL_VAR=false" >> var.env
artifacts:
reports:
dotenv: var.env
trigger_your_original_pipeline:
...
variables:
MY_CONDITIONAL_VAR: "$MY_CONDITIONAL_VAR"
trigger:
project: "project_namespance/project"
The downstream would be your original pipeline
Downstream
...
migration:
...
rules:
- if: '$MY_CONDITIONAL_VAR == "true"'
Now the MY_CONDITIONAL_VAR will be available at the start of the pipeline, so you can impose rules to add or not the migration job
I'm creating a Continuous Integration pipeline that uses Bash script tasks in order to create the initial variables for runtime.
I have a variable that I call: datebuild, which is formatted accordingly : $(date +%Y%m%d_%H%M%S).
Currently I'm using the pipeline variable that's how I'm declaring it
When using the datebuild variable under Bash#3 task, it successfully formatting it.
Afterwards I want to take the formatted output in order to use it on different tasks inside one agent job.
On the second task I need to copy file to the Artifact Staging Directory:
20200423_141808 is the file and Artifact Staging Directory is the Destination Directory, for example.
I've been reading that it can be used with feature called Output Variables.
Created the reference variable named: ref1, and on the task I want to take the output variable I'm using the ref1.datebuild in order to access the variable
Used the following documentation in order to use the output variable it doesn't seem to work.
here's the task inside the pipeline:
Trying to understand What I'm missing.
You can take the formatted date and set it as a variable for the next steps in the job.
For example, in YAML pipeline:
variables:
datebuild: '$(date +%Y%m%d_%H%M%S)'
steps:
- bash: |
formated="$(datebuild)"
echo "##vso[task.setvariable variable=formatedDate]$formated"
- bash: |
echo $(formatedDate)
In the editor:
The second bash task output is:
So I have been working off of master for a while, and just recently added a 'release' branch that I will be working off of from now on.
In my Release Pipeline I have a PowerShell script that sets a custom variable using predefined variables.
$branchName = $Env:BUILD_SOURCEBRANCHNAME
$buildNumber = $Env:BUILD_BUILDNUMBER
$release = $branchName + "." + $buildNumber.ToString()
$pipeline.variables.NameVar.value = $release
If I push code to my release branch, this script will run at the end of my pipeline, and the variable should be changed to release.xxxx, but it is changed to master.xxxx.
Is there a reason the build variable build.sourcebranchname does not return my release branch name, and instead returns master? The build.buildnumber variable returns the correct value.
Try to go to Get sources and check whether you select the correct branch.
I've tested your script and it returned expected result:
So I figured out my issue. After adding the new release branch, I needed to edit my build pipeline to setup multiple branches. Basically following this doc from Microsoft.
Adding release/* as a branch filter allowed the build pipeline to build on the release branch, and not just the master branch. From that point on, when I used the build variables in my release pipeline, they all returned the proper value.
The case I have is as follow:
I've created an Azure DevOps Pipeline with a Pipeline variable, let's say 'variable A'. The value of 'variable A' is 1. During the build, I change the value of 'variable A' to 2.
When the build runs for the second time I want the value of these 'variable A' but this is back 1 but I want that the value is 2 because on the previous build I set the value of 'variable A' to 2.
These are the methods I tried without success:
Method 1:
Write-Host "##vso[task.setvariable variable=A;]2"
Method 2:
$env:A = 2
The only thing that works but I don't think this is the way to go is to get the whole build definition via the rest api and put it back with the value of the variable changed.
Get
Update
Is there any other solution to this problem?
If you're specifically looking at an increasing number, then you can also use counters. These only woork in YAML based build definitions.
The format is as follows:
You can use any of the supported expressions for setting a variable. Here is an example of setting a variable to act as a counter that starts at 100, gets incremented by 1 for every run, and gets reset to 100 every day.
yaml
jobs:
- job:
variables:
a: $[counter(format('{0:yyyyMMdd}', pipeline.startTime), 100)]
steps:
- bash: echo $(a)
For more information about counters and other expressions, see expressions.
The counter is stored for the pipeline and is based on the prefix you provide in the counterr expression. The above expression uses the yyyymmdd to generate a prefix which is unique every day.
For UI driven build definitions, then indeed using the REST api to update the whole definiton would work, though it's really hard to work around all possibilities concerning paralelism.
How to change pipeline variables for usage in the next build in Azure DevOps
I am afraid you have to use the rest api to change the value of that pipeline variables.
That because when you use the script `"##vso[task.setvariable variable=testvar;]testvalue" to overwrite it, the overwrite value only work for current build pipeline.
When you use the execute the build again, it will still pull the value from pipeline variable value.
So, we have to update the value of that variables on the web portal. Then we need use the need the REST API (Definitions - Update) to update the value of the build pipeline definition variable from a build task:
Similar thread: How to modify Azure DevOps release definition variable from a release task?
Note:Change the API to the build definitions:
PUT https://dev.azure.com/{organization}/{project}/_apis/build/definitions/{definitionId}?api-version=5.0
Hope this helps.
I have found the easiest way to update variable values during a pipeline execution is to use the Azure CLI having also tried other methods with little or no success.
In a YAML pipeline, this may look something like so:
jobs:
- job: Update_Version
steps:
- task: AzureCLI#2
inputs:
azureSubscription: [your_subscription_id]
scriptType: 'pscore'
scriptLocation: 'inlineScript'
inlineScript: |
# set environment variable for current process
$env:AZURE_DEVOPS_EXT_PAT = $env:SYSTEM_ACCESSTOKEN
$oldVersionNumber = $(version-number)
$newVersionNumber = $oldVersionNumber + 1
az pipelines variable-group variable update --group-id [your_variable_group_id] --name version-number --organization [your_organization_url] --project [your_project_name] --value $newVersionNumber
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
The pipeline build service may also need permission to execute this command. To check, go to Pipelines -> Library -> Variable groups then edit the variable group containing your variable. Click on the Security button and make sure the user Project Collection Build Service has the Administrator role.
More information on the Azure CLI command can be found here. There is also another form of the command used to update variables that are not in a variable group, as described here.
Our Goal is to have Auto versioning for npm and NuGet packages in release definition of Azure Pipelines
As of now we are using the tokenization task based on the rev value we replacing the version numbers in both nuspec and package.json files.
So we getting the version numbers like 1.0.1, 1.0.2….. like this and auto versioning achieved in release definition
But the problem is whenever the release fails we are losing those numbers as our version numbers because rev value is increasing
Example: If my published version of artifacts is 1.0.1..
Next version to be published for me is 1.0.2
But if the 1.0.2 release is failed and if 1.0.3 is succeeded we are getting published number as 1.0.3 here for end user 1.0.2 is missing
Now we need the help to increment my version number based on last successful release or to reset the rev value based on the successful release or to get the published version of artifacts and increase that number
Or any other best practices to accomplish this task will be helpful.
Thank you in advance.
NPM with NuGet package auto versioning
I am afraid there is no such way to accomplish this task directly. Because $(Rev:.r) is the build number on this day. Use $(Rev:.r) to ensure that every completed build has a unique name. When a build is completed, if nothing else in the build number has changed, the Rev integer value is incremented by one. This value stores in database.
As test, I have created a Inline Powershell task to modify the value when build/release task fails.
I use the powershell script Write-Host "##vso[build.updatebuildnumber]$newVersionNumber" to update the build.updatebuildnumber (I use build.updatebuildnumber as nuget package version.) and set the option Run this task as Only when a previous task has failed:
To see which variables you can use for build and release pipelines, check these pages: - Build variables - Release variables.
Following is the powershell scripts to modify the build.updatebuildnumber:
$vstsCurrentVersionNumber = $Env:BUILD_BUILDNUMBER
$currentVersionNumber = $vstsCurrentVersionNumber.Split(".")
$revisionNumber = $currentVersionNumber[3]
$newRevisionNumber = [int]$revisionNumber -1
$newVersionNumber = $currentVersionNumber[0] + "." +
$currentVersionNumber[1] + "." + $currentVersionNumber[2] + "." +
$newRevisionNumber
$env:VersionNumber = $newVersionNumber
Write-Host "Update Build Number To: $newVersionNumber"
Write-Host "##vso[build.updatebuildnumber]$newVersionNumber"
Indeed, if the build/release task failed, the build number will be modified at this building.
However, when we execute the build/release next time, the buildnumber/$(Rev:.r) still increase based on the last failed build result:
As workaround, we could set a value for the nuget package version, like 1.0.0. And add a Inline Powershell task to increase the value of the version by one each time, when we successfully build/release. Do not execute Inline Powershell task when the build/release fails.
Hope this helps.