Currently i am in the process of converting my pipelines from classic over to azurepipelines.yml and im having an issue trying to find the correct syntax to reference release variables in a bash step.
The existing code in a bash task
namebuilder=$(RELEASE.ENVIRONMENTNAME)-$(RELEASE.RELEASEID)
will output the following
dev-2049
however when converted over to my new pipeline file the above code produces the the following error
/home/vsts/work/_temp/ac39e1d7-11bd-4c32-9b1b-1520dae11c5a.sh: line 1: RELEASE.ENVIRONMENTNAME: command not found
/home/vsts/work/_temp/ac39e1d7-11bd-4c32-9b1b-1520dae11c5a.sh: line 1: RELEASE.RELEASEID: command not found
[extracted from pipeline.yml]
- bash: |
namebuilder=$(RELEASE.ENVIRONMENTNAME)-$(RELEASE.RELEASEID)
i have even created a step trying a few different approaches without much luck
steps:
- bash: |
echo This multiline script always runs in Bash.
echo Even on Windows machines!
echo '$(release.environmentname)'
echo $(release.environmentname)
echo '$(RELEASE.ENVIRONMENTNAME)'
echo $(RELEASE.ENVIRONMENTNAME)
produces
This multiline script always runs in Bash.
Even on Windows machines!
$(release.environmentname)
$(RELEASE.ENVIRONMENTNAME)
/home/vsts/work/_temp/260dd504-a42d-45d6-bb1b-bf1f4b015cf8.sh: line 4: release.environmentname: command not found
/home/vsts/work/_temp/260dd504-a42d-45d6-bb1b-bf1f4b015cf8.sh: line 6: RELEASE.ENVIRONMENTNAME: command not found
is it also possible (in a much cleaner approach) to define this as a pipeline variable and reference at a global scope like below ?
variables:
namebuilder: '$(release.environmentname)-$(release.releaseid)'
stages:
- stage: Deploy
displayName: deploy infra
jobs:
- job: deploy_infra
displayName: deploy infra
continueOnError: true
workspace:
clean: outputs
steps:
- bash: |
echo This multiline script always runs in Bash.
echo Even on Windows machines!
echo '$(namebuilder)'
tia
It doesn't look like release.environment or any release variables are available for multi-stage pipelines. You could use the new environment concept and at that point environment.name would be available. I think you would likely go with $(environment.name)-$(build.buildid) for what you are after.
So I am not sure if the release pipelines you are converting are deploying to say an app service, or to a VM, or just using a hosted agent to publish something else? Disclaimer I have not used the Environment concept extensively yet just some reading and limited testing. Its all new!
So for deploying to VMs You can configure a Virtual Machine resource in an environment. This concept has a bunch of parallels with classic deployment group agents. You register an agent on a target machine. From there your pipeline steps can execute in that machine's context and you get a further set of environment variables.
The example pipeline below outputs any environment variables from the context the steps are running in and also outputs $(environment.name)-$(build.buildid)
A normal job in a hosted pipeline
A Deployment to an Environment
A Deployment to an Environment with a VM resource
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
variables:
namebuilder: '$(environment.name)-$(build.buildid)'
jobs:
- job: NormalJobInHostedPipeline
steps:
- task: PowerShell#2
name: EnvironmentVariables
inputs:
targetType: 'inline'
script: 'gci env:* | sort-object name'
- bash: |
echo This multiline script always runs in Bash.
echo Even on Windows machines!
echo '$(namebuilder)'
# track deployments on the environment
- deployment: DeploymentHostedContext
displayName: Runs in Hosted Pool
pool:
vmImage: 'Ubuntu-16.04'
# auto creates an environment if it doesn't exist
environment: 'Dev'
strategy:
runOnce:
deploy:
steps:
- task: PowerShell#2
name: EnvironmentVariables
inputs:
targetType: 'inline'
script: 'gci env:* | sort-object name'
- bash: |
echo This multiline script always runs in Bash.
echo Even on Windows machines!
echo '$(namebuilder)'
# Similar to Deployment Group Agent need to register them -stage will fail if resource does not exist
# https://learn.microsoft.com/en-us/azure/devops/pipelines/process/environments-virtual-machines?view=azure-devops
- deployment: DeploymentVirtualMachineContext
displayName: Run On Virtual Machine Agent
environment:
name: DevVM
resourceType: VirtualMachine
strategy:
runOnce:
deploy:
steps:
- task: PowerShell#2
name: EnvironmentVariables
inputs:
targetType: 'inline'
script: 'gci env:* | sort-object name'
- task: PowerShell#2
name: VariableName
inputs:
targetType: 'inline'
script: 'echo $(namebuilder)'
Use $(System.StageName) in place of $(Release.EnvironmentName), as for release Id, you'd need to use $(Build.BuildId)
I found that $(Environment.Name) doesn't work unless you're using environments. I'm not since it's still quite limited.
Related
Is there a way within a Release pipeline in Azure to pass variables created in one stage to the next stage?
I see lots of documentation about using echo "##vso[task..... - This however does not seem to work within the release pipeline.
I am mainly using bash scripts and I can reuse it within the same stage in different tasks, but not subsequent stages.
This seems like an essential feature to me, passing variables through stages...
Is there a way to do this?
If you want to pass variables from one stage to another stage in yml pipelines for release, you are supposed to use echo "##vso[task....." follow the doc
For a simple example:
stages:
- stage: BuildStage
jobs:
- job: BuildJob
steps:
- bash: echo "##vso[task.setvariable variable=TestArtifactName;isoutput=true]testValue"
name: printvar
- stage: DeployWebsiteStage
lockBehavior: sequential
dependsOn: BuildStage
condition: succeeded()
variables:
BuildStageArtifactFolderName: $[stageDependencies.BuildStage.BuildJob.outputs['printvar.TestArtifactName'] ]
jobs:
- deployment: DeployWebsite
environment:
name: webapplicationdeploy
strategy:
runOnce:
deploy:
steps:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: Write-Host "BuildStageArtifactFolderName:" $(BuildStageArtifactFolderName)
You are supposed to get the value set from stage 'BuildStage'.
If you want to pass variables from one stage to another stage in classic release pipelines,
1.Set Release Permission Manage releases for the Project Collection Build Service as Allow.
2.Toggle on 'Allow scripts to access the OAuth token' for the first stage
3.Set a variable like 'StageVar' in release scope.
4.ADD the first powershell task(inline) in the first stage for creating a variable 'myVar' in the first stage.
5.update the Release Definition and Release Variable (StageVar)
6.Add a powershell task in the second stage to retrieve the value of myVar via the Release Variable StageVar.
You could refer the blog for more details.
It works on my side,
results in the first stage:
results in the second stage:
You can make use of the Azure DevOps Variable group to store your variables and call them in your release pipelines across multiple stages and multiple pipelines within a project. You can make use of Azure CLI to make use of the Variable group.
I have stored 2 variables of database name and password in the SharedVariables group. I can call this variable group in 2 ways: 1) Via the YAML pipeline and 2) Via Classic/Release pipelines.
Yaml:-
trigger:
- main
pool:
vmImage: ubuntu-latest
variables:
- group: SharedVariables
steps:
- script: |
echo $(databaseserverpassword)
When I ran the pipeline the database server password was encrypted like the below:-
In Release pipeline:-
You can add this variable within multiple pipelines and multiple stages in your release like below:-
But the above method will help you store static values in the variable group and not the output variable of build to release Unless you specifically assign those variables manually in the variable group. You can make use of this extension > Variable Tools for Azure DevOps Services - Visual Studio Marketplace
With this extension, you can store your variables from the build in a JSON file and load that JSON file in the next release stage by calling the task from the extension.
Save the build variable in a file:-
Load the build variable in a release:-
Another method is to store the variables in a file as a build artifact and then call the build artifact in the release pipeline with the below yaml code:-
trigger:
- dev
pool:
vmImage: windows-latest
parameters:
- name: powerenvironment
displayName: Where to deploy?
type: string
steps:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
$variable = '${{parameters.powerenvironment}}'
$variable | Out-File $(Build.ArtifactStagingDirectory)\filewithvariable.txt
Get-Content $(Build.ArtifactStagingDirectory)\filewithvariable.txt
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'drop'
publishLocation: 'Container'
And download the artifact and run the tasks in your release pipeline.
Reference:-
Pass parameters from build to release pipelines on Azure DevOps - GeralexGR
Another simple method is to run the PowerShell script to store the build output as JSON in the published artifact and read the content in the release pipeline like below:-
ConvertTo-Json | Out-File "file.json"
Get-Content "file.json" | themnvertFrom-Json
You can also reference the dependencies from various stages and call them in another stage within a pipeline with below yaml code :-
stages:
- stage: A
jobs:
- job: A1
steps:
- bash: echo "##vso[task.setvariable variable=shouldrun;isOutput=true]true"
# or on Windows:
# - script: echo ##vso[task.setvariable variable=shouldrun;isOutput=true]true
name: printvar
- stage: B
condition: and(succeeded(), eq(dependencies.A.outputs['A1.printvar.should run], 'true'))
dependsOn: A
jobs:
- job: B1
steps:
- script: echo hello from Stage B
Reference:-
bash - How to pass a variable from build to release in azure build to release pipeline - Stack Overflow By PatrickLu-MSFT
azure devops - How to get the variable value in TFS/AzureDevOps from Build to Release Pipeline? - Stack Overflow By jessehouwing
VSTS : Can I access the Build variables from Release definition? By Calidus
Expressions - Azure Pipelines | Microsoft Learn
I'm using the below azure-pipeline.yml file to build docker image, push to Azure docker registry & restart the Azure docker app servicer.
This yaml file uses variable set in azure pipeline, screenshot attached.
My issue is, I need to create 2-3 pipelines every-week for different projects I need to add every variable manually for each project and copy paste from my config. Is there a way I can import a .env file or add multiple variables all at once while creating the pipeline.
Objectively I need to cut down the single variable copy paste time & avoid errors that might occurr
1.You could use variable group to reuse variables.
trigger:
- none
pool:
vmImage: ubuntu-latest
variables:
- group: forTest
steps:
- script: |
echo $(test1)
echo $(test2)
displayName: 'Run a multi-line script'
2.You could use variable template.
trigger:
- none
pool:
vmImage: ubuntu-latest
variables:
- template: vars.yml
steps:
- script: |
echo $(test1)
echo $(test2)
displayName: 'Run a multi-line script'
I want to tag my pipeline build during the build process itself. For which based on the official document I tried echo ##vso[build.addbuildtag]testing in the pipeline yaml. There was no error but the build was not tagged either.
I'm able to add tags from portal successfully.
My pipleine yaml below.
pool:
vmImage: ubuntu-latest
jobs:
- job: addingtag
displayName: addingtag
steps:
- script: |
echo ##vso[build.addbuildtag]testing
displayName: addingtag
Below are other combinations I tried and still failed.
echo ##vso[build.addbuildtag] testing
echo ##vso[build.addbuildtag]Tag_testing
You may need to add double quotes, I could successfully add tag by using YAML script like below.
- script: |
echo "##vso[build.addbuildtag]testing"
displayName: addingtag
I am working on a multi stage pipeline that build and deploy some c# code from staging to production
Everything works just fine but I wanted to try and customise a bit more the pipeline so I can see the actual version that is being built and deployed as part of the name of the stage.
At the current stage, this my multi stage pipeline
trigger:
batch: true
tags:
include:
- '*'
branches:
exclude:
- main
- staging
pool:
vmImage: 'ubuntu-latest'
variables:
buildNumber: "$[variables['Build.BuildNumber']]"
DOCKER_BUILDKIT: 1
dockerRegistryServiceConnectionStaging: '<My-Connection-String>'
imageRepositoryStaging: '<My-Repo-Name>'
containerRegistryStaging: '<My-Container-Name>'
dockerRegistryServiceConnectionProd: '<My-Connection-String>'
imageRepositoryProd: 'My-Repo-Name>'
containerRegistryProd: '<My-Container-Name>'
dockerfilePath: 'pathTo/Dockerfile'
solution: 'path/To/Solution.csproj'
tag: '$(Build.BuildNumber)'
stages:
- stage: 'Build_Staging'
displayName: 'Build_Staging'
jobs:
- job: buildStaging
displayName: 'DotNet Core publish and dockerize'
steps:
- powershell: |
# Write your PowerShell commands here.
Write-Host "Update Build.BuildNumber"
cd $(System.DefaultWorkingDirectory)
$Latesttag = $(git describe --tags $(git rev-list --tags --max-count=1))
Write-Host "The latest git tag is $Latesttag "
Write-Host
"##vso[build.updatebuildNumber]$Latesttag"
- task: DotNetCoreCLI#2
displayName: 'DotNet - Restore'
inputs:
command: 'restore'
projects: $(solution)
noCache: true
versioningScheme: 'off'
vstsFeed: '<Feed>'
- task: DotNetCoreCLI#2
name: 'DotnetPublish'
displayName: 'dotnet - Publish'
inputs:
command: 'publish'
projects: $(solution)
arguments: '-o publish/solution -c release'
modifyOutputPath: false
zipAfterPublish: false
publishWebProjects: false
- task: Docker#2
name: 'dockerBuildAndPush'
displayName: 'docker - Build & Push $(tag)'
inputs:
repository: $(imageRepositoryStaging)
Dockerfile: $(dockerfilePath)
containerRegistry: ${{ variables.dockerRegistryServiceConnectionStaging }}
buildContext: ${{ variables.buildContext }}
tags: |
$(Build.BuildNumber)
latest
- stage: 'Deploy_Staging'
jobs:
- deployment: 'Deploy'
environment: 'Staging'
variables:
EnvironmentName: 'Staging'
strategy:
runOnce:
deploy:
steps:
- task: AzureRmWebAppDeployment#4
displayName: 'Deploy Azure App Service To Staging'
inputs:
azureSubscription: '<Azure-Subscription>'
appType: 'webAppContainer'
DockerNamespace: '<container-namespace>'
DockerRepository: '<Repository>'
DockerImageTag: '$(Build.BuildNumber)'
WebAppName: '<WebAppName>'
The Powershell command is to override the Build.BuildNumber with the tag I am pushing to GitHub.
When I run this pipeline, in azure DevOps, I see the stage name Build_Staging_$(Build.BuildNumber) as a string.
What I would really like to see is, if I push the tag 'v1.0.0` for example, is to see the stage name like:
Build_Staging_v1.0.0
I tried to use the displayName and the output is not the one I was looking for and if I try with name instead of displayName I get the error unexpected value name
Can please please anyone help understand what am I doing wrong and how I can achieve this?
Please if my question is not 100% clear and missing any important detail, just let me know
UPDATE:
I did update the post with my entire pipeline.
This pipeline, before it used to be a single job process, and everything was working fine. But to get my hand dirty, I wanted to add stages to split and workflow based on resources and environment.
The process is still working and this is what I am expecting.
In my GitHub, when I create a tag on the main branch, this will trigger my build stage. Which thanks to the Powershell script to update the BuildNumber with the tag, I am able to build the docker image in my container registry in the following format:
docker-image-name:v1.0.1
That version can be seen at this level also:
This updated buildNumber (now is Tag) is use in Azure pipelines App Slack to check the version that has been pushed.
So far everything is good.
But I am facing the problem with the deployment job, at that level I am not able to set any Powershell script to update that same BuildNumber with the tag. I checked the documentation and nothing is mentioned about how I can add another job or step. I tried implementing this but I get errors that the value is unexpected.
Let me just share another screenshot to fully explain the issue.
Assuming I am deploying the docker image v1.0.1, everything works perfectly, but the build number in deployment stage, is not being updated, in fact in slack channel, I see the normal build number, as follow:
Instead of having the buildNumber, I would like to have my tag.
Please any help here?
Unfortunately, you won't be able to set a stage name to a dynamic variable that is set within one of its child's steps. You'll have to set it to a pipeline-level variable or predefined variable
Variable evaluation goes top-down from stages to tasks:
stages
jobs
tasks
To help explain exactly why, let's talk about how variable evaluation works in general with regard to this structure:
VARIABLE EVALUATION: Using stages as an example, you can set a stage name using any dynamic value that's present when the stage is evaluated. This means the variable is only accessed when the stage is initially "rendered". Azure DevOps requires that the variable be present before evaluation and will not retroactively update the UI if that variable is changed within a child step.
Let's talk about each and their respective limitations on what variables you can use in their names:
STAGES: pipeline-level variables, parameters (in the case of templates), or predefined variables
JOBS: stage-level variables, pipeline-level variables, parameters (in the case of templates), or predefined variables
TASKS: job-level variables, stage-level variables, pipeline-level variables, parameters (in the case of templates), or predefined variables
I did something similar by setting my build number to a repo tag. Here is the PowerShell function that sets the Build.Buildnumber variable to the tag value. You can just call it straight out or base it off a parameter if you have other version number logic.
function getTagVersion() {
$tag = iex "git describe --long --tags --always"
$a = [regex]"\d+\.\d+\.\d+\.\d+"
$b = $a.Match($tag)
$b = $b.Captures[0].value
$b = $b -replace '-', '.'
$b = $b -replace 'v', ''
Write-Host "Version found: $b"
$newBuildNumber = '$(Build.BuildNumber)' -replace $a,$b
Write-Host "##vso[build.updatebuildnumber]$newBuildNumber"
return $b
}
I can't claim credit for this code as I found it on someone's blog. But it works and I use for my release builds. You just have to call the function and it will reset the build.buildnumber to the latest tag in your repo. Its important to note, that the tag should be in normal version number format.
Example:
Tag Name: 10.1.100.0
Trying to deploy an app to kubernetes using azure. I have a build pipeline yml file and in the pipeline i've set a variable called "discordToken". I tried setting it two different ways, one is called discordToken and the other is MY_MAPPED_ENV_VAR.
in my node project i'm doing
console.log( process.env.discordToken )
console.log( process.env.MY_MAPPED_ENV_VAR )
but everything keeps coming back as undefined.
stages:
- stage: Build
displayName: Build stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
environment:
discordToken: $(discordToken)
steps:
- powershell: |
env:
MY_MAPPED_ENV_VAR: $(discordToken)
- task: Docker#2
displayName: Build and push an image to container registry
inputs:
command: buildAndPush
repository: $(imageRepository)
dockerfile: $(dockerfilePath)
containerRegistry: $(dockerRegistryServiceConnection)
tags: |
$(tag)
How should i be setting the environment variables?
As far as I know, if you directly set the environment variable (env:xxx) in Powershell Task, the variable can only be used by the current task.
You could try to set the variables with script, then the variables could be used by following task.
For example:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
# Write your PowerShell commands here.
echo "##vso[task.setvariable variable=MY_MAPPED_ENV_VAR]$(discordToken)"
You could set the reference variable in Settings -> Variables:
Here is a ticket about set environment variables in dockerfile, it may help you.