I have three environments: dev, hml and qa.
In my pipeline depending on the branch the stage has a condition to check whether it will run or not:
- stage: Project_Deploy_DEV
condition: eq(variables['Build.SourceBranch'], 'refs/heads/dev')
dependsOn: Project_Build
- stage: Project_Deploy_HML
condition: eq(variables['Build.SourceBranch'], 'refs/heads/hml')
dependsOn: Project_Build
I'm doing the qa stage, and I'd like to put a condition, depending on the branch, the dependson parameter will change:
- stage: Project_QA
condition:
${{ if eq(variables['Build.SourceBranchName'], 'dev') }}:
dependsOn: 'Project_Deploy_DEV'
${{ if eq(variables['Build.SourceBranchName'], 'hml') }}:
dependsOn: 'Project_Deploy_HML'
However, the condition above is not working, would anyone know the best way to perform this condition?
Thanks
From your YAML sample, it seems that there is a format issue. When you use if expression, there is no need to add condition field in YAML.
You can try the following sample and check if it can work.
stages:
- stage: A
jobs:
- job: test
steps:
- xxx
- stage: B
jobs:
- job: test
steps:
- xxx
- stage: C
${{ if eq(variables['Build.SourceBranchName'], 'main') }}:
dependsOn: A
${{ if eq(variables['Build.SourceBranchName'], 'dev') }}:
dependsOn: B
jobs:
- job: test
steps:
- xxx
Related
Currently, our inhertited pipelines (not the best designed, but it's what I've got for now), look something like this:
build -> provision + deploy (pipeline per env) -> acceptance test
I only want to be able to do one deployment at a time, so I'm looking at Exclusive Locks.
Unfortunately, these seem to work at the step level, not the pipeline level. So given my provision + deploy pipeline contains multiple steps, how can I prevent step 1 from pipeline run 1 running then step 1 from pipeline 2 running etc?
I can't see a way to apply the exclusive lock at the pipeline level.
Our QA provision + deploy pipeline yml looks like this (before adding the locks):
pool:
vmImage: "ubuntu-latest"
resources:
pipelines:
- pipeline: apiBuild
source: "API/Build"
trigger:
enabled: true
branches:
include:
- main
trigger: none
pr: none
variables:
- template: _pipeline/variables/allVariables.yml
parameters:
deployEnvironment: Qa
stages:
- stage: provisionInfrastructureTemplates
displayName: Provision Templates infrastructure
dependsOn: []
jobs:
- template: _pipeline/stages/jobs/provisionTemplates.yml
- stage: templatesAcceptanceTests
displayName: Templates acceptance tests
dependsOn: provisionInfrastructureTemplates
jobs:
- template: _pipeline/stages/jobs/acceptanceTestsTemplates.yml
- stage: provisionInfrastructureClients
displayName: Provision Clients infrastructure
dependsOn: []
jobs:
- template: _pipeline/stages/jobs/provisionClients.yml
- stage: clientsAcceptanceTests
displayName: Clients acceptance tests
dependsOn: provisionInfrastructureClients
jobs:
- template: _pipeline/stages/jobs/acceptanceTestsClients.yml
- stage: provisionInfrastructureReports
displayName: Provision Reports infrastructure
dependsOn: []
jobs:
- template: _pipeline/stages/jobs/provisionReports.yml
- stage: reportsAcceptanceTests
displayName: Reports acceptance tests
dependsOn: provisionInfrastructureReports
jobs:
- template: _pipeline/stages/jobs/acceptanceTestsReports.yml
- stage: upgradePreviewImage
displayName: Upgrade preview image
dependsOn: []
jobs:
- template: _pipeline/stages/jobs/upgradePreviewImage.yml
- stage: provisionInfrastructureDocuments
displayName: Provision Documents infrastructure
dependsOn: upgradePreviewImage
jobs:
- template: _pipeline/stages/jobs/provisionDocuments.yml
- stage: documentsAcceptanceTests
displayName: Documents acceptance tests
dependsOn: provisionInfrastructureDocuments
jobs:
- template: _pipeline/stages/jobs/acceptanceTestsDocuments.yml
- stage: provisionInfrastructureNotifications
displayName: Provision Notifications infrastructure
dependsOn: []
jobs:
- template: _pipeline/stages/jobs/provisionNotifications.yml
- stage: provisionEventGridSubscriptions
displayName: Provision Event Grid Subscriptions
dependsOn: [clientsAcceptanceTests, templatesAcceptanceTests, reportsAcceptanceTests, documentsAcceptanceTests, provisionInfrastructureNotifications]
jobs:
- template: _pipeline/stages/jobs/provisionEventGridSubscriptions.yml
- stage: workflowTests
displayName: Workflow tests
dependsOn: provisionEventGridSubscriptions
jobs:
- template: _pipeline/stages/jobs/workflowTests.yml
As an aside, I know our services ought to be independently deployable, they're not currently, that's part of the tech debt we're dealing with so, as things stand, they need deploying together.
When you put the jobs in separate stages. The exclusive lock of the environment will be judged separately.
It is not able to apply the exclusive lock at the pipeline level.
To meet your requirement, you can define required the jobs in the same stage. The Exclusive Lock will be judged before running.
When set the environments exclusive lock in the same stage, all environments referenced in a stage will be locked. Only when the current Pipeline Run Stage finished, the next Pipeline run will continue to run.
Here is an example:
stages:
- stage: Test
displayName: Test
lockBehavior: sequential
jobs:
- deployment: Test1
displayName: 'Test'
environment: Test2
strategy:
runOnce:
deploy:
steps:
- script: echo "Test"
- deployment: Test
displayName: 'Test2'
environment: Test2
strategy:
runOnce:
deploy:
steps:
- script: echo "Test"
I have the below Azure YAML pipeline file whose stages get executed based on a tag some-tag-v* reference pushed to the Azure repo.
trigger:
branches:
include:
- refs/tags/some-tag-v*
exclude:
- refs/tags/some-branch
pool:
name: some-name
demands:
- Agent.Name -equals some-name
stages:
- stage: A
jobs:
- job: A
steps:
- checkout: self
- bash: |
some-bash-commands
- stage: B
jobs:
- job: B
steps:
- bash: |
More-bash-commands
Problem: I want stage B to be executed only when some-tag-2-v* is pushed instead of some-tag-v*. I am looking at the condition-based triggering but I am not sure if that's the right way to do it.
I think you can make use of YAML conditions.
stages:
- stage: A
condition: ...
jobs:
- job: A
steps:
- checkout: self
- bash: |
some-bash-commands
For the condition, you can probably leverage pipeline and build variables, e.g.:
variables:
isSomeTag: $[eq(variables['Build.SourceBranch'], 'refs/tags/some-tag-v')]
I suggest that you could directly use the IF Expression in Stages. This will be more convenient.
Here is an example:
trigger:
branches:
include:
- - refs/tags/some-tag-v*
exclude:
- refs/tags/some-branch
stages:
- ${{ if eq(variables['Build.SourceBranch'], 'refs/tags/some-tag-v') }}:
- stage: A
jobs:
- job: A
steps:
- checkout: self
- bash: echo 1
- ${{ if eq(variables['Build.SourceBranch'], 'some-tag-2-v') }}:
- stage: B
jobs:
- job: B
steps:
- bash: echo 2
When the pipeline is triggered by some-tag-2-v, it will only run the stage B.
The structure of my PR pipeline looks like this:
trigger: none
resources:
- repo: self
variables:
- template: templates/variables.yaml
pool:
vmIMage: $(vmImageName)
stages:
- template: templates/changed.yaml
parameters:
comparedTo: origin/production
- template: templates/unitTests.yaml
- template: templates/build.yaml
parameters:
services:
- api
- admin
- admin-v2
- client
tag: test
tagVersion: latest
- template: templates/integrationTests.yaml
- template: templates/seleniumTests.yaml
The change.yaml basically determines what microservices have been changed so that it can avoid building, testing, and deploying services if there have been no changes to them.
parameters:
- name: comparedTo
default: ''
stages:
- stage: Changed
displayName: Checks for changes in services and configs...
jobs:
- job: Changes
displayName: Checking for changes in services and configs...
steps:
- bash: |
mapfile -t servicesChanged < <(git diff HEAD ${{ parameters.comparedTo }} --name-only | awk -F'/' 'NF!=1{print $1}' | sort -u)
echo "Total Changed: ${#servicesChanged[#]}"
if [[ ${#servicesChanged[#]} > 0 ]]; then
echo "Any services changed: True"
echo "##vso[task.setvariable variable=anyServicesChanged;isOutput=true]true"
fi
name: detectChanges
There are then job conditions in subsequent stages that determine if the job should run, or not, if anyServicesChanged = true.
What I am currently testing with, this true always.
It gets to the unitTests.yaml and runs normally (it passes):
stages:
- stage: UnitTests
displayName: Run unit tests on service...
condition: succeeded()
dependsOn: Changed
jobs:
- job: UnitTests
condition: or(eq(stageDependencies.Changed.Changes.outputs['detectChanges.anyServicesChanged'], true), eq(variables['Build.Reason'], 'Manual'))
displayName: Running unit tests...
steps:
- bash: |
echo "Running unit tests..."
Then it gets to build.yaml and it skips it:
parameters:
- name: services
type: object
default: []
- name: tag
default: ''
- name: tagVersion
default: ''
stages:
- stage: BuildAndPush
displayName: Build and Push Docker images of services...
condition: succeeded()
dependsOn: UnitTests
jobs:
- job: BuildAndPush
condition: or(eq(stageDependencies.Changed.Changes.outputs['detectChanges.anyServicesChanged'], true), eq(variables['Build.Reason'], 'Manual'))
displayName: Building and Push Docker images of services...
steps:
- ${{ each service in parameters.services }}:
- task: Docker#2
displayName: Build and Push ${{ service }} Docker image
inputs:
command: buildAndPush
repository: $(imageRepository)-${{ service }}
dockerfile: $(dockerfilePath)/${{ service }}/Dockerfile
containerRegistry: $(dockerRegistryServiceConnection)
tags: |
${{ parameters.tag }}-${{ parameters.tagVersion }}
The job condition: is exactly the same, but this time it says it is NULL:
6 queue time variables used
Expanded: or(eq(Null, True), eq('PullRequest', 'Manual'))
Result: False
I'm not sure why this is as I thought it would persist where every I need to use that variable.
Suggestions for why this is happening and how to fix it?
In build.yaml, you need to add depend on Changed stage. Otherwise, the variable set in the Changed stage cannot be obtained in the BuildAndPush stage.
stages:
- stage: BuildAndPush
displayName: Build and Push Docker images of services...
condition: succeeded()
dependsOn:
- UnitTests
- Changed
I have pipeline where running template as one of steps.
I need execute template with two different scopes of parameters depends on value of one of pipeline variable.
For example, if pipeline variable 'mode' has value 'dev', I need run template with one scope, if value is 'stage', another scope.
- ${{ if eq(parameters.mode, 'dev') }}:
- template: ../template.yaml
parameters:
mode: dev
namespace: dev-namespace
fqdn: dev-fqdn
- ${{ if eq(variables.mode, 'staging') }}:
- template: ../template.yaml
parameters:
mode: staging
namespace: staging-namespace
fqdn: staging-fqdn
You should use this syntax ${{ if eq(variables['mode'], 'staging') }}:
It depends at what time the variables will be read in at and used. There are more then possible answer.
If using these for the future execution of a template I'd recommend using condition and defining the parameters you have in separate yml variable files.
condition: eq('${{ variables.mode}}', 'staging)
That way the call would be just
job:
condition: eq('${{ variables.mode}}', 'staging')
variables: staging.yml
- template: ../template.yaml
parameters:
mode: ${{ variables.mode }}
namespace: ${{ variables.namepsace }}
fqdn: ${{ variables.fqdn }}
And staging.yml would look like:
variables:
mode: staging
namespace: staging-namespace
fqdn: staging-fqdn
This way it is a little less decoupled and scalable not having to hard code it every time.
1.If you're using runtime parameters, you can use this format:
trigger:
- master
parameters:
- name: mode
displayName: Choose mode
type: string
default: dev
values:
- dev
- staging
pool:
vmImage: 'windows-latest'
steps:
- ${{ if eq(parameters.mode, 'dev') }}:
- template: template.yml
parameters:
mode: dev
- ${{ if eq(parameters.mode, 'staging') }}:
- template: template.yml
parameters:
mode: staging
For more details you can check Parameters to select a template at runtime.
2.If you're using variables defined in yaml directly, check my another sample:
trigger:
- master
variables:
mode: staging
pool:
vmImage: 'windows-latest'
stages:
- stage: Dev
condition: eq(variables.mode, 'dev')
jobs:
- job:
steps:
- template: template.yml
parameters:
mode: dev
- stage: Staging
condition: eq(variables.mode, 'staging')
jobs:
- job:
steps:
- template: template.yml
parameters:
mode: staging
I would recommend runtime parameters way cause it's more convenient and much more flexible.
Update1:
Here's my simple template.yml file which is just for test:
parameters:
- name: mode
type: string
default: default
steps:
- script: echo ${{ parameters.mode }}
I'm trying to set the variable group according to one varible that exists in the pipeline. The yaml looks like this:
But i'm getting the following error when i'm running the pipeline:
If i remove the "- group : QA" or "- group : PROD" the pipeline runs without any problem. What am I doing wrong?
This is slightly different solution but you may achieve your goal - which is if I understood conditional selection of variable group.
You can use runtime parameters:
parameters:
- name: environment
displayName: Environment
type: string
default: QA
values:
- QA
- PROD
stages:
- stage:
displayName: 'Build and Restore'
variables:
- group: ${{ parameters.environment }}
jobs:
- job:
steps:
- script: echo $(name)
than running a build you can select your envrionment:
Note: I have defined two variable groups QA and PROD with variable name in both groups.
Try with below schema:
variables:
isProd: true
stages:
- stage: Test
displayName: Build and restore
variables:
- ${{ if eq(variables['isProd'], 'false') }}:
- group: QA
- ${{ if eq(variables['isProd'], 'true') }}:
- group: PROD
jobs:
- job: A
steps:
- bash: echo $(groupname)
Note: You may receive some red warning when you defining above YAML scipt. Ignore that confused warning and continue to run it.