I have a multi-configuration job which runs on Linux, Windows and macOS, and the idea is that the user can pass parameters to select on which OSs should the job run. Right now I'm doing the following:
jobs:
- job: Build stuff
strategy:
matrix:
Linux:
enabled: ${{ parameters.ENABLE_LINUX }}
Windows:
enabled: ${{ parameters.ENABLE_WINDOWS }}
macOS:
enabled: ${{ parameters.ENABLE_MACOS }}
condition: eq(variables['enabled'], 'true')
However, I see that variables['enabled'] is always expanded to null for some reason. Is there any way to do this, other than testing for variables['enabled'] in every single step of the job?
Related
This is my first time working with azure pipelines, I started creating my azure-pipeline.yml. I am trying to execute the azure DevOps pipeline. However I am getting to errors where the variable are not referenced as declared.
deploy.sh deploy_azr ${{ variables.subPref }} ${{ variables.rgType }} ${{ variables.location }} ${{ variables.config }}
Here is the start of my template
# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml
trigger:
branches:
include:
- main
paths:
include:
- 'bicep/*'
- 'azure-pipelines.yml'
exclude:
- '*'
pool:
vmImage: ubuntu-latest
variables:
${{ if eq(variables['Build.SourceBranchName'], 'test_branch') }}:
deployTarget: tst
subscription: testsubscription
subscriptionId: 26455-trt31-******
rgType: tstrg
subPref: *****
config: tstjson
location: eastus2
${{ if eq(variables['Build.SourceBranchName'], 'main') }}:
deployTarget: prd
subscription: prdsub
subscriptionId: ***********************
rgType: prdrg
subPref: ******
config: prd.json
location: eastus2
stages:
- stage: Deploylib
jobs:
- deployment: lib
environment: ${{ variables.subscription }}
strategy:
runOnce:
deploy:
steps:
- checkout: self
- task: AzureCLI#2
inputs:
azureSubscription: ${{ variables.subscription }}
scriptType: 'bash'
scriptLocation: 'inlineScript'
inlineScript: |
set -e
set -x
sudo apt install -y gridsite-clients
cd 'bicep'
echo "starting the lib deployment"
deploy.sh deploy_azr ${{ variables.subPref }} ${{ variables.rgType }} ${{ variables.location }} ${{ variables.config }}
any help would be appreciated.
I think the problem is that you need to specify the correct environment name on this line,
jobs:
- deployment: lib
environment: <environment name>
You can create an environment on the DevOps page, see the reference here, then copy the name to the YAML above.
The concept of environment here represents a collection of resources you will deploy your code. Once you have run a deployment, you should be able to see the history of deployment in the target environment.
/azure-pipelines.yml (Line: 42, Col: 22): Unexpected value ''
Test the same YAML sample and reproduce the same issue.
The cause of the issue is that you are using the format: ${{ variables.subscription }} in YAML sample.
The variable will be processed at compile time.
To solve this issue, you can change to use the format: $(subscription)
For example:
jobs:
- deployment: lib
environment: $(subscription)
Result:
For more detailed info, you can refer to this doc: Runtime expression syntax
for example, I have something like this:
- job: Build
dependsOn: CheckTest
pool: ${{ parameters.setPool }}
services:
redis: redis
rabbitmq: rabbitmq
steps:
- checkout: self
My question is - is it possible to manage services list? For example, for some cases I need only redis container, for another only rabbitmq and sometimes I don't need any containers at all. Is it possible to implement dynamic services list?
This can be done using object parameters.
Pipeline:
parameters:
- name: myObject
type: object
default:
serviceToDeploy:
- redis
- rabbitmq
steps:
- script: echo "Test step before parameter validation"
- ${{ if ne(length(parameters.myObject.serviceToDeploy), 0) }}:
- ${{ each service in parameters.myObject.serviceToDeploy }}:
- script: echo ${{ service }}
displayName: "Task for installing ${{ service }}"
Run pipeline with 4 services:
Result:
Run pipeline with "No" services
Result:
PS: I do not have any container pools to test, but the conditions and loop should work exactly the same
I would consider choosing with a conditional approach. For example
variables:
${{ if eq(parameters.Environment, 'Production') }}:
serviceToDeploy: redis
${{ else }}:
serviceToDeploy: rabbitmq
And then you could use
services:
service1: $(serviceToDeploy)
I am trying to loop through user-defined variables in an Azure DevOps YAML pipeline.
The variables have been created through the UI:
Below the YAML pipeline code that I'm using:
trigger:
- dev
- main
pr:
- dev
pool:
vmImage: ubuntu-latest
stages:
- stage:
jobs:
- job: TestVars
steps:
- ${{ each var in variables }}:
- script: |
echo ${{ var.key }}
echo ${{ var.value }}
displayName: ${{ var.key }}
When running the above pipeline only system and build variables are listed (e.g. system, system.hostType, build.queuedBy, etc.).
Any help to loop through user-defined variables would be much appreciated.
Unfortunately, no luck fetching the variables defined in UI. However, if your variables are non-secrets, you can bring them over into the YAML, and they will show up in the loop.
- stage:
variables:
myyamlvar: 1000 # this will show up in the loop
jobs:
- job: TestVars
steps:
- ${{ each var in variables }}:
- script: |
echo ${{ var.key }}
echo ${{ var.value }}
displayName: ${{ var.key }}
Alternatively, instead of using a compile time expression, you can list variables using a runtime construct, for example:
- job: TestRuntimeVars
steps:
- script: |
for var in $(compgen -e); do
echo $var ${!var};
done
This will list all variables including ones defined in the UI.
From the Microsoft docs link you provided, it specifies that:
"Unlike a normal variable, they are not automatically decrypted into
environment variables for scripts. You need to explicitly map secret
variables."
However, one workaround could potentially be to run an azure cli task and get the pipeline variables using az pipelines variable list
Assuming your intention is to get the actual values, in which case maybe that won't suffice. Having said that, you should consider a variable group even if you're not using them in other pipelines since the group can be linked to an Azure KeyVault and map the secrets as variables. You can store your sensitive values in a KeyVault and link it to the variable group which can be used like regular variables in your pipeline.
Or you can access KeyVault secrets right from the AzureKeyVault pipeline task.
To expand on the awnser below. It is a bit round about but you can use the azure devopps CLI. This may be a bit overkill but it does do the job.
trigger:
- main
pool:
vmImage: ubuntu-latest
steps:
- bash: az --version
displayName: 'Show Azure CLI version'
- bash: az devops configure --defaults organization=$(System.TeamFoundationCollectionUri) project=$(System.TeamProject) --use-git-aliases true
displayName: 'Set default Azure DevOps organization and project'
- bash: |
az pipelines variable list --pipeline-id $(System.DefinitionId)
displayName: 'Show build list varibales'
env:
AZURE_DEVOPS_EXT_PAT: $(System.AccessToken)
This approach was taken from a combination of:
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#list-variables
and
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#list-variables
If the agent is self hosted you may need to install the dev opps cli.
I want one yaml if at all possible. However, the circumstances dictate that this single yaml will have auto triggers for certain branches (develop/release), but still can be manually built by any other branch qa wishes to test (i.e. they want to test a feature branch, they can build it using this yaml).
I need to dynamically set the buildconfig property.
I have:
trigger:
branches:
include:
- develop
- release/*
- Release/*
pool:
name: 'my-agent'
variables:
isRelease: $[contains(variables['Build.SourceBranch'], 'release')]
if {{$(isRelease)}}:
buildConfiguration: 'Release'
// if not release then set build config to 'Debug' but I'm struggling with the syntax. I can't find a not operator working well, or notIn...can someone help with this?
I would negate the if condition to get the "debug" condition. Also, there is no else, so you have to stack them:
${{ if variables.isRelease }}:
buildConfiguration: 'Release'
${{ if not(variables.isRelease) }}:
buildConfiguration: 'NotRelease'
Now we have if else expression and we can use it as follows:
${{ if variables.isRelease }}:
buildConfiguration: 'Release'
${{ else }}:
buildConfiguration: 'NotRelease'
I have a template that defines an array of branches for each environment, so that I can control which branches cause a deploy to a particular environment. For example, the only branch I want deployed to production is master, but for UAT I want release, hotfix, and master. I've set-up a parent template that calls downstream templates in an 'each' loop. Inside the deploy template, I want to compare the array of allowed branches against the current branch to determine whether to proceed or not. Here's part of the parent template that passes the branches arrays:
- template: pipeline-templates/environment-pipeline.yml
parameters:
envs:
- name: devtest
branches:
- 'refs/heads/master'
- 'refs/heads/develop'
- 'refs/heads/hotfix/'
- 'refs/heads/release/'
- name: nightly
branches:
- 'refs/heads/master'
- 'refs/heads/develop'
- 'refs/heads/hotfix/'
- 'refs/heads/release/'
- name: qa
branches:
- 'refs/heads/master'
- 'refs/heads/hotfix/'
- 'refs/heads/release/'
- name: prod
branches:
- 'refs/heads/master'
The environment-pipeline.yml below then calls the deploy template for each environment.
parameters:
- name: envs # collection of environments to create/deploy
type: object
default:
env:
name: ''
branches: []
stages:
- ${{ each env in parameters.envs }}:
- template: deploy.yml
parameters:
environmentName: ${{ env.name }}
onlyForBranches: ${{ env.branches }}
The above all works fine (note, I've deleted a lot of the YAML to simplify the example). The next stage is to use a condition with the array of branches in the deploy template. This is how I was hoping it would work, but doesn't.
parameters:
environmentName: ''
onlyForBranches: []
stages:
- stage: branch_gate
condition: and(succeeded(), in(variables['Build.SourceBranch'], ${{ parameters.onlyForBranches }}))
# then the deploy stages go here if above condition passes
This results in the following error:
Unable to convert from Array to String.
Is there some way to make this work, or should I use a different approach?
in is not the function to use. It takes n arguments and is true if the first argument equals any of the rest: in('B', 'A', 'B', 'C') => true
Try containsvalue instead:
condition: and(succeeded(), containsvalue(${{ parameters.onlyForBranches }}, variables['Build.SourceBranch']))
I agree with schummar on the use of containsValue, however the suggested approach did not work for me either.
Instead try this:
condition: and(succeeded(), ${{ containsValue(parameters.onlyForBranches, variables['Build.SourceBranch']) }})