for example, I have something like this:
- job: Build
dependsOn: CheckTest
pool: ${{ parameters.setPool }}
services:
redis: redis
rabbitmq: rabbitmq
steps:
- checkout: self
My question is - is it possible to manage services list? For example, for some cases I need only redis container, for another only rabbitmq and sometimes I don't need any containers at all. Is it possible to implement dynamic services list?
This can be done using object parameters.
Pipeline:
parameters:
- name: myObject
type: object
default:
serviceToDeploy:
- redis
- rabbitmq
steps:
- script: echo "Test step before parameter validation"
- ${{ if ne(length(parameters.myObject.serviceToDeploy), 0) }}:
- ${{ each service in parameters.myObject.serviceToDeploy }}:
- script: echo ${{ service }}
displayName: "Task for installing ${{ service }}"
Run pipeline with 4 services:
Result:
Run pipeline with "No" services
Result:
PS: I do not have any container pools to test, but the conditions and loop should work exactly the same
I would consider choosing with a conditional approach. For example
variables:
${{ if eq(parameters.Environment, 'Production') }}:
serviceToDeploy: redis
${{ else }}:
serviceToDeploy: rabbitmq
And then you could use
services:
service1: $(serviceToDeploy)
Related
This is my first time working with azure pipelines, I started creating my azure-pipeline.yml. I am trying to execute the azure DevOps pipeline. However I am getting to errors where the variable are not referenced as declared.
deploy.sh deploy_azr ${{ variables.subPref }} ${{ variables.rgType }} ${{ variables.location }} ${{ variables.config }}
Here is the start of my template
# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml
trigger:
branches:
include:
- main
paths:
include:
- 'bicep/*'
- 'azure-pipelines.yml'
exclude:
- '*'
pool:
vmImage: ubuntu-latest
variables:
${{ if eq(variables['Build.SourceBranchName'], 'test_branch') }}:
deployTarget: tst
subscription: testsubscription
subscriptionId: 26455-trt31-******
rgType: tstrg
subPref: *****
config: tstjson
location: eastus2
${{ if eq(variables['Build.SourceBranchName'], 'main') }}:
deployTarget: prd
subscription: prdsub
subscriptionId: ***********************
rgType: prdrg
subPref: ******
config: prd.json
location: eastus2
stages:
- stage: Deploylib
jobs:
- deployment: lib
environment: ${{ variables.subscription }}
strategy:
runOnce:
deploy:
steps:
- checkout: self
- task: AzureCLI#2
inputs:
azureSubscription: ${{ variables.subscription }}
scriptType: 'bash'
scriptLocation: 'inlineScript'
inlineScript: |
set -e
set -x
sudo apt install -y gridsite-clients
cd 'bicep'
echo "starting the lib deployment"
deploy.sh deploy_azr ${{ variables.subPref }} ${{ variables.rgType }} ${{ variables.location }} ${{ variables.config }}
any help would be appreciated.
I think the problem is that you need to specify the correct environment name on this line,
jobs:
- deployment: lib
environment: <environment name>
You can create an environment on the DevOps page, see the reference here, then copy the name to the YAML above.
The concept of environment here represents a collection of resources you will deploy your code. Once you have run a deployment, you should be able to see the history of deployment in the target environment.
/azure-pipelines.yml (Line: 42, Col: 22): Unexpected value ''
Test the same YAML sample and reproduce the same issue.
The cause of the issue is that you are using the format: ${{ variables.subscription }} in YAML sample.
The variable will be processed at compile time.
To solve this issue, you can change to use the format: $(subscription)
For example:
jobs:
- deployment: lib
environment: $(subscription)
Result:
For more detailed info, you can refer to this doc: Runtime expression syntax
I am trying to loop through user-defined variables in an Azure DevOps YAML pipeline.
The variables have been created through the UI:
Below the YAML pipeline code that I'm using:
trigger:
- dev
- main
pr:
- dev
pool:
vmImage: ubuntu-latest
stages:
- stage:
jobs:
- job: TestVars
steps:
- ${{ each var in variables }}:
- script: |
echo ${{ var.key }}
echo ${{ var.value }}
displayName: ${{ var.key }}
When running the above pipeline only system and build variables are listed (e.g. system, system.hostType, build.queuedBy, etc.).
Any help to loop through user-defined variables would be much appreciated.
Unfortunately, no luck fetching the variables defined in UI. However, if your variables are non-secrets, you can bring them over into the YAML, and they will show up in the loop.
- stage:
variables:
myyamlvar: 1000 # this will show up in the loop
jobs:
- job: TestVars
steps:
- ${{ each var in variables }}:
- script: |
echo ${{ var.key }}
echo ${{ var.value }}
displayName: ${{ var.key }}
Alternatively, instead of using a compile time expression, you can list variables using a runtime construct, for example:
- job: TestRuntimeVars
steps:
- script: |
for var in $(compgen -e); do
echo $var ${!var};
done
This will list all variables including ones defined in the UI.
From the Microsoft docs link you provided, it specifies that:
"Unlike a normal variable, they are not automatically decrypted into
environment variables for scripts. You need to explicitly map secret
variables."
However, one workaround could potentially be to run an azure cli task and get the pipeline variables using az pipelines variable list
Assuming your intention is to get the actual values, in which case maybe that won't suffice. Having said that, you should consider a variable group even if you're not using them in other pipelines since the group can be linked to an Azure KeyVault and map the secrets as variables. You can store your sensitive values in a KeyVault and link it to the variable group which can be used like regular variables in your pipeline.
Or you can access KeyVault secrets right from the AzureKeyVault pipeline task.
To expand on the awnser below. It is a bit round about but you can use the azure devopps CLI. This may be a bit overkill but it does do the job.
trigger:
- main
pool:
vmImage: ubuntu-latest
steps:
- bash: az --version
displayName: 'Show Azure CLI version'
- bash: az devops configure --defaults organization=$(System.TeamFoundationCollectionUri) project=$(System.TeamProject) --use-git-aliases true
displayName: 'Set default Azure DevOps organization and project'
- bash: |
az pipelines variable list --pipeline-id $(System.DefinitionId)
displayName: 'Show build list varibales'
env:
AZURE_DEVOPS_EXT_PAT: $(System.AccessToken)
This approach was taken from a combination of:
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#list-variables
and
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#list-variables
If the agent is self hosted you may need to install the dev opps cli.
Microsoft provides documentation link and Azure CLI extension for deploying single container (no group of containers, using docker-compose or other containers grouping tool) to Azure Container Instance:
https://learn.microsoft.com/en-us/azure/container-instances/container-instances-github-action#configure-github-workflow
https://github.com/Azure/deploy-to-azure-cli-extension
How to modify this example GitHub workflow YAML file from documentation to achieve same effect for containers group, using in example docker-compose:
on: [push]
name: Linux_Container_Workflow
jobs:
build-and-deploy:
runs-on: ubuntu-latest
steps:
# checkout the repo
- name: 'Checkout GitHub Action'
uses: actions/checkout#main
- name: 'Login via Azure CLI'
uses: azure/login#v1
with:
creds: ${{ secrets.AZURE_CREDENTIALS }}
- name: 'Build and push image'
uses: azure/docker-login#v1
with:
login-server: ${{ secrets.REGISTRY_LOGIN_SERVER }}
username: ${{ secrets.REGISTRY_USERNAME }}
password: ${{ secrets.REGISTRY_PASSWORD }}
- run: |
docker build . -t ${{ secrets.REGISTRY_LOGIN_SERVER }}/sampleapp:${{ github.sha }}
docker push ${{ secrets.REGISTRY_LOGIN_SERVER }}/sampleapp:${{ github.sha }}
- name: 'Deploy to Azure Container Instances'
uses: 'azure/aci-deploy#v1'
with:
resource-group: ${{ secrets.RESOURCE_GROUP }}
dns-name-label: ${{ secrets.RESOURCE_GROUP }}${{ github.run_number }}
image: ${{ secrets.REGISTRY_LOGIN_SERVER }}/sampleapp:${{ github.sha }}
registry-login-server: ${{ secrets.REGISTRY_LOGIN_SERVER }}
registry-username: ${{ secrets.REGISTRY_USERNAME }}
registry-password: ${{ secrets.REGISTRY_PASSWORD }}
name: aci-sampleapp
location: 'west us'
Now I think the idea of creating YAML GitHub workflow file to handle group of containers is bad approach. Building & pushing just one single image of container is enough: if we update image of concrete container, which is already in group of containers, this container will be automatically updated. Our container image's Azure Container Registry is linked to container group's Azure Container Instance
I have pipeline where running template as one of steps.
I need execute template with two different scopes of parameters depends on value of one of pipeline variable.
For example, if pipeline variable 'mode' has value 'dev', I need run template with one scope, if value is 'stage', another scope.
- ${{ if eq(parameters.mode, 'dev') }}:
- template: ../template.yaml
parameters:
mode: dev
namespace: dev-namespace
fqdn: dev-fqdn
- ${{ if eq(variables.mode, 'staging') }}:
- template: ../template.yaml
parameters:
mode: staging
namespace: staging-namespace
fqdn: staging-fqdn
You should use this syntax ${{ if eq(variables['mode'], 'staging') }}:
It depends at what time the variables will be read in at and used. There are more then possible answer.
If using these for the future execution of a template I'd recommend using condition and defining the parameters you have in separate yml variable files.
condition: eq('${{ variables.mode}}', 'staging)
That way the call would be just
job:
condition: eq('${{ variables.mode}}', 'staging')
variables: staging.yml
- template: ../template.yaml
parameters:
mode: ${{ variables.mode }}
namespace: ${{ variables.namepsace }}
fqdn: ${{ variables.fqdn }}
And staging.yml would look like:
variables:
mode: staging
namespace: staging-namespace
fqdn: staging-fqdn
This way it is a little less decoupled and scalable not having to hard code it every time.
1.If you're using runtime parameters, you can use this format:
trigger:
- master
parameters:
- name: mode
displayName: Choose mode
type: string
default: dev
values:
- dev
- staging
pool:
vmImage: 'windows-latest'
steps:
- ${{ if eq(parameters.mode, 'dev') }}:
- template: template.yml
parameters:
mode: dev
- ${{ if eq(parameters.mode, 'staging') }}:
- template: template.yml
parameters:
mode: staging
For more details you can check Parameters to select a template at runtime.
2.If you're using variables defined in yaml directly, check my another sample:
trigger:
- master
variables:
mode: staging
pool:
vmImage: 'windows-latest'
stages:
- stage: Dev
condition: eq(variables.mode, 'dev')
jobs:
- job:
steps:
- template: template.yml
parameters:
mode: dev
- stage: Staging
condition: eq(variables.mode, 'staging')
jobs:
- job:
steps:
- template: template.yml
parameters:
mode: staging
I would recommend runtime parameters way cause it's more convenient and much more flexible.
Update1:
Here's my simple template.yml file which is just for test:
parameters:
- name: mode
type: string
default: default
steps:
- script: echo ${{ parameters.mode }}
I'm trying to set the variable group according to one varible that exists in the pipeline. The yaml looks like this:
But i'm getting the following error when i'm running the pipeline:
If i remove the "- group : QA" or "- group : PROD" the pipeline runs without any problem. What am I doing wrong?
This is slightly different solution but you may achieve your goal - which is if I understood conditional selection of variable group.
You can use runtime parameters:
parameters:
- name: environment
displayName: Environment
type: string
default: QA
values:
- QA
- PROD
stages:
- stage:
displayName: 'Build and Restore'
variables:
- group: ${{ parameters.environment }}
jobs:
- job:
steps:
- script: echo $(name)
than running a build you can select your envrionment:
Note: I have defined two variable groups QA and PROD with variable name in both groups.
Try with below schema:
variables:
isProd: true
stages:
- stage: Test
displayName: Build and restore
variables:
- ${{ if eq(variables['isProd'], 'false') }}:
- group: QA
- ${{ if eq(variables['isProd'], 'true') }}:
- group: PROD
jobs:
- job: A
steps:
- bash: echo $(groupname)
Note: You may receive some red warning when you defining above YAML scipt. Ignore that confused warning and continue to run it.