This is part of my Yml file
I need to re-call this template if it failed.it should be rerun again. After a few seconds, ideally.I am new to yml files
I tried to use retryCountOnTaskFailure but it should be under task but the template calling different hierarchy
https://learn.microsoft.com/en-us/azure/devops/release-notes/2021/sprint-195-update#automatic-retries-for-a-task
- template: ${{variables['System.DefaultWorkingDirectory']}}
parameters:
Test: ${{ parameters.isTestRelease }}
Environment: ${{ parameters.deploymentTarget }}
Component: '${{ parameters.component }}'
The retryCountOnTaskFailure feature is applied to individual tasks within a pipeline. Templates aren't tasks, they act more like an include that expands the contents of the template into your pipeline.
# pipeline.yml
trigger: none
parameters:
- name: isTestRelease
type: boolean
default: false
- name: deploymentTarget
type: string
default: DEV
values:
- DEV
- QA
- UAT
- PROD
- name: component
type: string
default: 'x'
stages:
- template: my-template.yml
parameters:
Test: ${{ parameters.isTestRelease }}
Environment: ${{ parameters.deploymentTarget }}
Component: ${{ parameters.component }}
And within the template, you'd want to add the retry logic to specific tasks:
# my-template.yml
parameters:
- name: isTestRelease
type: boolean
- name: component
type: string
stages:
- stage: Test
jobs:
- job: 1
steps:
- task: ...
- task: ...
- task: ...
retryCountOnFailure: 2
- task: ...
It's also located under "Control Options" when the "Command Line" task is added:
Related
I want to get the environments and the associated variableGroup - loop thru the environments array in Stages.yml and create a stage per environment... insert correct variableGroup per stage... use variablesGroup values to perform jobs in jobs.yml - each variable group contains the same vars.. with different values.
``
I have a main.yml file in which I reference my environments and the associated group variable
main.yaml :
name: $(BuildID)
trigger: none
pool:
vmImage: 'leetchi-agentspool-windows'
variables:
- template: vars/variables-api.yaml
extends:
template: deploy-stages.yaml
parameters:
environements:
- environement: 'DV1'
variableGroupName: tf-dv1
- environement: 'IN1'
variableGroupName: tf-in1
A stage template which is supposed to apply with the environment parameters and the group variable.
stage.yaml :
---
parameters:
- name: environements
type: object
default: []
stages:
- ${{ each environement in parameters.environements }}:
- stage: 'Deploy to ${{ environement.environements }}'
variables:
- group: ${{ environement.variableGroup }}
jobs:
- template: template-terraform-deployment.yml
parameters:
environement: ${{environement.environements}}
project: $(project)
definition: $(def)
project1: $(project1)
definition1: $(def1)
stateKey: ${{environement.environement}}$(tfkey)
TerraformDirectory: $(TfDirectory)
azureSubscription: $(service_connection_name)
TfStateResourceGroupName: $(tf_resource_group)
TfStateStorageAccountName: $(tf_storage_account)
TStateContainerName: $(tf_container)
commandOptions: -out=$(System.DefaultWorkingDirectory)/terraform.tfplan -detailed-exitcode
I start in YAML. Could you help me please? Thank you very much in advance
You can modify your main.yaml as below:
trigger: none
pool:
vmImage: 'leetchi-agentspool-windows'
variables:
- template: vars/variables-api.yaml
extends:
template: deploy-stages.yaml
parameters:
- name: environment
values:
- DV1
- IN1
- name: variableGroupName
values:
- tf-dv1
- tf-in1
stage.yaml :
stages:
- stage: 'Deploy to ${{ parameters.environment }}'
variables:
- group: ${{ parameters.variableGroupName}}
jobs:
- template: template-terraform-deployment.yml
parameters:
environment: ${{ parameters.environment }}
project: $(project)
definition: $(def)
project1: $(project1)
definition1: $(def1)
stateKey: ${{ parameters.environment }} + $(tfkey)
TerraformDirectory: $(TfDirectory)
azureSubscription: $(service_connection_name)
TfStateResourceGroupName: $(tf_resource_group)
TfStateStorageAccountName: $(tf_storage_account)
TStateContainerName: $(tf_container)
commandOptions: -out=$(System.DefaultWorkingDirectory)/terraform.tfplan -detailed-exitcode
I am facing an issue I could not resolve.
regarding this documentation it seams to be possible to set a variable as output of a job in the first stage, then use it as input to set a job variable in second stage.
The pbm I'm facing, is that my stages are parametrised, so their name are not constant.
here my sample code :
name: mypipeline
parameters:
- name: Application
displayName: Application to deploy the front end using front door
type: string
- name: stage_vars
displayName: stage VARS
type: object
default:
- stage_1
- stage_2
- none
trigger: none
stages:
- stage: Build
displayName: ${{ parameters.Application }} - Build.
pool:
name: Azure Pipelines
vmImage: windows-2019
jobs:
- template: ../templates/job_BuildAndPublish.yml
parameters:
BuildId: $(Build.BuildId)
importTerratest: false
importARM: true
- ${{ each stageregion in parameters.stage_vars }}:
- ${{ if ne(stageregion, 'none') }}:
- template: ../templates/isolated_web/tf_plan_stage.yml
parameters:
BuildId: $(system.BuildId)
Predecessors: 'Build'
- template: ../templates/isolated_web/tf_apply_stage.yml
parameters:
Predecessors: '${{stageregion}}_prepare'
build template is not usefull, but the ones from tf_plan_stage and tf_apply_stage are below :
tf_plan_stage.yml :
parameters:
- name: BuildId
type: string
default: $(system.BuildId)
- name: stage
type: string
default: Deploy_prepare
- name: Predecessors
type: string
default: none
stages:
- stage: ${{ parameters.stage }}
? ${{ if and(ne(parameters.Predecessors, 'previous'), ne(parameters.Predecessors, 'none') ) }}
: dependsOn: ${{parameters.Predecessors}}
displayName: ${{ parameters.TF_VAR_STAGE }} Terraform plan & publish artifact for ${{ parameters.TF_VAR_APPLICATION }}
pool:
name: Azure Pipelines
vmImage: windows-2019
jobs:
- deployment: PreDeployTerraform
displayName: ${{ parameters.TF_VAR_STAGE }} Terraform plan & publish artifact
environment: PLAN_${{ parameters.TF_VAR_ENVIRONMENT }}
timeoutInMinutes: 480
strategy:
runOnce:
deploy:
steps:
- checkout: none
# set output planAttempt output available for later Apply job
- powershell: |
echo "jobAttempt is $(System.JobAttempt)"
echo "##vso[task.setvariable variable=planAttempt;isOutput=true;]$(System.JobAttempt)"
name: setVarJobAttempt
- powershell: |
echo "variable setVarJobAttempt.planAttempt value is : $(setVarJobAttempt.planAttempt)"
name: getplanAttemptVar
tf_apply_stage.yml :
parameters:
- name: stage
type: string
default: Deploy_prepare
- name: Predecessors
type: string
default: none
stages:
- stage: ${{ parameters.stage }}
? ${{ if and(ne(parameters.Predecessors, 'previous'), ne(parameters.Predecessors, 'none') ) }}
: dependsOn: ${{parameters.Predecessors}}
displayName: ${{ parameters.TF_VAR_STAGE }} download artifact & Terraform apply changes to ${{ parameters.TF_VAR_APPLICATION }}
pool:
name: Azure Pipelines
vmImage: windows-2019
jobs:
- deployment: DeployTerraform
displayName: ${{ parameters.TF_VAR_STAGE }} download artifact & Terraform apply changes
variables:
# this one fails in pipeline : "[not recognise"
planJobAttempt: $[ stageDependencies.[parameters.Predecessors].PreDeployTerraform.outputs['setVarJobAttempt.planAttempt'] ]
# this one runs with no result in pipeline : planJobAttempt is the string "stageDependencies['parameters.Predecessors'].PreDeployTerraform.outputs['setVarJobAttempt.planAttempt']"
# planJobAttempt: $[ stageDependencies['parameters.Predecessors'].PreDeployTerraform.outputs['setVarJobAttempt.planAttempt'] ]
environment: ${{ parameters.TF_VAR_ENVIRONMENT }}
timeoutInMinutes: 480
strategy:
runOnce:
deploy:
steps:
- checkout: none
- powershell: |
echo "jobAttempts outputs are : $[ stageDependencies.[parameters.Predecessors].PreDeployTerraform.outputs ]"
- powershell: |
echo "jobAttempt is $[ stageDependencies['parameters.Predecessors'].PreDeployTerraform.outputs['setVarJobAttempt.planAttempt'] ]"
- powershell: |
echo "plan file for terraform is : $(pipeline_artifact_folder_download)/$(planJobAttempt)_${{ parameters.Predecessors }}/$(artefact_terraform_plan)_${{ parameters.TF_VAR_STAGE }}"
I tried differents things to get my system.jobAttempts from tf_plan_stage stage to tf_apply_stage stage, but without any success, as the variable in the last stage (tf_apply) seams unable to find the value from a "parametrised" stage name.
is there a way for that ?
thank-you for any answers.
Okay,
Once again, I will answer my own question :D
I've finallt find the way to use the parameters.Predecessors value in my variable declaration.
And, second, the Microsoft documentation seams not to be updated.
The tf_apply_stage.yml should be like that below ;
parameters:
- name: stage
type: string
default: Deploy_prepare
- name: Predecessors
type: string
default: none
stages:
- stage: ${{ parameters.stage }}
? ${{ if and(ne(parameters.Predecessors, 'previous'), ne(parameters.Predecessors, 'none') ) }}
: dependsOn: ${{parameters.Predecessors}}
displayName: ${{ parameters.TF_VAR_STAGE }} download artifact & Terraform apply changes to ${{ parameters.TF_VAR_APPLICATION }}
pool:
name: Azure Pipelines
vmImage: windows-2019
jobs:
- deployment: DeployTerraform
displayName: ${{ parameters.TF_VAR_STAGE }} download artifact & Terraform apply changes
variables:
*# this one give he good value for Variable*
planJobAttempt: $[ stageDependencies.${{ parameters.Predecessors }}.PreDeployTerraform.outputs['PreDeployTerraform.setVarJobAttempt.planAttempt'] ]
environment: ${{ parameters.TF_VAR_ENVIRONMENT }}
timeoutInMinutes: 480
strategy:
runOnce:
deploy:
steps:
- checkout: none
- powershell: |
echo "plan file for terraform is : $(pipeline_artifact_folder_download)/$(planJobAttempt)_${{ parameters.Predecessors }}/$(artefact_terraform_plan)_${{ parameters.TF_VAR_STAGE }}"
I put in BOLD the main changes above :
planJobAttempt: $[ stageDependencies.${{ parameters.Predecessors
}}.PreDeployTerraform.outputs['PreDeployTerraform.setVarJobAttempt.planAttempt']
]
The Job name have to be set twice in variable declaration !
Thanks for those who read my post ;)
And many thanks to this post from #Kay07949 who gives the good answers ;)
I have template
dockerbuild.yml
steps:
- task: Bash#3
displayName: Build an image to container registry
inputs:
script : echo $(PATH)
then a variable file
var.build.yml
- name: PATH
value: 'docker/path'
- name: PATH1
value: 'docker/oldpath'
- name: PATH2
value: 'docker/newpath'
azurepipeline1.yml
resources:
repositories:
- repository: templates
type: git
name: components/pipeline_templates
trigger:
- none
pool:
name: PoolA
variables:
- template: variabletemplates/var.build.yml#templates
jobs:
steps:
- template: CI-CD/dockerbuild.yml#templates # Template reference
parameters:
PATH: ${{ variables.PATH }}
- template: CI-CD/dockerbuild.yml#templates # Template reference
parameters:
PATH: ${{ variables.PATH1 }}
- template: CI-CD/dockerbuild.yml#templates # Template reference
parameters:
PATH: ${{ variables.PATH2 }}
The above code works
But if I change the parameter to
parameters:
PATH: ${{ variables.PATH1 }}
OR
parameters:
PATH: ${{ variables.PATH2 }}
In the output it shows
Output:
docker/path
$(PATH1)
$(PATH2)
I need to reuse the template with many times within the pipeline with different paths. Please help in resolving the issue
Your template needs a parameters block.
parameters:
- name: PATH
type: string
steps:
- task: Bash#3
displayName: Build an image to container registry
inputs:
script : echo ${{ parameters.PATH }}
I have a Azure DevOps YAML Pipeline to execute a Terraform deployment using the Terraform by MS DevLabs extension and an Azure Resource Manager service connection.
The last working state was using a pipeline template yaml file however I had to configure a parameter within the template and call the variable using the template expression syntax.
...
...
stages:
- stage: Plan
displayName: Terrafom Plan
jobs:
- job: DEV PLAN
displayName: Plan (DEV)
pool:
vmImage: "ubuntu-latest"
variables:
az_service_connection: "MyServiceConnection"
tf_environment: "DEV"
tf_state_rg: "DEV"
tz_state_location: "canadacentral"
tf_state_stgacct_name: "mystorageaccuontname1231231"
tf_state_container_name: "tfstate"
steps:
- template: templates/terraform-plan.yml
parameters:
az_service_connection: ${{ variables.az_service_connection }}
...
...
steps:
- task: terraformInstaller#0
displayName: "Install Terraform $(tf_version)"
inputs:
terraformVersion: $(tf_version)
- task: TerraformTaskV1#0
displayName: "Run > terraform init"
inputs:
command: "init"
commandOptions: "-input=false"
backendServiceArm: ${{ parameters.az_service_connection }}
...
...
I believe the reason why this works is because the template expression syntax ${{ variables.varname}} evaluates at compile time vs. runtime. If I didn't do it this way, i'd either get $(az_service_connection) passed into the backendServiceArm input or an empty value.
With the introduction of variable groups, i'm now facing similar behavior. I expect that the variable group evaluates after the template expression variable which causes ${{ variables.az_service_connection }} to have an empty value. I am unsure how to get this working.
How can I use variable groups with a pipeline template that uses a service connection?
I used $() syntax to pass arm connection to template:
Template file:
parameters:
- name: 'instances'
type: object
default: {}
- name: 'server'
type: string
default: ''
- name: 'armConnection'
type: string
default: ''
steps:
- task: TerraformTaskV1#0
inputs:
provider: 'azurerm'
command: 'init'
backendServiceArm: '${{ parameters.armConnection }}'
backendAzureRmResourceGroupName: 'TheCodeManual'
backendAzureRmStorageAccountName: 'thecodemanual'
backendAzureRmContainerName: 'infra'
backendAzureRmKey: 'some-terrform'
- ${{ each instance in parameters.instances }}:
- script: echo ${{ parameters.server }}:${{ instance }}
Main file:
trigger:
branches:
include:
- master
paths:
include:
- stackoverflow/09-array-parameter-for-template/*
# no PR triggers
pr: none
pool:
vmImage: 'ubuntu-latest'
variables:
- group: my-variable-group
- name: my-passed-variable
value: $[variables.myhello] # uses runtime expression
steps:
- template: template.yaml
parameters:
instances:
- test1
- test2
server: $(myhello)
armConnection: $(armConnection)
Note: Group my-variable-group contains armConnection variable
I'm trying to come up with a way to allow an easy selection of a given agent within the build pipeline just for debugging purposes. So far I have the below snippet. Both work without the if snippets wrapped around but I was trying to do one or the other based on either params being set or inturn variables being set so that if it was in debug mode it would select and agent and if it wasn't then it would just use the pool to select an agent to run the builds against. So far no luck though.
variables:
debugMode: 'false'
parameters:
- name: poolOption
type: string
default: 'ZupaDeploymentPool'
- name: debugMode
type: string
default: 'true'
- name: debugMachine
type: string
default: 'ZUPBUILD03'
trigger:
batch: true
branches:
include:
- master
paths:
exclude:
- README.md
${{ if ne($(debugMode), 'false') }}:
pool: ${{ parameters.poolOption }}
${{ if ne($(debugMode), 'true') }}:
pool:
name: ${{ parameters.poolOption }}
demands:
- Agent.Name -equals ${{ parameters.debugMachine }}
So after having a quick chat with kevin-lu-msft above lead me to this solution for handling selecting a specific agent in the pool.
parameters:
- name: debugMachine
displayName: 'Run on selected agent:'
type: string
default: 'Auto Select From Pool'
values:
- 'Auto Select From Pool'
- 'MACHINE01'
- 'MACHINE02'
- 'MACHINE03'
- 'MACHINE04'
- 'MACHINE05'
jobs:
-job: build
# -----------------------------------------------------------------------------------------------
# This is the pool selection section if a specific debug machine is passed in via the params
# It will select that specific one to run the build on. unfortunately azure doesnt let you pass
# vars or params to the process string in the demands which would have made this alot cleaner.
pool:
name: 'YOUAGENTSPOOLNAME'
${{ if ne(parameters.debugMachine, 'Auto Select From Pool') }}:
${{ if eq(parameters.debugMachine, 'MACHINE01') }}:
demands:
- Agent.Name -equals MACHINE01
${{ if eq(parameters.debugMachine, 'MACHINE02') }}:
demands:
- Agent.Name -equals MACHINE02
${{ if eq(parameters.debugMachine, 'MACHINE03') }}:
demands:
- Agent.Name -equals MACHINE03
${{ if eq(parameters.debugMachine, 'MACHINE04') }}:
demands:
- Agent.Name -equals MACHINE04
${{ if eq(parameters.debugMachine, 'MACHINE05') }}:
demands:
- Agent.Name -equals MACHINE05
steps:
- script: "echo finally it works"
I tested your YAML sample and made some modifications. You could try to set the "Expressions" under a stage and check if it could work as expected.
Here is an example, you could refer to it.
trigger:
- master
parameters:
- name: poolOption
type: string
default: 'Windows-latest'
- name: debugMode
type: string
default: false
values:
- true
- false
- name: debugMachine
type: string
default: 'ubuntu-16.04'
stages:
- stage: A
jobs:
- job: testjob
pool:
${{ if eq(parameters.debugmode, 'true') }}:
vmImage: ${{ parameters.poolOption }}
${{ if eq(parameters.debugmode, 'false') }}:
vmImage: ${{ parameters.debugMachine }}
steps:
- script : "echo Hello, world!"
Note: I am using Microsoft-hosted agents, so I am using the vmImage field.
You can specify specific self-hosted agents (name field) according to your needs.