Is there a way to handle a single parameter as a build number OR source branch in an azure pipeline? - azure

I'm trying to allow for a single variable that can specify the build id or branch to deploy from. This means I need to type coerce, and it's failing if I try to call a ge (greater than or equal) or lt (less than) on a string. It looks like I might have to use bash commands to do the type conversion for me, so I was wondering if anyone had this handy.
Here's the steps I'm using:
- ${{ each deployment in parameters.deployments }}:
- deployment: Deploy_${{deployment.serviceName}}_${{ parameters.region }}
pool:
vmImage: ${{ parameters.vmImage }}
displayName: 'Deploy ${{ deployment.serviceName }} ${{ parameters.region }}'
${{ if not(eq(parameters.kubernetesServiceEndpoint, '')) }}:
environment: ${{ parameters.kubernetesServiceEndpoint }}
${{ elseif not(and(eq(parameters.azureResourceGroup, ''), eq(parameters.kubernetesCluster, ''))) }}:
environment: ${{ parameters.environment }}
strategy:
runOnce:
deploy:
steps:
# Disable the automatic downloading of artifacts, because we will specifying exactly what we need
- download: none
- task: DownloadPipelineArtifact#2
condition: ${{ ge(0, deployment.branchBuildId) }}
inputs:
source: 'specific'
project: ${{ deployment.project }}
pipeline: ${{ deployment.pipeline }}
runVersion: 'latestFromBranch'
runBranch: ${{ deployment.branchBuildId }}
patterns: 'DeploymentData/*'
displayName: 'Download Latest Artifacts'
- task: DownloadPipelineArtifact#2
condition: ${{ lt(0, deployment.branchBuildId) }}
inputs:
source: 'specific'
project: ${{ deployment.project }}
pipeline: ${{ deployment.pipeline }}
runVersion: 'specific'
runId: ${{ deployment.branchBuildId }}
#runBranch: ${{ deployment.branch }}
patterns: 'DeploymentData/*'
displayName: 'Download Specific Artifacts (${{ deployment.buildId }})'
I want these to be mutually exclusive (run the first if pulling the latest from a specific branch, and run the second if downloading from a specific build id). By using a single parameter I can avoid another control parameter to decide which version should be used.

Two things:
1, If you use script to handle, then 'compile time usage' is unable to use.
You need to use runtime usage:
variables['xxx']
2, Variables only support passing string, you can compare them and then use logging command to output the result:
trigger:
- none
pool:
vmImage: ubuntu-latest
parameters:
- name: str1
default: 110
- name: str2
default: 130
steps:
- task: PowerShell#2
name: outputresult
inputs:
targetType: 'inline'
script: |
$str1 = "${{parameters.str1}}"
$str2 = "${{parameters.str2}}"
Write-Host $str1
Write-Host $str2
# powershell compare the strings greater than or less than
if ($str1 -gt $str2) {
$result = "greater than"
Write-Host $result
Write-Host "##vso[task.setvariable variable=result;isoutput=true]$result"
} elseif ($str1 -lt $str2) {
$result = "less than"
Write-Host $result
Write-Host "##vso[task.setvariable variable=result;isoutput=true]$result"
} else {
$result = "equal"
Write-Host $result
Write-Host "##vso[task.setvariable variable=result;isoutput=true]$result"
}
#Below is the handling logic
- task: PowerShell#2
condition: eq('greater than',variables['outputresult.result'])
inputs:
targetType: 'inline'
script: |
# Write your PowerShell commands here.
Write-Host "greater than"
- task: PowerShell#2
condition: eq('less than',variables['outputresult.result'])
inputs:
targetType: 'inline'
script: |
# Write your PowerShell commands here.
Write-Host "less than"
Result:

Related

Aure pipeline : use output variable in multi-stage parametrised

I am facing an issue I could not resolve.
regarding this documentation it seams to be possible to set a variable as output of a job in the first stage, then use it as input to set a job variable in second stage.
The pbm I'm facing, is that my stages are parametrised, so their name are not constant.
here my sample code :
name: mypipeline
parameters:
- name: Application
displayName: Application to deploy the front end using front door
type: string
- name: stage_vars
displayName: stage VARS
type: object
default:
- stage_1
- stage_2
- none
trigger: none
stages:
- stage: Build
displayName: ${{ parameters.Application }} - Build.
pool:
name: Azure Pipelines
vmImage: windows-2019
jobs:
- template: ../templates/job_BuildAndPublish.yml
parameters:
BuildId: $(Build.BuildId)
importTerratest: false
importARM: true
- ${{ each stageregion in parameters.stage_vars }}:
- ${{ if ne(stageregion, 'none') }}:
- template: ../templates/isolated_web/tf_plan_stage.yml
parameters:
BuildId: $(system.BuildId)
Predecessors: 'Build'
- template: ../templates/isolated_web/tf_apply_stage.yml
parameters:
Predecessors: '${{stageregion}}_prepare'
build template is not usefull, but the ones from tf_plan_stage and tf_apply_stage are below :
tf_plan_stage.yml :
parameters:
- name: BuildId
type: string
default: $(system.BuildId)
- name: stage
type: string
default: Deploy_prepare
- name: Predecessors
type: string
default: none
stages:
- stage: ${{ parameters.stage }}
? ${{ if and(ne(parameters.Predecessors, 'previous'), ne(parameters.Predecessors, 'none') ) }}
: dependsOn: ${{parameters.Predecessors}}
displayName: ${{ parameters.TF_VAR_STAGE }} Terraform plan & publish artifact for ${{ parameters.TF_VAR_APPLICATION }}
pool:
name: Azure Pipelines
vmImage: windows-2019
jobs:
- deployment: PreDeployTerraform
displayName: ${{ parameters.TF_VAR_STAGE }} Terraform plan & publish artifact
environment: PLAN_${{ parameters.TF_VAR_ENVIRONMENT }}
timeoutInMinutes: 480
strategy:
runOnce:
deploy:
steps:
- checkout: none
# set output planAttempt output available for later Apply job
- powershell: |
echo "jobAttempt is $(System.JobAttempt)"
echo "##vso[task.setvariable variable=planAttempt;isOutput=true;]$(System.JobAttempt)"
name: setVarJobAttempt
- powershell: |
echo "variable setVarJobAttempt.planAttempt value is : $(setVarJobAttempt.planAttempt)"
name: getplanAttemptVar
tf_apply_stage.yml :
parameters:
- name: stage
type: string
default: Deploy_prepare
- name: Predecessors
type: string
default: none
stages:
- stage: ${{ parameters.stage }}
? ${{ if and(ne(parameters.Predecessors, 'previous'), ne(parameters.Predecessors, 'none') ) }}
: dependsOn: ${{parameters.Predecessors}}
displayName: ${{ parameters.TF_VAR_STAGE }} download artifact & Terraform apply changes to ${{ parameters.TF_VAR_APPLICATION }}
pool:
name: Azure Pipelines
vmImage: windows-2019
jobs:
- deployment: DeployTerraform
displayName: ${{ parameters.TF_VAR_STAGE }} download artifact & Terraform apply changes
variables:
# this one fails in pipeline : "[not recognise"
planJobAttempt: $[ stageDependencies.[parameters.Predecessors].PreDeployTerraform.outputs['setVarJobAttempt.planAttempt'] ]
# this one runs with no result in pipeline : planJobAttempt is the string "stageDependencies['parameters.Predecessors'].PreDeployTerraform.outputs['setVarJobAttempt.planAttempt']"
# planJobAttempt: $[ stageDependencies['parameters.Predecessors'].PreDeployTerraform.outputs['setVarJobAttempt.planAttempt'] ]
environment: ${{ parameters.TF_VAR_ENVIRONMENT }}
timeoutInMinutes: 480
strategy:
runOnce:
deploy:
steps:
- checkout: none
- powershell: |
echo "jobAttempts outputs are : $[ stageDependencies.[parameters.Predecessors].PreDeployTerraform.outputs ]"
- powershell: |
echo "jobAttempt is $[ stageDependencies['parameters.Predecessors'].PreDeployTerraform.outputs['setVarJobAttempt.planAttempt'] ]"
- powershell: |
echo "plan file for terraform is : $(pipeline_artifact_folder_download)/$(planJobAttempt)_${{ parameters.Predecessors }}/$(artefact_terraform_plan)_${{ parameters.TF_VAR_STAGE }}"
I tried differents things to get my system.jobAttempts from tf_plan_stage stage to tf_apply_stage stage, but without any success, as the variable in the last stage (tf_apply) seams unable to find the value from a "parametrised" stage name.
is there a way for that ?
thank-you for any answers.
Okay,
Once again, I will answer my own question :D
I've finallt find the way to use the parameters.Predecessors value in my variable declaration.
And, second, the Microsoft documentation seams not to be updated.
The tf_apply_stage.yml should be like that below ;
parameters:
- name: stage
type: string
default: Deploy_prepare
- name: Predecessors
type: string
default: none
stages:
- stage: ${{ parameters.stage }}
? ${{ if and(ne(parameters.Predecessors, 'previous'), ne(parameters.Predecessors, 'none') ) }}
: dependsOn: ${{parameters.Predecessors}}
displayName: ${{ parameters.TF_VAR_STAGE }} download artifact & Terraform apply changes to ${{ parameters.TF_VAR_APPLICATION }}
pool:
name: Azure Pipelines
vmImage: windows-2019
jobs:
- deployment: DeployTerraform
displayName: ${{ parameters.TF_VAR_STAGE }} download artifact & Terraform apply changes
variables:
*# this one give he good value for Variable*
planJobAttempt: $[ stageDependencies.${{ parameters.Predecessors }}.PreDeployTerraform.outputs['PreDeployTerraform.setVarJobAttempt.planAttempt'] ]
environment: ${{ parameters.TF_VAR_ENVIRONMENT }}
timeoutInMinutes: 480
strategy:
runOnce:
deploy:
steps:
- checkout: none
- powershell: |
echo "plan file for terraform is : $(pipeline_artifact_folder_download)/$(planJobAttempt)_${{ parameters.Predecessors }}/$(artefact_terraform_plan)_${{ parameters.TF_VAR_STAGE }}"
I put in BOLD the main changes above :
planJobAttempt: $[ stageDependencies.${{ parameters.Predecessors
}}.PreDeployTerraform.outputs['PreDeployTerraform.setVarJobAttempt.planAttempt']
]
The Job name have to be set twice in variable declaration !
Thanks for those who read my post ;)
And many thanks to this post from #Kay07949 who gives the good answers ;)

How do I reference a different Variable group per stage - stage per environment in environments - Azure Devops, template, variable groups

When I run the pipeline below I get an error within jobs.yml - it commplains about an "unexpected value" for the environment parameter... and within stages.yml "unexpected value: parameters" - what am I doing wrong here? The idea is to get the environments and the associated variableGroup - loop thru the environments array in Stages.yml and create a stage per environment... insert correct variableGroup per stage... use variablesGroup values to perform jobs in jobs.yml - each variable group contains the same vars.. with different values.
This is main.yml
#main.yml
trigger: none
extends:
template: /Build/Stages.yml
parameters:
environments:
- environment: Dev
variableGroup: Project-Dev-VarGrp
- environment: QA
variableGroup: Project-QA-VarGrp
dependsOn: Dev
- environment: UAT
variableGroup: Project-UAT-VarGrp
dependsOn: QA
- environment: UAT2
variableGroup: Project-UAT2-VarGrp
dependsOn: UAT
Then here is the next bit... Stages.yml
parameters:
- name: dataFolder
displayName: 'Data folder to process'
type: string
default: '/DataMigrations/Master_Data/'
- name: dataFiles
displayName: List of Data Files or Folder names
type: string
default: 'Dev_Data.zip'
- name: environments
type: object
default: []
stages:
- ${{ each environment in parameters.environments }}:
- stage: '${{ parameters.environment }}'
jobs:
- template: ./jobs.yml
variables:
- group: ${{ parameters.variableGroup }}
parameters:
environment: '${{ parameters.environment }}'
crmURL: $(crmURL)
oauthAppId: $(ClientID)
ClientPass: $(ClientPass)
dataFolder: '${{ parameters.dataFolder }}'
env: '${{ parameters.environment }}'
and here is jobs.yml
jobs:
- deployment: DeployData
displayName: 'Deploying Data to ${{ parameters.environment }}'
environment: ${{ parameters.environment }}
pool:
vmImage: 'windows-latest'
strategy:
runOnce:
deploy:
steps:
- checkout: self
clean: false
- powershell: |
Write-Host "##vso[task.setvariable variable=crmConnectionString]'AuthType=ClientSecret;Url=$(crmURL);ClientId=$(ClientID);ClientSecret=$(ClientPass)'"
displayName: 'PreDeploy configuration'
- task: PowerShell#2
displayName: 'Powershell: Run update-data.ps1'
inputs:
targetType: 'filePath'
filePath: $(System.DefaultWorkingDirectory)/DataMigrations/Scripts/update-data.ps1
arguments: -folderName '${{ parameters.dataFolder }}' -environment '${{ parameters.env }}'
workingDirectory: $(System.DefaultWorkingDirectory)/DataMigrations
- task: PowerShell#2
displayName: 'Powershell: Run zip-import-data.ps1'
inputs:
targetType: 'filePath'
filePath: $(System.DefaultWorkingDirectory)/DataMigrations/Scripts/zip-import-data.ps1
arguments: -folderName '${{ parameters.dataFolder }}' -dataMigrationFilenames '${{ parameters.dataFiles }}' -connectionString $(crmConnectionString)
workingDirectory: $(System.DefaultWorkingDirectory)/DataMigrations
Few things here:
When you're using ${{ each environment in parameters.environments }} then the nested environment and variableGroup are available using this syntax ${{ environment.environment }} and ${{ environment.variableGroup}}
In your Stages.yml file, you're trying to invoke the ./jobs.yml template but the associated parameters are defined after the - group: ${{ parameters.variableGroup }}. A valid syntax should looks like this:
stages:
- ${{ each environment in parameters.environments }}:
- stage: '${{ environment.environment }}'
variables:
- group: ${{ environment.variableGroup }}
jobs:
- template: ./jobs.yml
parameters:
environment: '${{ environment.environment }}'
crmURL: $(crmURL)
oauthAppId: $(ClientID)
ClientPass: $(ClientPass)
dataFolder: '${{ parameters.dataFolder }}'
env: '${{ environment.environment }}'
You also have few space typos. I know it's annoying but you need to have the exact yaml syntax otherwise the files can't be parsed.

Microsoft Dev Ops yaml variables between deployments

So at it's core what I'm essentially trying to do is this:
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/deployment-jobs?view=azure-devops&preserve-view=true#support-for-output-variables
Which breaks down to the fact that I have a single deployment template that produces an output variable that I want to map back to a variable in my main template so that I can use it in subsequent deployment templates.
I have created a github project with all of my yaml templates here
https://github.com/MCKRUZ/azure-pipelines
but the crux of the problem lies in these few lines. First I'm calling a template deployment in my main "development.yaml" file.
# Deploy
# ------
- stage: Deploy
displayName: Deploy - Development
jobs:
- template: jobs/jobs-deploy-spa-arm-template.yaml
parameters:
environment: '$(environment)'
serviceConnection: '$(serviceConnection)'
subscriptionId: '$(subscriptionId)'
resourceGroup: '$(resourceGroup)'
storageAccountName: '$(storageAccountName)'
cdnEndpointName: '$(cdnEndpointName)'
cdnProfileName: '$(cdnProfileName)'
deploymentLocation: '$(deploymentLocation)'
armTemplateArtifactName: ${{ variables['armTemplateArtifactName'] }}
storageAccountCSMFilePath: '$(storageAccountCSMFilePath)'
storageAccountCSMParametersFile: '$(storageAccountCSMParametersFile)'
deployCDN: '${{ variables.isProduction }}'
cdnCSMFilePath: '$(cdnCSMFilePath)'
cdnCSMParametersFile: '$(cdnCSMParametersFile)'
That template looks like this:
# Parameters
# ---------
parameters:
- name: environment
type: string
- name: serviceConnection
type: string
- name: subscriptionId
type: string
- name: resourceGroup
type: string
- name: storageAccountName
type: string
- name: cdnEndpointName
type: string
- name: cdnProfileName
type: string
- name: deploymentLocation
type: string
- name: armTemplateArtifactName
type: string
- name: storageAccountCSMFilePath
type: string
- name: storageAccountCSMParametersFile
type: string
- name: deployCDN
type: boolean
default: false
- name: cdnCSMFilePath
type: string
- name: cdnCSMParametersFile
type: string
- name: overrideParameterFile
type: boolean
default: true
# Jobs
# ---------
jobs:
# Job - Deploy SPA ARM Template
# ---------
- #deployment: '${{ parameters.armTemplateArtifactName }}'
deployment: armTemplates
variables:
storageAccountNameCalculated: ''
cdnEndpointNameCalculated: ''
displayName: Deploy SPA ARM Template
environment: ${{ parameters.environment }}
workspace:
clean: all
strategy:
runOnce:
deploy:
steps:
# Bash - Show Input Parameter Variables
# ---------
- bash: |
echo ''
echo '===== Deploy SPA ARM Template Input Variables ====='
echo ''
echo ' ===== Parameters ====='
echo ' environment: ${{ parameters.environment }}'
echo ' serviceConnection: ${{ parameters.serviceConnection }}'
echo ' subscriptionId: ${{ parameters.subscriptionId }}'
echo ' resourceGroup: ${{ parameters.resourceGroup }}'
echo ' storageAccountName: ${{ parameters.storageAccountName }}'
echo ' cdnEndpointName: ${{ parameters.cdnEndpointName }}'
echo ' cdnProfileName: ${{ parameters.cdnProfileName }}'
echo ' deploymentLocation: ${{ parameters.deploymentLocation }}'
echo ' armTemplateArtifactName: ${{ parameters.armTemplateArtifactName }}'
echo ' storageAccountCSMFilePath: ${{ parameters.storageAccountCSMFilePath }}'
echo ' storageAccountCSMParametersFile: ${{ parameters.storageAccountCSMParametersFile }}'
echo ' deployCDN: ${{ parameters.deployCDN }}'
echo ' cdnCSMFilePath: ${{ parameters.cdnCSMFilePath }}'
echo ' cdnCSMParametersFile: ${{ parameters.cdnCSMParametersFile }}'
echo ''
displayName: Debug - Input Parameters
# Task - Deploy Storage Account ARM Template
# ---------
- task: AzureResourceManagerTemplateDeployment#3
displayName: Deploy Storage Account ARM Template
inputs:
deploymentScope: 'Resource Group'
azureResourceManagerConnection: '${{ parameters.serviceConnection }}'
subscriptionId: '${{ parameters.subscriptionId }}'
action: 'Create Or Update Resource Group'
resourceGroupName: '${{ parameters.resourceGroup }}'
location: '${{ parameters.deploymentLocation }}'
templateLocation: 'Linked artifact'
csmFile: '$(Pipeline.Workspace)\${{ parameters.armTemplateArtifactName }}\${{ parameters.storageAccountCSMFilePath }}'
csmParametersFile: '$(Pipeline.Workspace)\${{ parameters.armTemplateArtifactName }}\${{ parameters.storageAccountCSMParametersFile }}'
deploymentMode: 'Incremental'
deploymentName: 'storageAccountDeployment'
deploymentOutputs: 'storageAccountDeploymentOutput'
${{ if eq(parameters.overrideParameterFile, 'true') }}:
overrideParameters: '-formatNames true -environment ${{ parameters.environment }} -storageAccountName ${{ parameters.storageAccountName }}'
# Task - Retrieve Output Variable
#----------
- task: AzurePowerShell#5
displayName: Retrieve Output Variables
name: retrieveStorageAccountDeploymentOutput
condition: succeeded()
inputs:
azureSubscription: '${{ parameters.serviceConnection }}'
scriptType: 'inlineScript'
azurePowerShellVersion: 'LatestVersion'
inline: |
$armOutput = '$(storageAccountDeploymentOutput)' | ConvertFrom-Json
Write-Output "##vso[task.setvariable variable=storageAccountNameCalculated;]$($armOutput.storageAccountName.value)"
- script: echo "##vso[task.setvariable variable=myOutputVar;isOutput=true]this is the deployment variable value"
name: setvarStep
- script: echo $(setvarStep.myOutputVar)
name: echovar
# Task - Add Static Hosting to Storage Account
# ---------
- task: AzurePowerShell#5
displayName: Add Static Hosting to Storage Account
condition: succeeded()
inputs:
azureSubscription: '${{ parameters.serviceConnection }}'
scriptType: 'inlineScript'
azurePowerShellVersion: latestVersion
inline: |
$StorageAccount = Get-AzStorageAccount -Name $(storageAccountNameCalculated) -ResourceGroupName '${{ parameters.resourceGroup }}'
Enable-AzStorageStaticWebsite -Context $StorageAccount.Context -IndexDocument index.html -ErrorDocument404Path index.html
# Task - Deploy CDN ARM Template
# ---------
- task: AzureResourceManagerTemplateDeployment#3
displayName: Deploy CDN ARM Template
condition: and(succeeded(), eq(${{ parameters.deployCDN }}, 'true'))
inputs:
deploymentScope: 'Resource Group'
azureResourceManagerConnection: '${{ parameters.serviceConnection }}'
subscriptionId: '${{ parameters.subscriptionId }}'
action: 'Create Or Update Resource Group'
resourceGroupName: '${{ parameters.resourceGroup }}'
location: '${{ parameters.deploymentLocation }}'
templateLocation: 'Linked artifact'
csmFile: '$(Pipeline.Workspace)\${{ parameters.armTemplateArtifactName }}\${{ parameters.cdnCSMFilePath }}'
csmParametersFile: '$(Pipeline.Workspace)\${{ parameters.armTemplateArtifactName }}\${{ parameters.cdnCSMParametersFile }}'
deploymentMode: 'Incremental'
deploymentName: 'cdnDeployment'
deploymentOutputs: 'CDNDeployOutput'
${{ if eq(parameters.overrideParameterFile, 'true') }}:
overrideParameters: '-formatNames true -environment ${{ parameters.environment }} -cdnEndpointName ${{ parameters.cdnEndpointName }} -cdnProfileName ${{ parameters.cdnProfileName }} -storageAccountPrimaryEndpoint -storageAccountPrimaryEndpoint $storageAccountNameCalculated.z13.web.core.windows.net'
# Task - Retrieve Output Variable
#----------
- task: AzurePowerShell#5
displayName: Retrieve Output Variables
condition: and(succeeded(), eq(${{ parameters.deployCDN }}, 'true'))
inputs:
azureSubscription: '${{ parameters.serviceConnection }}'
scriptType: 'inlineScript'
azurePowerShellVersion: 'LatestVersion'
inline: |
$armOutput = '$(CDNDeployOutput)' | ConvertFrom-Json
Write-Output "##vso[task.setvariable variable=cdnEndpointNameCalculated;isOutput=true]$($armOutput.cdnEndpointName)"
# Bash - Show Output Parameter Variables
# ---------
- bash: |
echo ''
echo '===== Deploy SPA ARM Template Output Variables ====='
echo ''
echo ' ===== Parameters ====='
echo ' storageAccountNameCalculated: $(storageAccountNameCalculated)'
echo ' cdnEndpointName: $(cdnEndpointNameCalculated)'
echo ''
displayName: Debug - Output Parameters
Where I'm using these two lines
- script: echo "##vso[task.setvariable variable=myOutputVar;isOutput=true]this is the deployment variable value"
name: setvarStep
- script: echo $(setvarStep.myOutputVar)
name: echovar
To try and isolate a way that I can create a variable that will be able to be used back in my main yaml file. When I look at the logs, the second echo successfully echos out the variable so I know that part works at least. Once this runs, back in my main yaml file I call this.
- job: job_deploy_code
displayName: Deploy Some Code
dependsOn: armTemplates
variables:
myVarFromDeploymentJob: $[ dependencies.armTemplates.outputs['deploy_Development.setvarStep.myOutputVar'] ]
steps:
- script: "echo $(myVarFromDeploymentJob)"
name: echovar
Which is just supposed to echo back what is in my variable. Right now it's blank. I know it's some stupid syntax that I'm not seeing but I can't for the life of me figure out what it is.
I solved this with the correct syntax
- job: job_deploy_code
displayName: Deploy Some Code
dependsOn: armTemplates
variables:
myVarFromDeploymentJob: $[ dependencies.armTemplates.outputs['armTemplates.setvarStep.myOutputVar'] ]

Azure DevOps Use a Variable as a Condition between Deployment Stages

I currently build all our Azure Infrastructure using Terraform via Azure DevOps Pipelines. This has been working well and we have a standard pipeline which calls two templates
Plan Stage
Deployment Job
Planning Template
Runs Every Time
Apply Stage
Deployment Job
Apply Template
Runs with manual approval check
Now this works fine, but what I want to do is only have the apply step run if there are changes to make. I have found other articles on how to get a variable set in the plan stage which I can do and it works fine.
I can call this same variable in the next step
variables:
varFromPlanStage: $[stageDependencies.Plan.planning_stage.outputs['planning_stage.terraformPlanResult.terraformChanges']]
steps:
- script: echo $(varFromPlanStage)
But the problem comes in when I try and use this same variable in a condition.
I found that the way you all it is different, needing a dependency instead of stagedependancy, but no matter what I try I can't get it to work.
The pipeline looks like this.
stages:
- stage: 'Plan'
displayName: 'Planning'
jobs:
- deployment: planning_stage
displayName: 'Planning Changes'
pool:
vmImage: 'Ubuntu-20.04'
environment: 'planning'
strategy:
runOnce:
deploy:
steps:
- template: /Pipelines/10-TEST-terraform-planning-template.yml # Run the Planning Template
parameters:
terraform_version: ${{ parameters.terraform_version }}
terraform_backend_service_arm: ${{ parameters.terraform_backend_service_arm }}
terraform_backend_resource_group: ${{ parameters.terraform_backend_resource_group }}
terraform_backend_storage_account: ${{ parameters.terraform_backend_storage_account }}
terraform_backend_storage_container: ${{ parameters.terraform_backend_storage_container }}
terraform_state_key: ${{ parameters.terraform_state_key }}
git_working_directory: ${{ parameters.git_working_directory }}
# This is the Build Stage - Only do this when on the master branch (which is via a PR)
- stage: 'Apply'
condition: and(succeeded(), eq(dependencies.Plan.planning_stage.outputs['planning_stage.terraformPlanResult.terraformChanges'], 'true'))
variables:
varFromPlanStage: $[stageDependencies.Plan.planning_stage.outputs['planning_stage.terraformPlanResult.terraformChanges']]
displayName: 'Applying Changes'
jobs:
- deployment: applying_stage
displayName: 'Lets Build'
pool:
vmImage: 'Ubuntu-20.04'
environment: 'building'
strategy:
runOnce:
deploy:
steps:
- script: echo $(varFromPlanStage) # Just a test
- template: /Pipelines/20-TEST-terraform-apply-template.yml # Run the Apply Template
parameters:
terraform_version: ${{ parameters.terraform_version }}
terraform_backend_service_arm: ${{ parameters.terraform_backend_service_arm }}
terraform_backend_resource_group: ${{ parameters.terraform_backend_resource_group }}
terraform_backend_storage_account: ${{ parameters.terraform_backend_storage_account }}
terraform_backend_storage_container: ${{ parameters.terraform_backend_storage_container }}
terraform_state_key: ${{ parameters.terraform_state_key }}
git_working_directory: ${{ parameters.git_working_directory }}
And the part of the Planning Template that exports the variable is called terraformPlanResult with the var being terraformChanges
Any idea what I am doing wrong here, and why I can't call the variable as a condition but I can as part of the steps?
Thx!
This appears to work differently depending on whether you're setting the variable in a 'deployment' job, or a 'job' job. After some trial, error and googling I managed to get working for both. Examples below :-)
Example passing variables from a 'deployment' job.
# Example passing variables from a 'deployment' job.
stages:
# Create some variables to pass to next stage.
- stage: 'A'
jobs:
- deployment: 'A1'
pool:
vmImage: 'windows-2019'
environment: 'test'
strategy:
runOnce:
deploy:
steps:
# Create a variable.
- task: PowerShell#2
name: foo
displayName: 'Create a variable.'
inputs:
targetType: 'inline'
script: |
echo "##vso[task.setvariable variable=bar;isOutput=true]apple"
# Check variable.
- task: PowerShell#2
displayName: 'Check a variable.'
inputs:
targetType: 'inline'
script: |
Write-Host "$env:MY_MAPPED_ENV_VAR"
env:
MY_MAPPED_ENV_VAR: $(foo.bar)
# Confirm condition works & variables are available for use.
- stage: 'B'
dependsOn:
- 'A'
variables:
- name: varFromStageA
# stageDependencies.stageName.deploymentName.outputs['deploymentName.stepName.variableName']
value: $[ stageDependencies.A.A1.outputs['A1.foo.bar'] ]
# dependencies.stageName.outputs['deploymentName.deploymentName.stepName.variableName']
condition: and(succeeded(), eq(dependencies.A.outputs['A1.A1.foo.bar'], 'apple'))
jobs:
- job: 'B1'
pool:
vmImage: 'windows-2019'
steps:
# Confirm variable has been passed between stages.
- task: PowerShell#2
displayName: 'Confirm var passed between stages'
inputs:
targetType: 'inline'
script: |
Write-Host "$env:MY_MAPPED_ENV_VAR"
env:
MY_MAPPED_ENV_VAR: $(varFromStageA)
Example passing variables from a 'job' job.
# Example passing variables from a 'job' job.
stages:
# Create some variables to pass to next stage.
- stage: 'A'
jobs:
- job: 'A1'
pool:
vmImage: 'windows-2019'
steps:
# Create a variable.
- task: PowerShell#2
name: foo
displayName: 'Create a variable.'
inputs:
targetType: 'inline'
script: |
echo "##vso[task.setvariable variable=bar;isOutput=true]apple"
# Check variable.
- task: PowerShell#2
displayName: 'Check a variable.'
inputs:
targetType: 'inline'
script: |
Write-Host "$env:MY_MAPPED_ENV_VAR"
env:
MY_MAPPED_ENV_VAR: $(foo.bar)
# Confirm condition works & variables are available for use.
- stage: 'B'
dependsOn:
- 'A'
variables:
- name: varFromStageA
# stageDependencies.stageName.jobName.outputs['stepName.variableName']
value: $[ stageDependencies.A.A1.outputs['foo.bar'] ]
# dependencies.stageName.outputs['jobName.stepName.variableName']
condition: and(succeeded(), eq(dependencies.A.outputs['A1.foo.bar'], 'apple'))
jobs:
- job: 'B1'
pool:
vmImage: 'windows-2019'
steps:
# Confirm variable has been passed between stages.
- task: PowerShell#2
displayName: 'Confirm var passed between stages'
inputs:
targetType: 'inline'
script: |
Write-Host "$env:MY_MAPPED_ENV_VAR"
env:
MY_MAPPED_ENV_VAR: $(varFromStageA)
What's working for me (by many trials and errors) is the format:
dependencies.stage_name.outputs['job_name.step_name.variable_name']
In your case, this would be:
dependencies.Plan.outputs['planning_stage.terraformPlanResult.terraformChanges']
If that doesn't work, the only unusual thing is that your planning_stage is a deployment. You might try and change it to a regular job.
Here's a pipeline that tests various ways of referencing outputs in conditions.
For me it looks like an issue on Azure Devops (later O will preapre minimial working example and create a bug for this). Because the same syntax works correctly for regular jobs but not for deployment. Also you can use condition like condition: in(dependencies.A.result, 'Succeeded', 'SucceededWithIssues', 'Skipped') but not outputs. And according to this
"dependencies": {
"<STAGE_NAME>" : {
"result": "Succeeded|SucceededWithIssues|Skipped|Failed|Canceled",
"outputs": {
"jobName.stepName.variableName": "value"
}
},
"...": {
// another stage
}
}
it is correct syntax.
I also checked this but using life cycle hook instead of job name doesn't helped me.
What is trange that you can use output variables in job condition if ther are in the same stage.
Here is link to the issue.

Is there a way to set a default value for an array of objects in a yaml template (Azure Pipelines)

I have a yaml template that contains parameters like below. In the array / list I want to be able to set the default value for one of the properties (the object has 3). The reason being is I have some if statements that check these properties and I want them to run regardless of whether or not the property is set in the yaml that uses the template (the third property was added later and I don't want to have to update every repo that uses this template).
Is this something that can be done in the parameter setup?
Notes: Using the below example if the property WebProject is not used I would like everything to go through the template as if it was set to false. I know there is some duplication where I check the property and copy files / publish the artifacts but at the moment I want to get it working and make it more efficient afterwards.
yaml template
parameters:
- name: ArtifactPublish
type: boolean
default: false
- name: Solution
type: string
default: '**/*.sln'
- name: Artifacts
type: object
default: []
jobs:
- job: Build
displayName: 'Build, Pack, and Publish'
pool:
vmImage: 'windows-latest'
variables:
solution: ${{ parameters.Solution }}
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
- task: VSBuild#1
displayName: 'Build Solution'
inputs:
solution: '$(solution)'
msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(Build.ArtifactStagingDirectory)"'
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
- ${{ if eq(parameters.ArtifactPublish, true) }}:
- ${{ each artifact in parameters.Artifacts }}:
- ${{ if eq(artifact.WebProject, false) }}:
- task: CopyFiles#2
displayName: 'Copy .artifactignore: ${{ artifact.ArtifactPath }}'
inputs:
SourceFolder: '$(Build.SourcesDirectory)'
Contents: '.artifactignore'
TargetFolder: '$(Build.SourcesDirectory)/${{ artifact.ArtifactPath }}'
- task: PublishPipelineArtifact#1
displayName: 'Publish Artifact: ${{ artifact.ArtifactName }}'
inputs:
targetPath: '$(Build.SourcesDirectory)/${{ artifact.ArtifactPath }}'
artifactName: '${{ artifact.ArtifactName }}'
- ${{ if eq(artifact.WebProject, true) }}:
- task: CopyFiles#2
displayName: '${{ artifact.ArtifactPath }}*'
inputs:
SourceFolder: $(Build.ArtifactStagingDirectory)
Contents: '${{ artifact.ArtifactPath }}*'
TargetFolder: '$(Build.ArtifactStagingDirectory)/${{ artifact.ArtifactPath }}'
- task: PublishPipelineArtifact#1
displayName: 'Publish Artifact: ${{ artifact.ArtifactName }} (WEB)'
inputs:
targetPath: $(Build.ArtifactStagingDirectory)/${{ artifact.ArtifactPath }}
artifactName: '${{ artifact.ArtifactName }}'
Example yaml that uses the template:
trigger:
- release
- development
- master
- feature/*
- task/*
resources:
repositories:
- repository: templates
name: Project/RepoName
type: git
ref: refs/heads/release
jobs:
- template: Templates/example.yml#templates
parameters:
ArtifactPublish: true
Artifacts:
- ArtifactPath: 'example/path'
ArtifactName: 'exampleName'
- ArtifactPath: 'example/path'
ArtifactName: 'exampleName'
WebProject: true
I am afraid setting the default value for one of the properties cannot be done in the parameters setup.
But you can use the coalesce expression to check if the WebProject parameter is set or not:
Evaluates the parameters in order, and returns the value that does not equal null or empty-string.
Min parameters:
Max parameters: N
Example: coalesce(variables.couldBeNull, variables.couldAlsoBeNull, 'literal so it always works')
So you can use the expression coalesce(artifact.WebProject, false) in your template yaml as below:
- ${{ if eq(parameters.ArtifactPublish, true) }}:
- ${{ each artifact in parameters.Artifacts }}:
- ${{ if eq(coalesce(artifact.WebProject, false),false) }}:
With this expression - ${{ if eq(coalesce(artifact.WebProject, false),false) }}, No matter whether parameter WebProject is set to false or not used, this expression will always be true.

Resources