Debugging yaml pipeline script - azure

I'm trying to debug a pipeline script in Azure to see what the variables are, but I'm not sure of the syntax.
I have looked through the documentation
What I have tried so far
parameters:
deploymentName: ""
dependsOn: ""
env: ""
dockerfilePath: ""
buildContext: ""
repository: ""
envGroup: ""
dockerRegistryServiceConnection: ""
tag: ""
token: ""
runTests: ""
jobs:
- deployment: ${{ parameters.deploymentName }}
dependsOn: ${{ parameters.dependsOn }}
pool: "Private Docker"
environment: "${{ parameters.envGroup }}"
strategy:
runOnce:
deploy:
steps:
- checkout: self
- task: Docker#2
displayName: "ACR Login"
inputs:
command: login
containerRegistry: "${{ parameters.dockerRegistryServiceConnection }}"
##[debug]Debug!!!!!1 $runTests
##[debug]Debug!!!!!2 $parameters.runTests
##[debug]Debug!!!!!3 ${{parameters.runTests}}
- script: echo runTests in dockerbuild.yml ${{parameters.runTests}}
- script: echo runTests in dockerbuild.yml $parameters.runTests
- script: echo runTests in dockerbuild.yml $runTests
- task: Docker#2
displayName: Build
inputs:
command: build
repository: "${{ parameters.repository }}"
buildContext: "${{ parameters.buildContext }}"
dockerfile: "${{ parameters.dockerfilePath }}"
arguments: "--build-arg TOKEN=${{ parameters.token }} --build-arg RUNTESTS=${{ parameters.runTests }}"
tags: |
$(Build.BuildId)
latest
Problem
The debug and echo do not print anything in the azure pipeline build:
##[debug]Debug!!!!!1 $runTests
##[debug]Debug!!!!!2 $parameters.runTests
##[debug]Debug!!!!!3 ${{parameters.runTests}}
- script: echo runTests in dockerbuild.yml ${{parameters.runTests}}
- script: echo runTests in dockerbuild.yml $parameters.runTests
- script: echo runTests in dockerbuild.yml $runTests
Question
Is this the correct syntax for printing to the azure bash when running the yaml file?

Debugging yaml pipeline script
It seems that you did not define the parameters correctly. We need provide the name, displayName, type, default, values like following (Not every attribute is required):
parameters:
- name: deploymentName
type: string
default: Product
Then we could use the ${{ parameters.deploymentName }} to get the value.
Please check this document Runtime parameters for some more details.
My test YAML file:
parameters:
- name: deploymentName
type: string
default: Product
- name: dependsOn
type: string
default: Test
- name: envGroup
type: string
default: test
- name: runTests
type: number
default: 123
stages:
- stage: Build
jobs:
- job: Build
displayName: Build
pool:
name: MyPrivateAgent
steps:
- script: echo This is for Build
- stage: Dev
jobs:
- job: Test
displayName: Test
pool:
name: MyPrivateAgent
steps:
- script: echo This is for test
- deployment: ${{ parameters.deploymentName }}
dependsOn: ${{ parameters.dependsOn }}
pool:
name: MyPrivateAgent
environment: "${{ parameters.envGroup }}"
strategy:
runOnce:
deploy:
steps:
- checkout: self
- script: echo runTests in dockerbuild.yml ${{parameters.runTests}}
The test result:

Try this:
${{ parameters.runTests}}

Related

Publish file content to service bus from CI pipeline

In my CI pipeline I am trying to publish message to service bus and its working when its just some hardcoded text or variables, here Using "PublishToAzureServiceBus" task .
But problem is when trying to a read file from repository and then publish that to service bus.
I have tried using read file using scripting language and put to variable but its not able to work as variable is not storing big json file.
Is there any way to read file directly when publishing message to service bus.
Below is sample code snippet for debugging
trigger:
- none
pool:
vmImage: ubuntu-latest
parameters:
- name: ProjectName
displayName: Project Name
type: string
default: DevOpsDemo
- name: repoName
displayName: repo Name
type: string
default: ProjectCode
- name: branchRef
displayName: Branch Name
type: string
default: main
variables:
- name: jobStatus
value: "Failed"
- name: projectFile
value: ""
stages:
- stage: Stage1
displayName: Stage 1
jobs:
- job: CheckOutRepo
displayName: CheckOut-Repo Display
steps:
- script: |
echo "Checkout for " ${{ parameters.ProjectName}} : ${{ parameters.repoName}} : ${{ parameters.branchRef}}
name: PrintMessage
- checkout: git://${{ parameters.ProjectName}}/${{ parameters.repoName}}#refs/heads/${{ parameters.branchRef}}
name: Checkout
- task: PythonScript#0
inputs:
scriptSource: 'inline'
script: |
import json
import requests
f = open('project-release.json')
projectFile = json.load(f)
print(projectFile)
f.close()
print("Afterclosing")
print(projectFile)
- script: |
echo "Project release file" $(cat project-release.json)
name: TestPrint
- task: CopyFiles#2
inputs:
SourceFolder: 'services'
Contents: '**'
TargetFolder: $(Build.ArtifactStagingDirectory)
name: CopyFiles
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: $(Build.ArtifactStagingDirectory)
ArtifactName: 'drop'
publishLocation: 'Container'
name: PublishArtifacts
- bash: |
echo "##vso[task.setvariable variable=jobStatus]Success"
name: setVar
- bash: |
echo "##vso[task.setvariable variable=jobStatus;isOutput=true]$(jobStatus)"
echo "##vso[task.setvariable variable=projectFile;isOutput=true]$(cat project-release.json)"
name: SetStatus
condition: always()
- stage: Stage2
displayName: Stage 2
condition: always()
jobs:
- job: Publish
pool: server
variables:
jobStatus: $[ stageDependencies.Stage1.CheckOutRepo.outputs['SetStatus.jobStatus'] ]
projectFile: $[ stageDependencies.Stage1.CheckOutRepo.outputs['SetStatus.projectFile'] ]
steps:
- task: PublishToAzureServiceBus#1
inputs:
azureSubscription: 'SBConnection'
messageBody: |
{
"Status": "$(jobStatus)",
"BuildID": "$(build.buildid)",
"BuildNumber":"$(build.buildnumber)",
"projectFile":$(cat project-release.json)
}
signPayload: false
waitForCompletion: false
condition: always()
I am able to solve this by using setvariable in bash script as below
pool:
vmImage: ubuntu-latest
stages:
- stage: Stage1
displayName: Stage 1
jobs:
- job: CheckOutRepo
displayName: CheckOut-Repo Display
steps:
- checkout: git://${{ parameters.ProjectName}}/${{ parameters.repoName}}#refs/heads/${{ parameters.branchRef}}
name: Checkout
- bash: |
data=$(cat project-release.json)
echo "##vso[task.setvariable variable=jobStatus;isOutput=true]$(jobStatus)"
echo "##vso[task.setvariable variable=data;isOutput=true]"$data
name: SetStatus
condition: always()
- stage: Stage2
displayName: Stage 2
condition: always()
jobs:
- job: Publish
pool: server
variables:
jobStatus: $[ stageDependencies.Stage1.CheckOutRepo.outputs['SetStatus.jobStatus'] ]
projectFile: $[ stageDependencies.Stage1.CheckOutRepo.outputs['SetStatus.data'] ]
steps:
- task: PublishToAzureServiceBus#1
inputs:
azureSubscription: 'SBConnection'
messageBody: |
{
"Status": "$(jobStatus)",
"BuildID": "$(build.buildid)",
"BuildNumber":"$(build.buildnumber)",
"projectFile":$(projectFile)
}
signPayload: false
waitForCompletion: false
condition: always()

Aure pipeline : use output variable in multi-stage parametrised

I am facing an issue I could not resolve.
regarding this documentation it seams to be possible to set a variable as output of a job in the first stage, then use it as input to set a job variable in second stage.
The pbm I'm facing, is that my stages are parametrised, so their name are not constant.
here my sample code :
name: mypipeline
parameters:
- name: Application
displayName: Application to deploy the front end using front door
type: string
- name: stage_vars
displayName: stage VARS
type: object
default:
- stage_1
- stage_2
- none
trigger: none
stages:
- stage: Build
displayName: ${{ parameters.Application }} - Build.
pool:
name: Azure Pipelines
vmImage: windows-2019
jobs:
- template: ../templates/job_BuildAndPublish.yml
parameters:
BuildId: $(Build.BuildId)
importTerratest: false
importARM: true
- ${{ each stageregion in parameters.stage_vars }}:
- ${{ if ne(stageregion, 'none') }}:
- template: ../templates/isolated_web/tf_plan_stage.yml
parameters:
BuildId: $(system.BuildId)
Predecessors: 'Build'
- template: ../templates/isolated_web/tf_apply_stage.yml
parameters:
Predecessors: '${{stageregion}}_prepare'
build template is not usefull, but the ones from tf_plan_stage and tf_apply_stage are below :
tf_plan_stage.yml :
parameters:
- name: BuildId
type: string
default: $(system.BuildId)
- name: stage
type: string
default: Deploy_prepare
- name: Predecessors
type: string
default: none
stages:
- stage: ${{ parameters.stage }}
? ${{ if and(ne(parameters.Predecessors, 'previous'), ne(parameters.Predecessors, 'none') ) }}
: dependsOn: ${{parameters.Predecessors}}
displayName: ${{ parameters.TF_VAR_STAGE }} Terraform plan & publish artifact for ${{ parameters.TF_VAR_APPLICATION }}
pool:
name: Azure Pipelines
vmImage: windows-2019
jobs:
- deployment: PreDeployTerraform
displayName: ${{ parameters.TF_VAR_STAGE }} Terraform plan & publish artifact
environment: PLAN_${{ parameters.TF_VAR_ENVIRONMENT }}
timeoutInMinutes: 480
strategy:
runOnce:
deploy:
steps:
- checkout: none
# set output planAttempt output available for later Apply job
- powershell: |
echo "jobAttempt is $(System.JobAttempt)"
echo "##vso[task.setvariable variable=planAttempt;isOutput=true;]$(System.JobAttempt)"
name: setVarJobAttempt
- powershell: |
echo "variable setVarJobAttempt.planAttempt value is : $(setVarJobAttempt.planAttempt)"
name: getplanAttemptVar
tf_apply_stage.yml :
parameters:
- name: stage
type: string
default: Deploy_prepare
- name: Predecessors
type: string
default: none
stages:
- stage: ${{ parameters.stage }}
? ${{ if and(ne(parameters.Predecessors, 'previous'), ne(parameters.Predecessors, 'none') ) }}
: dependsOn: ${{parameters.Predecessors}}
displayName: ${{ parameters.TF_VAR_STAGE }} download artifact & Terraform apply changes to ${{ parameters.TF_VAR_APPLICATION }}
pool:
name: Azure Pipelines
vmImage: windows-2019
jobs:
- deployment: DeployTerraform
displayName: ${{ parameters.TF_VAR_STAGE }} download artifact & Terraform apply changes
variables:
# this one fails in pipeline : "[not recognise"
planJobAttempt: $[ stageDependencies.[parameters.Predecessors].PreDeployTerraform.outputs['setVarJobAttempt.planAttempt'] ]
# this one runs with no result in pipeline : planJobAttempt is the string "stageDependencies['parameters.Predecessors'].PreDeployTerraform.outputs['setVarJobAttempt.planAttempt']"
# planJobAttempt: $[ stageDependencies['parameters.Predecessors'].PreDeployTerraform.outputs['setVarJobAttempt.planAttempt'] ]
environment: ${{ parameters.TF_VAR_ENVIRONMENT }}
timeoutInMinutes: 480
strategy:
runOnce:
deploy:
steps:
- checkout: none
- powershell: |
echo "jobAttempts outputs are : $[ stageDependencies.[parameters.Predecessors].PreDeployTerraform.outputs ]"
- powershell: |
echo "jobAttempt is $[ stageDependencies['parameters.Predecessors'].PreDeployTerraform.outputs['setVarJobAttempt.planAttempt'] ]"
- powershell: |
echo "plan file for terraform is : $(pipeline_artifact_folder_download)/$(planJobAttempt)_${{ parameters.Predecessors }}/$(artefact_terraform_plan)_${{ parameters.TF_VAR_STAGE }}"
I tried differents things to get my system.jobAttempts from tf_plan_stage stage to tf_apply_stage stage, but without any success, as the variable in the last stage (tf_apply) seams unable to find the value from a "parametrised" stage name.
is there a way for that ?
thank-you for any answers.
Okay,
Once again, I will answer my own question :D
I've finallt find the way to use the parameters.Predecessors value in my variable declaration.
And, second, the Microsoft documentation seams not to be updated.
The tf_apply_stage.yml should be like that below ;
parameters:
- name: stage
type: string
default: Deploy_prepare
- name: Predecessors
type: string
default: none
stages:
- stage: ${{ parameters.stage }}
? ${{ if and(ne(parameters.Predecessors, 'previous'), ne(parameters.Predecessors, 'none') ) }}
: dependsOn: ${{parameters.Predecessors}}
displayName: ${{ parameters.TF_VAR_STAGE }} download artifact & Terraform apply changes to ${{ parameters.TF_VAR_APPLICATION }}
pool:
name: Azure Pipelines
vmImage: windows-2019
jobs:
- deployment: DeployTerraform
displayName: ${{ parameters.TF_VAR_STAGE }} download artifact & Terraform apply changes
variables:
*# this one give he good value for Variable*
planJobAttempt: $[ stageDependencies.${{ parameters.Predecessors }}.PreDeployTerraform.outputs['PreDeployTerraform.setVarJobAttempt.planAttempt'] ]
environment: ${{ parameters.TF_VAR_ENVIRONMENT }}
timeoutInMinutes: 480
strategy:
runOnce:
deploy:
steps:
- checkout: none
- powershell: |
echo "plan file for terraform is : $(pipeline_artifact_folder_download)/$(planJobAttempt)_${{ parameters.Predecessors }}/$(artefact_terraform_plan)_${{ parameters.TF_VAR_STAGE }}"
I put in BOLD the main changes above :
planJobAttempt: $[ stageDependencies.${{ parameters.Predecessors
}}.PreDeployTerraform.outputs['PreDeployTerraform.setVarJobAttempt.planAttempt']
]
The Job name have to be set twice in variable declaration !
Thanks for those who read my post ;)
And many thanks to this post from #Kay07949 who gives the good answers ;)

How do I reference a different Variable group per stage - stage per environment in environments - Azure Devops, template, variable groups

When I run the pipeline below I get an error within jobs.yml - it commplains about an "unexpected value" for the environment parameter... and within stages.yml "unexpected value: parameters" - what am I doing wrong here? The idea is to get the environments and the associated variableGroup - loop thru the environments array in Stages.yml and create a stage per environment... insert correct variableGroup per stage... use variablesGroup values to perform jobs in jobs.yml - each variable group contains the same vars.. with different values.
This is main.yml
#main.yml
trigger: none
extends:
template: /Build/Stages.yml
parameters:
environments:
- environment: Dev
variableGroup: Project-Dev-VarGrp
- environment: QA
variableGroup: Project-QA-VarGrp
dependsOn: Dev
- environment: UAT
variableGroup: Project-UAT-VarGrp
dependsOn: QA
- environment: UAT2
variableGroup: Project-UAT2-VarGrp
dependsOn: UAT
Then here is the next bit... Stages.yml
parameters:
- name: dataFolder
displayName: 'Data folder to process'
type: string
default: '/DataMigrations/Master_Data/'
- name: dataFiles
displayName: List of Data Files or Folder names
type: string
default: 'Dev_Data.zip'
- name: environments
type: object
default: []
stages:
- ${{ each environment in parameters.environments }}:
- stage: '${{ parameters.environment }}'
jobs:
- template: ./jobs.yml
variables:
- group: ${{ parameters.variableGroup }}
parameters:
environment: '${{ parameters.environment }}'
crmURL: $(crmURL)
oauthAppId: $(ClientID)
ClientPass: $(ClientPass)
dataFolder: '${{ parameters.dataFolder }}'
env: '${{ parameters.environment }}'
and here is jobs.yml
jobs:
- deployment: DeployData
displayName: 'Deploying Data to ${{ parameters.environment }}'
environment: ${{ parameters.environment }}
pool:
vmImage: 'windows-latest'
strategy:
runOnce:
deploy:
steps:
- checkout: self
clean: false
- powershell: |
Write-Host "##vso[task.setvariable variable=crmConnectionString]'AuthType=ClientSecret;Url=$(crmURL);ClientId=$(ClientID);ClientSecret=$(ClientPass)'"
displayName: 'PreDeploy configuration'
- task: PowerShell#2
displayName: 'Powershell: Run update-data.ps1'
inputs:
targetType: 'filePath'
filePath: $(System.DefaultWorkingDirectory)/DataMigrations/Scripts/update-data.ps1
arguments: -folderName '${{ parameters.dataFolder }}' -environment '${{ parameters.env }}'
workingDirectory: $(System.DefaultWorkingDirectory)/DataMigrations
- task: PowerShell#2
displayName: 'Powershell: Run zip-import-data.ps1'
inputs:
targetType: 'filePath'
filePath: $(System.DefaultWorkingDirectory)/DataMigrations/Scripts/zip-import-data.ps1
arguments: -folderName '${{ parameters.dataFolder }}' -dataMigrationFilenames '${{ parameters.dataFiles }}' -connectionString $(crmConnectionString)
workingDirectory: $(System.DefaultWorkingDirectory)/DataMigrations
Few things here:
When you're using ${{ each environment in parameters.environments }} then the nested environment and variableGroup are available using this syntax ${{ environment.environment }} and ${{ environment.variableGroup}}
In your Stages.yml file, you're trying to invoke the ./jobs.yml template but the associated parameters are defined after the - group: ${{ parameters.variableGroup }}. A valid syntax should looks like this:
stages:
- ${{ each environment in parameters.environments }}:
- stage: '${{ environment.environment }}'
variables:
- group: ${{ environment.variableGroup }}
jobs:
- template: ./jobs.yml
parameters:
environment: '${{ environment.environment }}'
crmURL: $(crmURL)
oauthAppId: $(ClientID)
ClientPass: $(ClientPass)
dataFolder: '${{ parameters.dataFolder }}'
env: '${{ environment.environment }}'
You also have few space typos. I know it's annoying but you need to have the exact yaml syntax otherwise the files can't be parsed.

Conditional Stage Execution in Azure DevOps Pipelines

I want a stage in an Azure DevOps pipeline to be executed depending on the content of a variable set in a previous stage.
Here is my pipeline:
stages:
- stage: plan_dev
jobs:
- job: terraform_plan_dev
steps:
- bash: echo '##vso[task.setvariable variable=terraform_plan_exitcode;isOutput=true]2'
name: terraform_plan
- stage: apply_dev
dependsOn: plan_dev
condition: eq(stageDependencies.plan_dev.terraform_plan_dev.outputs['terraform_plan.terraform_plan_exitcode'], '2')
jobs:
- deployment: "apply_dev"
...
The idea is to skip the apply_dev stage, if the plan_dev stage shows no changes. Background is that we have manual approval for the deployment in the plan_dev stage that we want to skip if there are no changes to be approved.
Unfortunately this doesn't seem to work. No matter whether the variable terraform_plan_exitcode is set with the expected value (2) or not, the apply_dev stage is skipped.
For the syntax, I followed the documentation here that says:
stageDependencies.StageName.JobName.outputs['StepName.VariableName']
I have seen this same issue. You need to use the dependencies variable instead of the stageDependencies:
stages:
- stage: plan_dev
jobs:
- job: terraform_plan_dev
steps:
- bash: echo '##vso[task.setvariable variable=terraform_plan_exitcode;isOutput=true]2'
name: terraform_plan
- stage: apply_dev
dependsOn: plan_dev
condition: eq(dependencies.plan_dev.outputs['terraform_plan_dev.terraform_plan.terraform_plan_exitcode'], '2')
jobs:
- deployment: "apply_dev"
The following is a more complete example of something I have working with Terraform Plan + conditional Apply:
stages:
- stage: Build_zip_plan
displayName: Build portal, zip files and terraform plan
jobs:
- job: Build_portal_zip_files_terraform_plan
pool:
vmImage: 'ubuntu-latest'
steps:
- task: Cache#2
displayName: 'Register TF cache'
inputs:
key: terraform | $(Agent.OS) | $(Build.BuildNumber) | $(Build.BuildId) | $(Build.SourceVersion) | $(prefix)
path: ${{ parameters.tfExecutionDir }}
- task: TerraformInstaller#0
displayName: 'Install Terraform'
inputs:
terraformVersion: ${{ parameters.tfVersion }}
- task: TerraformTaskV1#0
displayName: 'Terraform Init'
inputs:
provider: 'azurerm'
command: 'init'
workingDirectory: ${{ parameters.tfExecutionDir }}
backendServiceArm: ${{ parameters.tfStateServiceConnection }}
backendAzureRmResourceGroupName: ${{ parameters.tfStateResourceGroup }}
backendAzureRmStorageAccountName: ${{ parameters.tfStateStorageAccount }}
backendAzureRmContainerName: ${{ parameters.tfStateStorageContainer }}
backendAzureRmKey: '$(prefix)-$(environment).tfstate'
- task: TerraformTaskV1#0
displayName: 'Terraform Plan'
inputs:
provider: 'azurerm'
command: 'plan'
commandOptions: '-input=false -out=deployment.tfplan -var="environment=$(environment)" -var="prefix=$(prefix)" -var="tenant=$(tenant)" -var="servicenow={username=\"$(servicenowusername)\",instance=\"$(servicenowinstance)\",password=\"$(servicenowpassword)\",assignmentgroup=\"$(servicenowassignmentgroup)\",company=\"$(servicenowcompany)\"}" -var="clientid=$(clientid)" -var="username=$(username)" -var="password=$(password)" -var="clientsecret=$(clientsecret)" -var="mcasapitoken=$(mcasapitoken)" -var="portaltenantid=$(portaltenantid)" -var="portalclientid=$(portalclientid)" -var="customerdisplayname=$(customerdisplayname)" -var="reportonlymode=$(reportonlymode)"'
workingDirectory: ${{ parameters.tfExecutionDir }}
environmentServiceNameAzureRM: ${{ parameters.tfServiceConnection }}
- task: PowerShell#2
displayName: 'Check Terraform plan'
name: "Check_Terraform_Plan"
inputs:
filePath: '$(Build.SourcesDirectory)/Pipelines/Invoke-CheckTerraformPlan.ps1'
arguments: '-TfPlan ''${{ parameters.tfExecutionDir }}/deployment.tfplan'''
pwsh: true
- stage:
dependsOn: Build_zip_plan
displayName: Terraform apply
condition: eq(dependencies.Build_zip_plan.outputs['Build_portal_zip_files_terraform_plan.Check_Terraform_Plan.TFChangesPending'], 'yes')
jobs:
- deployment: DeployHub
displayName: Apply
pool:
vmImage: 'ubuntu-latest'
environment: '$(prefix)'
strategy:
runOnce:
deploy:
steps:
- checkout: self
- task: Cache#2
displayName: 'Get Cache for TF Artifact'
inputs:
key: terraform | $(Agent.OS) | $(Build.BuildNumber) | $(Build.BuildId) | $(Build.SourceVersion) | $(prefix)
path: ${{ parameters.tfExecutionDir }}
- task: TerraformInstaller#0
displayName: 'Install Terraform'
inputs:
terraformVersion: ${{ parameters.tfVersion }}
- task: TerraformTaskV1#0
displayName: 'Terraform Apply'
inputs:
provider: 'azurerm'
command: 'apply'
commandOptions: 'deployment.tfplan'
workingDirectory: ${{ parameters.tfExecutionDir }}
environmentServiceNameAzureRM: ${{ parameters.tfServiceConnection }}
#Marius is correct. So this works
stages:
- stage: plan_dev
jobs:
- job: terraform_plan_dev
steps:
- bash: echo '##vso[task.setvariable variable=terraform_plan_exitcode;isOutput=true]2'
name: terraform_plan
- stage: apply_dev
dependsOn: plan_dev
variables:
varFromA: $[ stageDependencies.plan_dev.terraform_plan_dev.outputs['terraform_plan.terraform_plan_exitcode'] ]
condition: eq(dependencies.plan_dev.outputs['terraform_plan_dev.terraform_plan.terraform_plan_exitcode'], 2)
jobs:
- job: apply_dev
steps:
- bash: echo 'apply $(varFromA)'
name: terraform_apply
When you refer stage to stage dependencies you have different syntax
"dependencies": {
"<STAGE_NAME>" : {
"result": "Succeeded|SucceededWithIssues|Skipped|Failed|Canceled",
"outputs": {
"jobName.stepName.variableName": "value"
}
},
"...": {
// another stage
}
}
And when you refer to job to job across stage you have different syntax
"stageDependencies": {
"<STAGE_NAME>" : {
"<JOB_NAME>": {
"result": "Succeeded|SucceededWithIssues|Skipped|Failed|Canceled",
"outputs": {
"stepName.variableName": "value"
}
},
"...": {
// another job
}
},
"...": {
// another stage
}
}
What is funny when you have job to job in one stage we use dependecies syntax again
"dependencies": {
"<JOB_NAME>": {
"result": "Succeeded|SucceededWithIssues|Skipped|Failed|Canceled",
"outputs": {
"stepName.variableName": "value1"
}
},
"...": {
// another job
}
}
This is a bit confusing and consider this in this as
when you are on some level stage, job and refer to the same level from job to job or from stage to stage you have dependencies syntax
when you want to refer from deeper level like from job to stage you should use stageDependencies
What is funny, in above example I used this on stage level:
variables:
varFromA: $[ stageDependencies.plan_dev.terraform_plan_dev.outputs['terraform_plan.terraform_plan_exitcode'] ]
but this is evaluated at runtime and is evaluated from the job, so it is correct and is evaluated correctly.
I hope it added a value to previous answer.
TerraformTaskV2 has changesPresent output variable now, which can be used to skip apply stage.
add name: to the plan task
stages:
- stage: terraform_plan_STAGE
jobs:
- job: plan_JOB
...
steps:
...
- task: TerraformTaskV2#2
name: 'plan_TASK' # <===========
displayName: 'plan'
inputs:
provider: 'azurerm'
command: 'plan'
...
add condition: to apply stage and check if changesPresent is true
- stage: terraform_apply
dependsOn: [terraform_plan]
condition: eq(dependencies.terraform_plan_STAGE.outputs['plan_JOB.plan_TASK.changesPresent'], 'true')
reference:
https://github.com/microsoft/azure-pipelines-terraform/tree/main/Tasks/TerraformTask/TerraformTaskV2#output-variables
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#use-outputs-in-a-different-stage

Unexpected Behavior With Azure Pipelines Variables Using Variable Groups and Templates

I have a Azure DevOps YAML Pipeline to execute a Terraform deployment using the Terraform by MS DevLabs extension and an Azure Resource Manager service connection.
The last working state was using a pipeline template yaml file however I had to configure a parameter within the template and call the variable using the template expression syntax.
...
...
stages:
- stage: Plan
displayName: Terrafom Plan
jobs:
- job: DEV PLAN
displayName: Plan (DEV)
pool:
vmImage: "ubuntu-latest"
variables:
az_service_connection: "MyServiceConnection"
tf_environment: "DEV"
tf_state_rg: "DEV"
tz_state_location: "canadacentral"
tf_state_stgacct_name: "mystorageaccuontname1231231"
tf_state_container_name: "tfstate"
steps:
- template: templates/terraform-plan.yml
parameters:
az_service_connection: ${{ variables.az_service_connection }}
...
...
steps:
- task: terraformInstaller#0
displayName: "Install Terraform $(tf_version)"
inputs:
terraformVersion: $(tf_version)
- task: TerraformTaskV1#0
displayName: "Run > terraform init"
inputs:
command: "init"
commandOptions: "-input=false"
backendServiceArm: ${{ parameters.az_service_connection }}
...
...
I believe the reason why this works is because the template expression syntax ${{ variables.varname}} evaluates at compile time vs. runtime. If I didn't do it this way, i'd either get $(az_service_connection) passed into the backendServiceArm input or an empty value.
With the introduction of variable groups, i'm now facing similar behavior. I expect that the variable group evaluates after the template expression variable which causes ${{ variables.az_service_connection }} to have an empty value. I am unsure how to get this working.
How can I use variable groups with a pipeline template that uses a service connection?
I used $() syntax to pass arm connection to template:
Template file:
parameters:
- name: 'instances'
type: object
default: {}
- name: 'server'
type: string
default: ''
- name: 'armConnection'
type: string
default: ''
steps:
- task: TerraformTaskV1#0
inputs:
provider: 'azurerm'
command: 'init'
backendServiceArm: '${{ parameters.armConnection }}'
backendAzureRmResourceGroupName: 'TheCodeManual'
backendAzureRmStorageAccountName: 'thecodemanual'
backendAzureRmContainerName: 'infra'
backendAzureRmKey: 'some-terrform'
- ${{ each instance in parameters.instances }}:
- script: echo ${{ parameters.server }}:${{ instance }}
Main file:
trigger:
branches:
include:
- master
paths:
include:
- stackoverflow/09-array-parameter-for-template/*
# no PR triggers
pr: none
pool:
vmImage: 'ubuntu-latest'
variables:
- group: my-variable-group
- name: my-passed-variable
value: $[variables.myhello] # uses runtime expression
steps:
- template: template.yaml
parameters:
instances:
- test1
- test2
server: $(myhello)
armConnection: $(armConnection)
Note: Group my-variable-group contains armConnection variable

Resources