I have on my azure pipelines a libary varrible named:
deployHostMachine
and in it a variable named:
Host
I need to run a task using if with condition on Host How do i do it?
My code:
parameters:
trigger:
- none
variables:
- group: DeploymentHost
- name: localForHost
value: $(Host)
- name: testEmpty
value: ''
stages:
- stage: "ManageDeployMachines"
jobs:
- job: "ManageDeployMachines"
pool:
vmImage: 'windows-latest'
steps:
- ${{ if eq(parameters.paramHost,'rnd-3.sepio.systems') }}:
- task: ManageVariableGroupTask#0
displayName: 'Set deploy host url'
inputs:
pat: 'yxtloxscphyvuevrzafozok533orcv3pqon4a7e3kzpc4je23zha'
project: '70e3803b-2a31-4f00-83a1-9fc825f4447b'
vgname: 'DeploymentHost'
orgname: 'SepioDevOfficial'
op: 'Update'
varops: '= Host => "rnd-4.sepio.systems"'
Related
I want to get the environments and the associated variableGroup - loop thru the environments array in Stages.yml and create a stage per environment... insert correct variableGroup per stage... use variablesGroup values to perform jobs in jobs.yml - each variable group contains the same vars.. with different values.
``
I have a main.yml file in which I reference my environments and the associated group variable
main.yaml :
name: $(BuildID)
trigger: none
pool:
vmImage: 'leetchi-agentspool-windows'
variables:
- template: vars/variables-api.yaml
extends:
template: deploy-stages.yaml
parameters:
environements:
- environement: 'DV1'
variableGroupName: tf-dv1
- environement: 'IN1'
variableGroupName: tf-in1
A stage template which is supposed to apply with the environment parameters and the group variable.
stage.yaml :
---
parameters:
- name: environements
type: object
default: []
stages:
- ${{ each environement in parameters.environements }}:
- stage: 'Deploy to ${{ environement.environements }}'
variables:
- group: ${{ environement.variableGroup }}
jobs:
- template: template-terraform-deployment.yml
parameters:
environement: ${{environement.environements}}
project: $(project)
definition: $(def)
project1: $(project1)
definition1: $(def1)
stateKey: ${{environement.environement}}$(tfkey)
TerraformDirectory: $(TfDirectory)
azureSubscription: $(service_connection_name)
TfStateResourceGroupName: $(tf_resource_group)
TfStateStorageAccountName: $(tf_storage_account)
TStateContainerName: $(tf_container)
commandOptions: -out=$(System.DefaultWorkingDirectory)/terraform.tfplan -detailed-exitcode
I start in YAML. Could you help me please? Thank you very much in advance
You can modify your main.yaml as below:
trigger: none
pool:
vmImage: 'leetchi-agentspool-windows'
variables:
- template: vars/variables-api.yaml
extends:
template: deploy-stages.yaml
parameters:
- name: environment
values:
- DV1
- IN1
- name: variableGroupName
values:
- tf-dv1
- tf-in1
stage.yaml :
stages:
- stage: 'Deploy to ${{ parameters.environment }}'
variables:
- group: ${{ parameters.variableGroupName}}
jobs:
- template: template-terraform-deployment.yml
parameters:
environment: ${{ parameters.environment }}
project: $(project)
definition: $(def)
project1: $(project1)
definition1: $(def1)
stateKey: ${{ parameters.environment }} + $(tfkey)
TerraformDirectory: $(TfDirectory)
azureSubscription: $(service_connection_name)
TfStateResourceGroupName: $(tf_resource_group)
TfStateStorageAccountName: $(tf_storage_account)
TStateContainerName: $(tf_container)
commandOptions: -out=$(System.DefaultWorkingDirectory)/terraform.tfplan -detailed-exitcode
I am facing an issue I could not resolve.
regarding this documentation it seams to be possible to set a variable as output of a job in the first stage, then use it as input to set a job variable in second stage.
The pbm I'm facing, is that my stages are parametrised, so their name are not constant.
here my sample code :
name: mypipeline
parameters:
- name: Application
displayName: Application to deploy the front end using front door
type: string
- name: stage_vars
displayName: stage VARS
type: object
default:
- stage_1
- stage_2
- none
trigger: none
stages:
- stage: Build
displayName: ${{ parameters.Application }} - Build.
pool:
name: Azure Pipelines
vmImage: windows-2019
jobs:
- template: ../templates/job_BuildAndPublish.yml
parameters:
BuildId: $(Build.BuildId)
importTerratest: false
importARM: true
- ${{ each stageregion in parameters.stage_vars }}:
- ${{ if ne(stageregion, 'none') }}:
- template: ../templates/isolated_web/tf_plan_stage.yml
parameters:
BuildId: $(system.BuildId)
Predecessors: 'Build'
- template: ../templates/isolated_web/tf_apply_stage.yml
parameters:
Predecessors: '${{stageregion}}_prepare'
build template is not usefull, but the ones from tf_plan_stage and tf_apply_stage are below :
tf_plan_stage.yml :
parameters:
- name: BuildId
type: string
default: $(system.BuildId)
- name: stage
type: string
default: Deploy_prepare
- name: Predecessors
type: string
default: none
stages:
- stage: ${{ parameters.stage }}
? ${{ if and(ne(parameters.Predecessors, 'previous'), ne(parameters.Predecessors, 'none') ) }}
: dependsOn: ${{parameters.Predecessors}}
displayName: ${{ parameters.TF_VAR_STAGE }} Terraform plan & publish artifact for ${{ parameters.TF_VAR_APPLICATION }}
pool:
name: Azure Pipelines
vmImage: windows-2019
jobs:
- deployment: PreDeployTerraform
displayName: ${{ parameters.TF_VAR_STAGE }} Terraform plan & publish artifact
environment: PLAN_${{ parameters.TF_VAR_ENVIRONMENT }}
timeoutInMinutes: 480
strategy:
runOnce:
deploy:
steps:
- checkout: none
# set output planAttempt output available for later Apply job
- powershell: |
echo "jobAttempt is $(System.JobAttempt)"
echo "##vso[task.setvariable variable=planAttempt;isOutput=true;]$(System.JobAttempt)"
name: setVarJobAttempt
- powershell: |
echo "variable setVarJobAttempt.planAttempt value is : $(setVarJobAttempt.planAttempt)"
name: getplanAttemptVar
tf_apply_stage.yml :
parameters:
- name: stage
type: string
default: Deploy_prepare
- name: Predecessors
type: string
default: none
stages:
- stage: ${{ parameters.stage }}
? ${{ if and(ne(parameters.Predecessors, 'previous'), ne(parameters.Predecessors, 'none') ) }}
: dependsOn: ${{parameters.Predecessors}}
displayName: ${{ parameters.TF_VAR_STAGE }} download artifact & Terraform apply changes to ${{ parameters.TF_VAR_APPLICATION }}
pool:
name: Azure Pipelines
vmImage: windows-2019
jobs:
- deployment: DeployTerraform
displayName: ${{ parameters.TF_VAR_STAGE }} download artifact & Terraform apply changes
variables:
# this one fails in pipeline : "[not recognise"
planJobAttempt: $[ stageDependencies.[parameters.Predecessors].PreDeployTerraform.outputs['setVarJobAttempt.planAttempt'] ]
# this one runs with no result in pipeline : planJobAttempt is the string "stageDependencies['parameters.Predecessors'].PreDeployTerraform.outputs['setVarJobAttempt.planAttempt']"
# planJobAttempt: $[ stageDependencies['parameters.Predecessors'].PreDeployTerraform.outputs['setVarJobAttempt.planAttempt'] ]
environment: ${{ parameters.TF_VAR_ENVIRONMENT }}
timeoutInMinutes: 480
strategy:
runOnce:
deploy:
steps:
- checkout: none
- powershell: |
echo "jobAttempts outputs are : $[ stageDependencies.[parameters.Predecessors].PreDeployTerraform.outputs ]"
- powershell: |
echo "jobAttempt is $[ stageDependencies['parameters.Predecessors'].PreDeployTerraform.outputs['setVarJobAttempt.planAttempt'] ]"
- powershell: |
echo "plan file for terraform is : $(pipeline_artifact_folder_download)/$(planJobAttempt)_${{ parameters.Predecessors }}/$(artefact_terraform_plan)_${{ parameters.TF_VAR_STAGE }}"
I tried differents things to get my system.jobAttempts from tf_plan_stage stage to tf_apply_stage stage, but without any success, as the variable in the last stage (tf_apply) seams unable to find the value from a "parametrised" stage name.
is there a way for that ?
thank-you for any answers.
Okay,
Once again, I will answer my own question :D
I've finallt find the way to use the parameters.Predecessors value in my variable declaration.
And, second, the Microsoft documentation seams not to be updated.
The tf_apply_stage.yml should be like that below ;
parameters:
- name: stage
type: string
default: Deploy_prepare
- name: Predecessors
type: string
default: none
stages:
- stage: ${{ parameters.stage }}
? ${{ if and(ne(parameters.Predecessors, 'previous'), ne(parameters.Predecessors, 'none') ) }}
: dependsOn: ${{parameters.Predecessors}}
displayName: ${{ parameters.TF_VAR_STAGE }} download artifact & Terraform apply changes to ${{ parameters.TF_VAR_APPLICATION }}
pool:
name: Azure Pipelines
vmImage: windows-2019
jobs:
- deployment: DeployTerraform
displayName: ${{ parameters.TF_VAR_STAGE }} download artifact & Terraform apply changes
variables:
*# this one give he good value for Variable*
planJobAttempt: $[ stageDependencies.${{ parameters.Predecessors }}.PreDeployTerraform.outputs['PreDeployTerraform.setVarJobAttempt.planAttempt'] ]
environment: ${{ parameters.TF_VAR_ENVIRONMENT }}
timeoutInMinutes: 480
strategy:
runOnce:
deploy:
steps:
- checkout: none
- powershell: |
echo "plan file for terraform is : $(pipeline_artifact_folder_download)/$(planJobAttempt)_${{ parameters.Predecessors }}/$(artefact_terraform_plan)_${{ parameters.TF_VAR_STAGE }}"
I put in BOLD the main changes above :
planJobAttempt: $[ stageDependencies.${{ parameters.Predecessors
}}.PreDeployTerraform.outputs['PreDeployTerraform.setVarJobAttempt.planAttempt']
]
The Job name have to be set twice in variable declaration !
Thanks for those who read my post ;)
And many thanks to this post from #Kay07949 who gives the good answers ;)
Need a small help , can someone point me what is the issue here .
What i am trying to achieve , the pipeline has to run checkout irrespective of the branch i am in , but in the build stage if the pipeline run from master it has execute some templet if from other branch different templet , i tried many option no luck .
Any pointer much appreciated .
- name: release
displayName: ReleaseVersion
type: string
default: ' '
- name: Deployment
displayName: DeploymentVersion
type: string
default: ' '
- name: Library
displayName: Library Release
type: boolean
default: True
trigger:
- none
pool:
name: DOTAzure-Ubuntu-20.04
resources:
repositories:
- repository: azureTemplates
type: githubenterprise
name: AAAAA/azure-pipelines
endpoint: BBBBBB
# You should add below variables to work complete pipeline
variables:
- name: servicename
value: XXXXX
- name: countryCode
value: YYYYY
- name: targetPort
value: 8113
stages:
- stage: Checkout
displayName: Micro Service checkout and Secrets
jobs:
- job:
steps:
- template: templates/repocheckout.yml#azureTemplates
- template: templates/retrieving_secrets.yml#azureTemplates
- template: templates/Ingesting_secrets.yml#azureTemplates
- stage: Build
displayName: Microservice Build
jobs:
- job: Master
condition: eq(variables['Build.SourceBranchName'], 'master')
steps:
- template: templates/mavenbuild_new.yml#azureTemplates
- template: templates/push_artifact_jfrog.yml#azureTemplates
parameters:
service_name: '$(countryCode)-$(servicename)'
release_version: $(release)
- job: Develop
dependsOn: Master
condition: eq(variables['Build.SourceBranchName'], 'develop')
steps:
- template: templates/mavenbuild.yml#azureTemplates
- template: templates/push_artifact_jfrog.yml#azureTemplates
parameters:
service_name: '$(countryCode)-$(servicename)'
release_version: $(release)
This doesn't look like valid yaml.
Have a look at this: https://learn.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema%2Cparameter-schema
It looks like you are missing this at the top of the file:
stages:
i.e. your yaml should be:
stages:
- stage: Checkout
displayName: Micro Service checkout and Secrets
jobs:
etc
I have a Azure DevOps YAML Pipeline to execute a Terraform deployment using the Terraform by MS DevLabs extension and an Azure Resource Manager service connection.
The last working state was using a pipeline template yaml file however I had to configure a parameter within the template and call the variable using the template expression syntax.
...
...
stages:
- stage: Plan
displayName: Terrafom Plan
jobs:
- job: DEV PLAN
displayName: Plan (DEV)
pool:
vmImage: "ubuntu-latest"
variables:
az_service_connection: "MyServiceConnection"
tf_environment: "DEV"
tf_state_rg: "DEV"
tz_state_location: "canadacentral"
tf_state_stgacct_name: "mystorageaccuontname1231231"
tf_state_container_name: "tfstate"
steps:
- template: templates/terraform-plan.yml
parameters:
az_service_connection: ${{ variables.az_service_connection }}
...
...
steps:
- task: terraformInstaller#0
displayName: "Install Terraform $(tf_version)"
inputs:
terraformVersion: $(tf_version)
- task: TerraformTaskV1#0
displayName: "Run > terraform init"
inputs:
command: "init"
commandOptions: "-input=false"
backendServiceArm: ${{ parameters.az_service_connection }}
...
...
I believe the reason why this works is because the template expression syntax ${{ variables.varname}} evaluates at compile time vs. runtime. If I didn't do it this way, i'd either get $(az_service_connection) passed into the backendServiceArm input or an empty value.
With the introduction of variable groups, i'm now facing similar behavior. I expect that the variable group evaluates after the template expression variable which causes ${{ variables.az_service_connection }} to have an empty value. I am unsure how to get this working.
How can I use variable groups with a pipeline template that uses a service connection?
I used $() syntax to pass arm connection to template:
Template file:
parameters:
- name: 'instances'
type: object
default: {}
- name: 'server'
type: string
default: ''
- name: 'armConnection'
type: string
default: ''
steps:
- task: TerraformTaskV1#0
inputs:
provider: 'azurerm'
command: 'init'
backendServiceArm: '${{ parameters.armConnection }}'
backendAzureRmResourceGroupName: 'TheCodeManual'
backendAzureRmStorageAccountName: 'thecodemanual'
backendAzureRmContainerName: 'infra'
backendAzureRmKey: 'some-terrform'
- ${{ each instance in parameters.instances }}:
- script: echo ${{ parameters.server }}:${{ instance }}
Main file:
trigger:
branches:
include:
- master
paths:
include:
- stackoverflow/09-array-parameter-for-template/*
# no PR triggers
pr: none
pool:
vmImage: 'ubuntu-latest'
variables:
- group: my-variable-group
- name: my-passed-variable
value: $[variables.myhello] # uses runtime expression
steps:
- template: template.yaml
parameters:
instances:
- test1
- test2
server: $(myhello)
armConnection: $(armConnection)
Note: Group my-variable-group contains armConnection variable
I'm trying to come up with a way to allow an easy selection of a given agent within the build pipeline just for debugging purposes. So far I have the below snippet. Both work without the if snippets wrapped around but I was trying to do one or the other based on either params being set or inturn variables being set so that if it was in debug mode it would select and agent and if it wasn't then it would just use the pool to select an agent to run the builds against. So far no luck though.
variables:
debugMode: 'false'
parameters:
- name: poolOption
type: string
default: 'ZupaDeploymentPool'
- name: debugMode
type: string
default: 'true'
- name: debugMachine
type: string
default: 'ZUPBUILD03'
trigger:
batch: true
branches:
include:
- master
paths:
exclude:
- README.md
${{ if ne($(debugMode), 'false') }}:
pool: ${{ parameters.poolOption }}
${{ if ne($(debugMode), 'true') }}:
pool:
name: ${{ parameters.poolOption }}
demands:
- Agent.Name -equals ${{ parameters.debugMachine }}
So after having a quick chat with kevin-lu-msft above lead me to this solution for handling selecting a specific agent in the pool.
parameters:
- name: debugMachine
displayName: 'Run on selected agent:'
type: string
default: 'Auto Select From Pool'
values:
- 'Auto Select From Pool'
- 'MACHINE01'
- 'MACHINE02'
- 'MACHINE03'
- 'MACHINE04'
- 'MACHINE05'
jobs:
-job: build
# -----------------------------------------------------------------------------------------------
# This is the pool selection section if a specific debug machine is passed in via the params
# It will select that specific one to run the build on. unfortunately azure doesnt let you pass
# vars or params to the process string in the demands which would have made this alot cleaner.
pool:
name: 'YOUAGENTSPOOLNAME'
${{ if ne(parameters.debugMachine, 'Auto Select From Pool') }}:
${{ if eq(parameters.debugMachine, 'MACHINE01') }}:
demands:
- Agent.Name -equals MACHINE01
${{ if eq(parameters.debugMachine, 'MACHINE02') }}:
demands:
- Agent.Name -equals MACHINE02
${{ if eq(parameters.debugMachine, 'MACHINE03') }}:
demands:
- Agent.Name -equals MACHINE03
${{ if eq(parameters.debugMachine, 'MACHINE04') }}:
demands:
- Agent.Name -equals MACHINE04
${{ if eq(parameters.debugMachine, 'MACHINE05') }}:
demands:
- Agent.Name -equals MACHINE05
steps:
- script: "echo finally it works"
I tested your YAML sample and made some modifications. You could try to set the "Expressions" under a stage and check if it could work as expected.
Here is an example, you could refer to it.
trigger:
- master
parameters:
- name: poolOption
type: string
default: 'Windows-latest'
- name: debugMode
type: string
default: false
values:
- true
- false
- name: debugMachine
type: string
default: 'ubuntu-16.04'
stages:
- stage: A
jobs:
- job: testjob
pool:
${{ if eq(parameters.debugmode, 'true') }}:
vmImage: ${{ parameters.poolOption }}
${{ if eq(parameters.debugmode, 'false') }}:
vmImage: ${{ parameters.debugMachine }}
steps:
- script : "echo Hello, world!"
Note: I am using Microsoft-hosted agents, so I am using the vmImage field.
You can specify specific self-hosted agents (name field) according to your needs.