Azure Devops Pipeline - azure

Need a small help , can someone point me what is the issue here .
What i am trying to achieve , the pipeline has to run checkout irrespective of the branch i am in , but in the build stage if the pipeline run from master it has execute some templet if from other branch different templet , i tried many option no luck .
Any pointer much appreciated .
- name: release
displayName: ReleaseVersion
type: string
default: ' '
- name: Deployment
displayName: DeploymentVersion
type: string
default: ' '
- name: Library
displayName: Library Release
type: boolean
default: True
trigger:
- none
pool:
name: DOTAzure-Ubuntu-20.04
resources:
repositories:
- repository: azureTemplates
type: githubenterprise
name: AAAAA/azure-pipelines
endpoint: BBBBBB
# You should add below variables to work complete pipeline
variables:
- name: servicename
value: XXXXX
- name: countryCode
value: YYYYY
- name: targetPort
value: 8113
stages:
- stage: Checkout
displayName: Micro Service checkout and Secrets
jobs:
- job:
steps:
- template: templates/repocheckout.yml#azureTemplates
- template: templates/retrieving_secrets.yml#azureTemplates
- template: templates/Ingesting_secrets.yml#azureTemplates
- stage: Build
displayName: Microservice Build
jobs:
- job: Master
condition: eq(variables['Build.SourceBranchName'], 'master')
steps:
- template: templates/mavenbuild_new.yml#azureTemplates
- template: templates/push_artifact_jfrog.yml#azureTemplates
parameters:
service_name: '$(countryCode)-$(servicename)'
release_version: $(release)
- job: Develop
dependsOn: Master
condition: eq(variables['Build.SourceBranchName'], 'develop')
steps:
- template: templates/mavenbuild.yml#azureTemplates
- template: templates/push_artifact_jfrog.yml#azureTemplates
parameters:
service_name: '$(countryCode)-$(servicename)'
release_version: $(release)

This doesn't look like valid yaml.
Have a look at this: https://learn.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema%2Cparameter-schema
It looks like you are missing this at the top of the file:
stages:
i.e. your yaml should be:
stages:
- stage: Checkout
displayName: Micro Service checkout and Secrets
jobs:
etc

Related

Reference a different Variable group per stage - stage per environment in environments - Azure Devops, template, variable groups

I want to get the environments and the associated variableGroup - loop thru the environments array in Stages.yml and create a stage per environment... insert correct variableGroup per stage... use variablesGroup values to perform jobs in jobs.yml - each variable group contains the same vars.. with different values.
``
I have a main.yml file in which I reference my environments and the associated group variable
main.yaml :
name: $(BuildID)
trigger: none
pool:
vmImage: 'leetchi-agentspool-windows'
variables:
- template: vars/variables-api.yaml
extends:
template: deploy-stages.yaml
parameters:
environements:
- environement: 'DV1'
variableGroupName: tf-dv1
- environement: 'IN1'
variableGroupName: tf-in1
A stage template which is supposed to apply with the environment parameters and the group variable.
stage.yaml :
---
parameters:
- name: environements
type: object
default: []
stages:
- ${{ each environement in parameters.environements }}:
- stage: 'Deploy to ${{ environement.environements }}'
variables:
- group: ${{ environement.variableGroup }}
jobs:
- template: template-terraform-deployment.yml
parameters:
environement: ${{environement.environements}}
project: $(project)
definition: $(def)
project1: $(project1)
definition1: $(def1)
stateKey: ${{environement.environement}}$(tfkey)
TerraformDirectory: $(TfDirectory)
azureSubscription: $(service_connection_name)
TfStateResourceGroupName: $(tf_resource_group)
TfStateStorageAccountName: $(tf_storage_account)
TStateContainerName: $(tf_container)
commandOptions: -out=$(System.DefaultWorkingDirectory)/terraform.tfplan -detailed-exitcode
I start in YAML. Could you help me please? Thank you very much in advance
You can modify your main.yaml as below:
trigger: none
pool:
vmImage: 'leetchi-agentspool-windows'
variables:
- template: vars/variables-api.yaml
extends:
template: deploy-stages.yaml
parameters:
- name: environment
values:
- DV1
- IN1
- name: variableGroupName
values:
- tf-dv1
- tf-in1
stage.yaml :
stages:
- stage: 'Deploy to ${{ parameters.environment }}'
variables:
- group: ${{ parameters.variableGroupName}}
jobs:
- template: template-terraform-deployment.yml
parameters:
environment: ${{ parameters.environment }}
project: $(project)
definition: $(def)
project1: $(project1)
definition1: $(def1)
stateKey: ${{ parameters.environment }} + $(tfkey)
TerraformDirectory: $(TfDirectory)
azureSubscription: $(service_connection_name)
TfStateResourceGroupName: $(tf_resource_group)
TfStateStorageAccountName: $(tf_storage_account)
TStateContainerName: $(tf_container)
commandOptions: -out=$(System.DefaultWorkingDirectory)/terraform.tfplan -detailed-exitcode

Azure Pipelines Automatic retries for a task

This is part of my Yml file
I need to re-call this template if it failed.it should be rerun again. After a few seconds, ideally.I am new to yml files
I tried to use retryCountOnTaskFailure but it should be under task but the template calling different hierarchy
https://learn.microsoft.com/en-us/azure/devops/release-notes/2021/sprint-195-update#automatic-retries-for-a-task
- template: ${{variables['System.DefaultWorkingDirectory']}}
parameters:
Test: ${{ parameters.isTestRelease }}
Environment: ${{ parameters.deploymentTarget }}
Component: '${{ parameters.component }}'
The retryCountOnTaskFailure feature is applied to individual tasks within a pipeline. Templates aren't tasks, they act more like an include that expands the contents of the template into your pipeline.
# pipeline.yml
trigger: none
parameters:
- name: isTestRelease
type: boolean
default: false
- name: deploymentTarget
type: string
default: DEV
values:
- DEV
- QA
- UAT
- PROD
- name: component
type: string
default: 'x'
stages:
- template: my-template.yml
parameters:
Test: ${{ parameters.isTestRelease }}
Environment: ${{ parameters.deploymentTarget }}
Component: ${{ parameters.component }}
And within the template, you'd want to add the retry logic to specific tasks:
# my-template.yml
parameters:
- name: isTestRelease
type: boolean
- name: component
type: string
stages:
- stage: Test
jobs:
- job: 1
steps:
- task: ...
- task: ...
- task: ...
retryCountOnFailure: 2
- task: ...
It's also located under "Control Options" when the "Command Line" task is added:

How to use condition on library variable on azure pipeline yaml

I have on my azure pipelines a libary varrible named:
deployHostMachine
and in it a variable named:
Host
I need to run a task using if with condition on Host How do i do it?
My code:
parameters:
trigger:
- none
variables:
- group: DeploymentHost
- name: localForHost
value: $(Host)
- name: testEmpty
value: ''
stages:
- stage: "ManageDeployMachines"
jobs:
- job: "ManageDeployMachines"
pool:
vmImage: 'windows-latest'
steps:
- ${{ if eq(parameters.paramHost,'rnd-3.sepio.systems') }}:
- task: ManageVariableGroupTask#0
displayName: 'Set deploy host url'
inputs:
pat: 'yxtloxscphyvuevrzafozok533orcv3pqon4a7e3kzpc4je23zha'
project: '70e3803b-2a31-4f00-83a1-9fc825f4447b'
vgname: 'DeploymentHost'
orgname: 'SepioDevOfficial'
op: 'Update'
varops: '= Host => "rnd-4.sepio.systems"'

Aure pipeline : use output variable in multi-stage parametrised

I am facing an issue I could not resolve.
regarding this documentation it seams to be possible to set a variable as output of a job in the first stage, then use it as input to set a job variable in second stage.
The pbm I'm facing, is that my stages are parametrised, so their name are not constant.
here my sample code :
name: mypipeline
parameters:
- name: Application
displayName: Application to deploy the front end using front door
type: string
- name: stage_vars
displayName: stage VARS
type: object
default:
- stage_1
- stage_2
- none
trigger: none
stages:
- stage: Build
displayName: ${{ parameters.Application }} - Build.
pool:
name: Azure Pipelines
vmImage: windows-2019
jobs:
- template: ../templates/job_BuildAndPublish.yml
parameters:
BuildId: $(Build.BuildId)
importTerratest: false
importARM: true
- ${{ each stageregion in parameters.stage_vars }}:
- ${{ if ne(stageregion, 'none') }}:
- template: ../templates/isolated_web/tf_plan_stage.yml
parameters:
BuildId: $(system.BuildId)
Predecessors: 'Build'
- template: ../templates/isolated_web/tf_apply_stage.yml
parameters:
Predecessors: '${{stageregion}}_prepare'
build template is not usefull, but the ones from tf_plan_stage and tf_apply_stage are below :
tf_plan_stage.yml :
parameters:
- name: BuildId
type: string
default: $(system.BuildId)
- name: stage
type: string
default: Deploy_prepare
- name: Predecessors
type: string
default: none
stages:
- stage: ${{ parameters.stage }}
? ${{ if and(ne(parameters.Predecessors, 'previous'), ne(parameters.Predecessors, 'none') ) }}
: dependsOn: ${{parameters.Predecessors}}
displayName: ${{ parameters.TF_VAR_STAGE }} Terraform plan & publish artifact for ${{ parameters.TF_VAR_APPLICATION }}
pool:
name: Azure Pipelines
vmImage: windows-2019
jobs:
- deployment: PreDeployTerraform
displayName: ${{ parameters.TF_VAR_STAGE }} Terraform plan & publish artifact
environment: PLAN_${{ parameters.TF_VAR_ENVIRONMENT }}
timeoutInMinutes: 480
strategy:
runOnce:
deploy:
steps:
- checkout: none
# set output planAttempt output available for later Apply job
- powershell: |
echo "jobAttempt is $(System.JobAttempt)"
echo "##vso[task.setvariable variable=planAttempt;isOutput=true;]$(System.JobAttempt)"
name: setVarJobAttempt
- powershell: |
echo "variable setVarJobAttempt.planAttempt value is : $(setVarJobAttempt.planAttempt)"
name: getplanAttemptVar
tf_apply_stage.yml :
parameters:
- name: stage
type: string
default: Deploy_prepare
- name: Predecessors
type: string
default: none
stages:
- stage: ${{ parameters.stage }}
? ${{ if and(ne(parameters.Predecessors, 'previous'), ne(parameters.Predecessors, 'none') ) }}
: dependsOn: ${{parameters.Predecessors}}
displayName: ${{ parameters.TF_VAR_STAGE }} download artifact & Terraform apply changes to ${{ parameters.TF_VAR_APPLICATION }}
pool:
name: Azure Pipelines
vmImage: windows-2019
jobs:
- deployment: DeployTerraform
displayName: ${{ parameters.TF_VAR_STAGE }} download artifact & Terraform apply changes
variables:
# this one fails in pipeline : "[not recognise"
planJobAttempt: $[ stageDependencies.[parameters.Predecessors].PreDeployTerraform.outputs['setVarJobAttempt.planAttempt'] ]
# this one runs with no result in pipeline : planJobAttempt is the string "stageDependencies['parameters.Predecessors'].PreDeployTerraform.outputs['setVarJobAttempt.planAttempt']"
# planJobAttempt: $[ stageDependencies['parameters.Predecessors'].PreDeployTerraform.outputs['setVarJobAttempt.planAttempt'] ]
environment: ${{ parameters.TF_VAR_ENVIRONMENT }}
timeoutInMinutes: 480
strategy:
runOnce:
deploy:
steps:
- checkout: none
- powershell: |
echo "jobAttempts outputs are : $[ stageDependencies.[parameters.Predecessors].PreDeployTerraform.outputs ]"
- powershell: |
echo "jobAttempt is $[ stageDependencies['parameters.Predecessors'].PreDeployTerraform.outputs['setVarJobAttempt.planAttempt'] ]"
- powershell: |
echo "plan file for terraform is : $(pipeline_artifact_folder_download)/$(planJobAttempt)_${{ parameters.Predecessors }}/$(artefact_terraform_plan)_${{ parameters.TF_VAR_STAGE }}"
I tried differents things to get my system.jobAttempts from tf_plan_stage stage to tf_apply_stage stage, but without any success, as the variable in the last stage (tf_apply) seams unable to find the value from a "parametrised" stage name.
is there a way for that ?
thank-you for any answers.
Okay,
Once again, I will answer my own question :D
I've finallt find the way to use the parameters.Predecessors value in my variable declaration.
And, second, the Microsoft documentation seams not to be updated.
The tf_apply_stage.yml should be like that below ;
parameters:
- name: stage
type: string
default: Deploy_prepare
- name: Predecessors
type: string
default: none
stages:
- stage: ${{ parameters.stage }}
? ${{ if and(ne(parameters.Predecessors, 'previous'), ne(parameters.Predecessors, 'none') ) }}
: dependsOn: ${{parameters.Predecessors}}
displayName: ${{ parameters.TF_VAR_STAGE }} download artifact & Terraform apply changes to ${{ parameters.TF_VAR_APPLICATION }}
pool:
name: Azure Pipelines
vmImage: windows-2019
jobs:
- deployment: DeployTerraform
displayName: ${{ parameters.TF_VAR_STAGE }} download artifact & Terraform apply changes
variables:
*# this one give he good value for Variable*
planJobAttempt: $[ stageDependencies.${{ parameters.Predecessors }}.PreDeployTerraform.outputs['PreDeployTerraform.setVarJobAttempt.planAttempt'] ]
environment: ${{ parameters.TF_VAR_ENVIRONMENT }}
timeoutInMinutes: 480
strategy:
runOnce:
deploy:
steps:
- checkout: none
- powershell: |
echo "plan file for terraform is : $(pipeline_artifact_folder_download)/$(planJobAttempt)_${{ parameters.Predecessors }}/$(artefact_terraform_plan)_${{ parameters.TF_VAR_STAGE }}"
I put in BOLD the main changes above :
planJobAttempt: $[ stageDependencies.${{ parameters.Predecessors
}}.PreDeployTerraform.outputs['PreDeployTerraform.setVarJobAttempt.planAttempt']
]
The Job name have to be set twice in variable declaration !
Thanks for those who read my post ;)
And many thanks to this post from #Kay07949 who gives the good answers ;)

Azure Pipelines YAML: Unexpected value 'variables'

I'm using Azure Pipelines as a part of Azure DevOps.
I'm trying to define variables in my template file, because I need to use the same value multiple times.
This is my stage-template.yml:
parameters:
- name: param1
type: string
- name: param2
type: string
variables:
var1: path/${{ parameters.param2 }}/to-my-file.yml
stages:
- stage: Deploy${{ parameters.param2 }}
displayName: Deploy ${{ parameters.param1 }}
jobs:
...
When trying to use this pipeline, I get an error message:
/stage-template.yml (Line: 7, Col: 1): Unexpected value 'variables'
Why is this not working? What did I do wrong?
You can't have variables in a template that is included as a stage, job or step template (i.e. included below a stages, jobs or steps element in a pipeline). You can only use variables in a variable template.
The documentation sadly is not really clear about that.
Including a stage template
# pipeline-using-stage-template.yml
stages:
- stage: stage1
[...]
# stage template reference, no 'variables' element allowed in stage-template.yml
- template: stage-template.yml
Including a variable template
# pipeline-using-var-template.yml
variables:
# variable template reference, only variables allowed inside the template
- template: variables.yml
steps:
- script: echo A step.
If you are using a template to include variables in a pipeline, the included template can only be used to define variables.
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops#variable-reuse
you cant have parameters in the pipeline, only in the templateReferences:
name: string # build numbering format
resources:
pipelines: [ pipelineResource ]
containers: [ containerResource ]
repositories: [ repositoryResource ]
variables: # several syntaxes, see specific section
trigger: trigger
pr: pr
stages: [ stage | templateReference ]
if you want to use variables in templates you have to use proper syntax:
parameters:
- name: param1
type: string
- name: param2
type: string
stages:
- stage: Deploy${{ parameters.param2 }}
displayName: Deploy ${{ parameters.param1 }}
variables:
var1: path/${{ parameters.param2 }}/to-my-file.yml
jobs:
...
This works for me:
In your parent yaml:
stages:
- stage: stage1
displayName: 'stage from parent'
jobs:
- template: template1.yml
parameters:
param1: 'somevalueforparam1'
inside template1:
parameters:
param1: ''
param2: ''
jobs:
- job: job1
workspace:
clean: all
displayName: 'Install job'
pool:
name: 'yourpool'
variables:
- name: var1
value: 'value1'

Resources