azure-pipeline.yml
trigger:
- master
parameters:
- name: config
displayName: Execution Environment
type: string
default: QA
values:
- QA
- PreProd
- Prod
pool:
vmImage: 'windows-latest'
The above works perfectly, so in Azure the Execution Environment parameter is shown when I run the pipeline.
If however I attempt to put the parameters in a template as follows:
azure-pipeline.yml
trigger:
- master
extends:
template: parameters.yml
pool:
vmImage: 'windows-latest'
parameters.xml
parameters:
- name: config
displayName: Execution Environment
type: string
default: QA
values:
- QA
- PreProd
- Prod
Then when I run the pipeline the parameter is not shown.
In summary I'm trying to re-use a parameters.yml in different pipelines but extends: template: does not seem to work even though per this link it should:
https://learn.microsoft.com/en-us/azure/devops/pipelines/security/templates?view=azure-devops#set-required-templates
Runtime parameters are something different than templates parameters and having the second in your pipeline will not cause them to show on the UI. There is no way to template runtime parameters. You need to repeat them in each pipeline you expect to have them.
Related
I'm currently trying to setup my complete build/release pipeline with yaml files.
First I tried with different stages (dev/staging/prod) and it worked.
Now I wanted to add an approval that the deploy doesn't not happen automatically on each system.
Therefore I added an environment in the TFS with an approval check.
But when I try to setup the yaml file I always get an error.
I don't know how to setup this properly.
This is my main yaml file called release-pipeline.yaml
trigger:
- master
pool:
name: POOL
stages:
- stage: BuildSolution
jobs:
- job: BuildSolutionJob
displayName: Build
workspace:
clean: all
steps:
- template: yaml/BuildSolution.yml
- template: yaml/CopyFiles.yml
- template: yaml/PublishArtifact.yml
- stage: DeployOn_STAGING_System
dependsOn: BuildSolution
jobs:
- job: Deploy_STAGING
- template: yaml/Deploy.yml
parameters:
Environment: 'SITE'
Staging: 1
- stage: Deploy_DEV_System
dependsOn: BuildSolution
jobs:
- deployment: Deploy_DEV
environment: ENVCHECK_DEV
strategy:
runOnce:
deploy:
steps:
- template: yaml/Deploy.yml
parameters:
Environment: 'SITE'
ViewDeploy: 1
This is my Deploy.yml file which i want to execute (only some snips):
parameters:
- name: Environment
type: string
- name: ProdSystem
type: number
default: 0
- name: Staging
type: number
default: 0
- name: ViewDeploy
type: number
default: 0
jobs:
- job:
variables:
artifactName: $[stageDependencies.BuildSolution.BuildSolutionJob.outputs['SetVariables.artifactName']]
version: $[stageDependencies.BuildSolution.BuildSolutionJob.outputs['SetVariables.version']]
steps:
- task: PowerShell#2
displayName: Display given parameters
inputs:
targetType: inline
script: >
Write-Host "ArtifaceName: " $(artifactName)
Write-Host "Environment for Deploy: " ${{ parameters.Environment }}
Write-Host "##vso[task.setvariable variable=isStaging]${{ parameters.Staging }}"
failOnStderr: true
When I try to execute I get the following error:
/release-pipeline.yml: Unexpected value 'parameters'.
How do I need to change this that it will work with the template in both cases, with and without the environment approval check?
I tried https://samlearnsazure.blog/2020/02/05/approvals-in-environments/
and of course different structure for the calling. But nothing helped.
In the tutorials they always have steps below the "deploy" but because I have different sites and environments I want/need the template file to save work.
I found another post which goes in the same direction.
I reworked my complete template so that I can use the approach of this: DevOps template with conditional manual approval job
It's not the same as I wanted, but it works.
My goal was that I don't want to create a environment if I have no checks on this site. Only for sites where I wanted the approval check.
With the above mentioned solution I need to create an environment for each site, independent if I have checks or not.
Azure Pipelines support multiple stages in YAML. One typical example would be to have something like :
trigger:
- master
pool:
name: Default
demands:
- npm
- msbuild
- visualstudio
stages:
- stage: build
jobs:
- job: Build app
- stage: deploy
jobs:
- job: Deploy to dev
I'm not used to working like this. Usually, I would run the pipeline to build my application and drop artifacts to a drop folder. The pipeline would be the same regardless the environment that would later be targeted by the release.
Then I would choose to run a release, either Integration, UAT, or Production.
However, having multi-stages pipeline we are mixing the build and the release together. So how would I release in a given environment ? Do I have to duplicate this pipeline per environment ?
You can use template structure here. In this you will need to create separate files for different jobs and variables. Then you will need to execute templates with suitable variable template files for each stage.
Directory structure
Directory Structure
Pipeline:
Pipeline stages
Environments
Environments
Here's the sample pipeline:
trigger:
- master
variables:
- name: vmImage
value: 'ubuntu-latest'
stages:
- stage: Build
displayName: Build stage
jobs:
- job: BuildJob
pool:
vmImage: $(vmImage)
steps:
- template: Jobs/build.yml
- stage: NonProd
displayName: Deploy non prod stage
jobs:
- deployment: DeploymentJob1
pool:
vmImage: $(vmImage)
environment: non-prod
variables:
- template: Variables/non-prod.yml
strategy:
runOnce:
deploy:
steps:
- template: Jobs/deploy.yml
- stage: Prod
displayName: Deploy prod stage
jobs:
- deployment: DeploymentJob2
pool:
vmImage: $(vmImage)
environment: prod
variables:
- template: Variables/prod.yml
strategy:
runOnce:
deploy:
steps:
- template: Jobs/deploy.yml
Jobs/build.yml
steps:
- script: echo I am building!
displayName: 'Run Build'
Jobs/deploy.yml
steps:
- script: echo I am deploying to $(Environment)!
displayName: 'Run Deployment'
Variables/non-prod.yml
variables:
- name: Environment
value: non-prod
Variables/prod.yml
variables:
- name: Environment
value: prod
Please consider the following:
- job: Backend
steps:
- template: $(ClassLibraryTemplate)
parameters:
projectName: 'Core'
solutionPath: 'Source/Core.sln'
ClassLibraryTemplate is defined as a pipeline variable. But when I run the build, it fails because the variable is not replaced by its value and the template is not found.
Is it not possible to store the template name in a variable ?
For Azure DevOps YAML pipeline, the template get processed at compile time. However, the $(ClassLibraryTemplate) get processed at the runtime. That's why it fails.
More information: Understand variable syntax
You could define variable or parameter in your YAML pipeline, then use template expression. For parameter, you could specify value when queue/run pipeline in pop-up window.
For example:
parameters:
- name: temName
displayName: template name
type: string
default: steps/test.yml
trigger:
- none
variables:
- name: tem
value: steps/build.yml
jobs:
- job: Linux
pool:
vmImage: 'ubuntu-16.04'
steps:
- template: ${{ variables.tem }}
- template: ${{ parameters.temName }}
I have two .yml files in my repo. One for build, one for deployment. The main reason why I would like to keep build separate from the deployment is that I also would like to store variables for environments in my repo, e.i. in variables-dev.yml and variables-prod.yml files. So there is no need to create a new build every time (which includes running tests, docker image build etc.).
The file build.yml:
trigger:
paths:
exclude:
- build.yml
- deploy.yml
stages:
- stage: build
jobs:
...
And the deploy.yml, which I want to be triggered only on the completion of the build pipeline. That's why I add the first exclusion of all paths, but add one on pipeline resource.
trigger:
paths:
exclude:
- '*'
resources:
pipelines:
- pipeline: build
source: build
trigger:
branches:
include:
- '*'
stages:
- stage: dev
variables:
- template: variables-dev.yml
jobs:
- deployment: deploy_dev
environment: 'dev'
strategy:
runOnce:
deploy:
steps:
...
- stage: prod
dependsOn: dev
variables:
- template: variables-prod.yml
jobs:
- deployment: deploy_prod
environment: 'prod'
strategy:
runOnce:
deploy:
steps:
...
Unfortunately it does not seem to work. The top trigger blocks lower trigger. And if I remove the top trigger than the deploy pipeline is triggered at the same time with the build one.
you have to start your deploy.yml with trigger: none
trigger: none
resources:
pipelines:
- pipeline: ci-pipeline
source: my-build-pipeline
trigger:
enabled: true
branches:
include:
- master
Set your triggers for the second yml to none, then add this setting in the "Triggers" section of the UI. It will stage your builds as you describe
I read the environments documentation here and the issues opened under the environment resource, however I find it impossible to achieve my goal:
I would like to use a parametrized yaml template in order to deploy to multiple environments like below:
parameters:
pool_name: ''
aks_namespace: ''
environment: ''
jobs:
- job: preDeploy
displayName: preDeploy
pool:
name: $(pool_name)
steps:
- template: cd_step_prerequisites.yml
- deployment: Deploy
displayName: Deploy
dependsOn: preDeploy
condition: succeeded()
variables:
secret_name: acrecret
pool:
name: dockerAgents
**environment: '$(environment).$(aks_namespace)'**
strategy:
runOnce:
deploy:
steps:
- template: cd_step_aks_deploy.yml
- job: postDeploy
displayName: postDeploy
dependsOn: Deploy
condition: succeeded()
pool:
name: $(pool_name)
steps:
- template: cd_step_postrequisites.yml
I would like to use this approach so that I only host a minimal pipeline.yml next to my code, and then I would have all the templates in a different repo and call them from the main pipeline, as such:
resources:
repositories:
- repository: self
- repository: devops
type: git
name: devops
- stage: CD1
displayName: Deploy to Alpha
jobs:
**- template: pipeline/cd_job_api.yml#devops**
parameters:
pool_name: $(pool_name)
aks_namespace: $(aks_namespace)
app_name: $(app_name)
app_image_full_name: $(app_image_full_name)
environment: alpha
Then I would be able to pass the $environment variable in order to manipulate multiple deployment targets (AKS clusters/ groups of namespaces) from one template.
Currently this seems to be impossible as the default AzureDevOps parser fails when I try to run my pipeline, with the message "$(environment) environment does not contain x namespace" which tells me that the variable doesn't get expanded.
Is this planning to be implemented anytime soon? If not, are there any alternatives to use only one parametrized job template to deploy to multiple environments?
I think you would need to either parse the files and do a token replace with a script or there should be steps for that.
Your main alternative would be helm. It allows to create templates and pass in variables to render those templates.
Maybe I'm a bit late to the party, but I was also struggling with this problem and found this open thread.
I found this "closed" issue on github. The key points for me in the issue are this comment with a partial solution and this other comment pointing to the explanation of why is not working. Quoting Microsoft's article:
It also answers another common issue: why can't I use variables to resolve service connection / environment names? Resources are authorized before a stage can start running, so stage- and job-level variables aren't available. Pipeline-level variables can be used, but only those explicitly included in the pipeline. Variable groups are themselves a resource subject to authorization, so their data is likewise not available when checking resource authorization.
Regarding the solution, based on the first comment I reference, I ended up creating a new Variable Group with variables with the following naming convention: product.environment.varname. Then I added this group to the beginning of the pipeline (global scope) and then referenced the variables using macro syntax: $(var)
Quick example:
variables:
- group: Product.Pipelines.Environments
jobs:
- job: preDeploy
displayName: preDeploy
pool:
name: $(pool_name)
steps:
- template: cd_step_prerequisites.yml
- deployment: Deploy
displayName: Deploy
dependsOn: preDeploy
condition: succeeded()
variables:
secret_name: acrecret
pool:
name: dockerAgents
environment: $(product.dev.environmentname) #this is the variable within the variable group
strategy:
runOnce:
deploy:
steps:
- template: cd_step_aks_deploy.yml
The variable group will contain among other variables:
product.dev.environmentname: Development
product.stg.environmentname: Staging
product.prd.environmentname: Production