Azure yaml pipeline using "extends" - azure

I am trying to use extends as part of my pipeline. I am trying to get the first basic step working from Azure docs ie
# pythonparameter-template.yml
parameters:
- name: usersteps
type: stepList
default: []
steps:
- ${{ each step in parameters.usersteps }}
- ${{ step }}
# azure-pipelines.yml
trigger: none
resources:
repositories:
- repository: CI-CD-Templates
type: git
name: /CI-CD-Templates
ref: refs/heads/master
extends:
template: /pythonparameter-template.yml#CI-CD-Templates
parameters:
usersteps:
- script: echo This is my first step
- script: echo This is my second step
I keep getting the below error:
The directive 'each' is not allowed in this context. Directives are not supported for expressions that are embedded within a string. Directives are only supported when the entire value is an expression
Unexpected value '${{ each step in parameters.usersteps }} - ${{ step }}'
Also after I extend from a template can azure-pipelines.yml also have it's own custom steps ie
# azure-pipelines.yml
resources:
repositories:
- repository: templates
type: git
name: MyProject/MyTemplates
ref: tags/v1
extends:
template: template.yml#templates
parameters:
usersteps:
- script: echo This is my first step
- script: echo This is my second step
steps:
- template: /validation-template.yml#CI-CD-Templates
parameters:
commitMessage: $(commitMessage)
- template: /shared-template.yml#CI-CD-Templates
parameters:
buildArtifactDir: $(buildArtifactDir)

Update
Please refer the response in this DC link-- Yaml extends feature erroring out when looping through steps.
​The YAML has a validation task in #pythonparameter-template.yml
Comment out these 2 lines and your YAML will succeed. The template shown here is preventing any tasks to be used. This could be a scenario for someone with specific security requirements.
${{ if eq(pair.key, 'task') }}:
'${{ pair.value }}': error
Are you trying to combine two yml pythonparameter-template.yml azure-pipelines.yml in the same file? It's not supported.
parameters:
- name: usersteps
type: stepList
default: []
steps:
- ${{ each step in parameters.usersteps }}
- ${{ step }}
According to the error Directives are not supported for expressions that are embedded within a string. Directives are only supported when the entire value is an expression Unexpected value '${{ each step in parameters.usersteps }} - ${{ step }}'
You maybe use the wrong format. About the formats you could refer our official doc here--
Template types & usage
Besides, you can make azure-pipelines.yml also have it's own custom steps. But you need to put them under parameters in the pipeline, not like the way you use.
azure-pipelines.yml
trigger:
- master
extends:
template: pythonparameter-template.yml
parameters:
buildSteps:
- bash: echo Test #Passes
displayName: succeed
- bash: echo "Test"
displayName: succeed
- script: echo "Script Test"
displayName: Fail

Related

YAML - Execute template in Deploymentjob

I'm currently trying to setup my complete build/release pipeline with yaml files.
First I tried with different stages (dev/staging/prod) and it worked.
Now I wanted to add an approval that the deploy doesn't not happen automatically on each system.
Therefore I added an environment in the TFS with an approval check.
But when I try to setup the yaml file I always get an error.
I don't know how to setup this properly.
This is my main yaml file called release-pipeline.yaml
trigger:
- master
pool:
name: POOL
stages:
- stage: BuildSolution
jobs:
- job: BuildSolutionJob
displayName: Build
workspace:
clean: all
steps:
- template: yaml/BuildSolution.yml
- template: yaml/CopyFiles.yml
- template: yaml/PublishArtifact.yml
- stage: DeployOn_STAGING_System
dependsOn: BuildSolution
jobs:
- job: Deploy_STAGING
- template: yaml/Deploy.yml
parameters:
Environment: 'SITE'
Staging: 1
- stage: Deploy_DEV_System
dependsOn: BuildSolution
jobs:
- deployment: Deploy_DEV
environment: ENVCHECK_DEV
strategy:
runOnce:
deploy:
steps:
- template: yaml/Deploy.yml
parameters:
Environment: 'SITE'
ViewDeploy: 1
This is my Deploy.yml file which i want to execute (only some snips):
parameters:
- name: Environment
type: string
- name: ProdSystem
type: number
default: 0
- name: Staging
type: number
default: 0
- name: ViewDeploy
type: number
default: 0
jobs:
- job:
variables:
artifactName: $[stageDependencies.BuildSolution.BuildSolutionJob.outputs['SetVariables.artifactName']]
version: $[stageDependencies.BuildSolution.BuildSolutionJob.outputs['SetVariables.version']]
steps:
- task: PowerShell#2
displayName: Display given parameters
inputs:
targetType: inline
script: >
Write-Host "ArtifaceName: " $(artifactName)
Write-Host "Environment for Deploy: " ${{ parameters.Environment }}
Write-Host "##vso[task.setvariable variable=isStaging]${{ parameters.Staging }}"
failOnStderr: true
When I try to execute I get the following error:
/release-pipeline.yml: Unexpected value 'parameters'.
How do I need to change this that it will work with the template in both cases, with and without the environment approval check?
I tried https://samlearnsazure.blog/2020/02/05/approvals-in-environments/
and of course different structure for the calling. But nothing helped.
In the tutorials they always have steps below the "deploy" but because I have different sites and environments I want/need the template file to save work.
I found another post which goes in the same direction.
I reworked my complete template so that I can use the approach of this: DevOps template with conditional manual approval job
It's not the same as I wanted, but it works.
My goal was that I don't want to create a environment if I have no checks on this site. Only for sites where I wanted the approval check.
With the above mentioned solution I need to create an environment for each site, independent if I have checks or not.

How can I use variable created inside of pipeline (from gradle.properties file) within an template as a task

I have an java project which have gradle.properties file. Im extracting variables defined in gradle.properties as
##vso[task.setvariable variable=myVariable;]`my script to extract it from gradle.properties`
Then im using template from another repository that needs that variable but I can't use it within task, but when I try use it within - script: echo $variable as a step instead of task it is working.
When i try to use it within task it sees variable as $variable not a value.
Maybe there is a better way to extract variables to azure pipeline instead of using this approach?
Check the error message:
We get the error before the pipeline run the bash task, Since it cannot create the variable parampass, we get the parameters value is $(parampass) instead of the variable value.
Check this doc:
In a pipeline, template expression variables ${{ variables.var }} get processed at compile time, before runtime starts. Macro syntax variables $(var) get processed during runtime before a task runs. Runtime expressions $[variables.var] also get processed during runtime but were designed for use with conditions and expressions.
As a workaround:
pipeline.yml
pool:
vmImage: ubuntu-20.04
resources:
repositories:
- repository: common
type: git
name: organisation/repo-name
variables:
- name: parampass
value: xxx
stages:
- stage: "Build"
jobs:
- job: "Build"
steps:
- template: templatename.yml#common
parameters:
par1: ${{ variables.parampass}}
Result:
Probably you do not provide variable to your template
Example execution of template with provided parameter
- template: my/path/to/myTemplate.yml#MyAnotherRepositoryResourceName
parameters:
projectsToBeTested: $(someVariable)
And example template accepting parameters
steps:
- task: DotNetCoreCLI#2
displayName: 'Just testing'
inputs:
command: test
projects: ${{ parameters.projectsToBeTested}}
Please provide more information if it does not help.
Code looks like this:
pipeline.yml
pool:
vmImage: ubuntu-20.04
resources:
repositories:
- repository: common
type: git
name: organisation/repo-name
stages:
- stage: "Build"
jobs:
- job: "Build"
steps:
- bash: |
echo "##vso[task.setvariable variable=parampass]anything"
- template: templatename.yml#common
parameters:
par1: $(parampass)
templatename.yml
parameters:
- name: par1
steps:
- task: SonarCloudPrepare#1
displayName: SonarCloud analysis prepare
inputs:
SonarCloud: ${{ parameters.par1}}
organization: 'orgname'
scannerMode: 'Other'
extraProperties: |
# Additional properties that will be passed to the scanner,
# Put one key=value per line, example:
# sonar.exclusions=**/*.bin
sonar.projectKey= # same param pass case
sonar.projectName= # same param pass case
Generally, it does not matter if i do have parameters passed or if I'm using the template as if it were part of the pipeline code within. Output is always $(parampass) could not be found or smth

How to send different parameters to Azure pipeline templates based on the System.PullRequest.TargetBranch variable

I have a pipeline template that should receive a different input based on the Pull Request Target Branch.
Template:
parameters:
- name: BUILD_FOLDER
type: string
steps:
- script: |
echo "Build folder: ${{ parameters.BUILD_FOLDER }}"
displayName: 'Echo Build folder'
Pipeline YAML.
trigger: none
pr:
branches:
include:
- '*'
pool:
vmImage: ubuntu-latest
steps:
- template: templates/template.yml
parameters:
${{ if contains(variables['System.PullRequest.TargetBranch'], 'master') }}:
BUILD_FOLDER: base
${{ if not(contains(variables['System.PullRequest.TargetBranch'], 'master')) }}:
BUILD_FOLDER: $(System.PullRequest.TargetBranch)
I tried doing it like this but always it goes with ${{ if not(contains(variables['System.PullRequest.TargetBranch'], 'master')) }}: even the target branch is master in the Pull Request. Is there another way to do this?
Also the repository is in GitHub.
Thanks in advance.
I can reproduce the same issue. It is because the predefined variable System.PullRequest.TargetBranch cannot be evaluated at the compile time. It can be evaluated at run time (wrapped in $())without any problem. Expression wrapped in ${{}} will be evaluated at the build compile time. See Runtime expression syntax.
These variables that are marked not available in template in document cannot be parsed at compile time. The document causes little confusion, because it doesnot clearly state those variables cannot be parsed at compile time.
Since the template is evaluated at compile time. So the variable System.PullRequest.TargetBranch wrapped ${{}} is evaluated to an empty string.
I tested with below yaml. And the powershell task got executed:
- ${{ if eq(variables['System.PullRequest.TargetBranch'], '') }}:
- powershell: echo "i will out put empty"
The workaround for this is to set the variable value by script in an additional powershell task, as mentioned by Krzysztof Madej. I changed your yaml file a little bit. See below:
trigger: none
pr:
branches:
include:
- '*'
pool:
vmImage: ubuntu-latest
steps:
- powershell: |
$targetBranch = "$(System.PullRequest.TargetBranch)"
if($targetBranch -eq "master"){
Write-Host "##vso[task.setvariable variable=BUILD_FOLDER;]base"
}
if($targetBranch -ne "master"){
Write-Host "##vso[task.setvariable variable=BUILD_FOLDER;]$targetBranch"
}
- template: templates/template.yml
parameters:
BUILD_FOLDER: $(BUILD_FOLDER)
Some of variables can't be used in expressions like this. It is shown in the last column Available in templates? on this page
What you can do is move this to the template itself like this:
parameters:
- name: BASE_BUILD_FOLDER
type: string
default: base
steps:
- pwsh: |
$targetBranch = '$(System.PullRequest.TargetBranch)'
$buildFolder = '${{ parameters.BASE_BUILD_FOLDER }}'
if (!($targetBranch -like "*master")){
$buildFolder = $targetBranch
}
Write-Host "##vso[task.setvariable variable=BUILD_FOLDER;]$buildFolder"
- script: |
echo "Build folder: $(BUILD_FOLDER)"
displayName: 'Echo Build folder'
and then you can call it
trigger: none
pr:
branches:
include:
- '*'
pool:
vmImage: ubuntu-latest
steps:
- template: templates/template.yml

Azure Pipeline Get Template name from variable

Please consider the following:
- job: Backend
steps:
- template: $(ClassLibraryTemplate)
parameters:
projectName: 'Core'
solutionPath: 'Source/Core.sln'
ClassLibraryTemplate is defined as a pipeline variable. But when I run the build, it fails because the variable is not replaced by its value and the template is not found.
Is it not possible to store the template name in a variable ?
For Azure DevOps YAML pipeline, the template get processed at compile time. However, the $(ClassLibraryTemplate) get processed at the runtime. That's why it fails.
More information: Understand variable syntax
You could define variable or parameter in your YAML pipeline, then use template expression. For parameter, you could specify value when queue/run pipeline in pop-up window.
For example:
parameters:
- name: temName
displayName: template name
type: string
default: steps/test.yml
trigger:
- none
variables:
- name: tem
value: steps/build.yml
jobs:
- job: Linux
pool:
vmImage: 'ubuntu-16.04'
steps:
- template: ${{ variables.tem }}
- template: ${{ parameters.temName }}

How to pass a list of string parameters to an Azure DevOps template

In a previous SO question, I was trying to get create an Azure DevOps template.
Now that I know how to do that, I'm trying to pass in a list of strings to the template and use them in the template, like so:
azure_pipelines.yml
extends:
template: AzurePipelines/job.yml
parameters:
projects:
- FirstProject.csproj
- SecondProject.csproj
and in this template...
job.yml
steps:
(for each projectFileName in projects)
- bash echo $(projectFileName)
- run code in the build.yml template using $(projectFileName)
- run code in the test.yml template using $(projectFileName)
- run code in the publish.yml template using $(projectFileName)
So for each project name handed into the template, run the template steps.
I'm not sure how to pass in an array of strings and how to test that it works?
It seems you are looking for each function:
# my-template.yml
parameters:
- name: 'projects'
type: projectList
default: []
steps:
- ${{ each project in parameters.projects }}:
- task: PublishBuildArtifacts#1
displayName: 'Publish ${{ project }}'
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)/${{ project }}.zip'
# ci.yml
steps:
- template: my-template.yml
parameters:
projects:
- test1
- test2
The easiest way I found to accomplish the use of templates is by using the templates from within the main yaml file pipeline. This way you will have access to all the variables and variable groups defined in the main template. Using this architecture you will be able to bundle several steps into a template then run it from the main yaml file. Eg:
azure_pipelines.yml
variables:
- name: projectFileName
value: "Variable 1 value"
- name: VAR2
value: "Variable 2 value"
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
steps:
- template: templates/job.yml
- template: templates/test.yml
templates/job.yml
steps:
- task: Bash#3
displayName: 'Run Bash Command'
inputs:
targetType: 'inline'
script: |
print "Variable value: " $(projectFileName)
Hope it helps

Resources