I am trying to checkout repository in Azure pipeline which is different then self repo but in same Organization. Here repository name and project name will be passed as input parameter .
I have tried by following example in https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/multi-repo-checkout?view=azure-devops but not able to pass with parameter.
I have tried with syntex as below, but not get success.
resources:
repositories:
- repository: MyAzureReposGitRepository
type: git
name: $(project)/$(repo)
ref: $(branch)
Also have tried like
- checkout: git://${{ variables.repoName}}#${{ variables.branchRef }}
But here getting error at time of running pipeline
The String must have at least one character. Parameter name:repositoryName
Please help if you have any other way to make it success.
Checkout different Repository as per input in Azure Pipeline
According to this thread Pipeline resource Version property as a variable:
While we can’t allow variables in that field, this is an excellent use case for runtime parameters.
So, we could not use variable $(project)/$(repo) in the resources.
To resolve this issue, we could use the Checking out a specific ref:
parameters:
- name: ProjectName
displayName: Project Name
type: string
default: LeoTest
values:
- LeoTest
- name: repoName
displayName: repo Name
type: string
default: TestRepo
values:
- TestRepo
- name: branchRef
displayName: Branch Name
type: string
default: Dev
values:
- Dev
And
- checkout: git://${{ parameters.ProjectName}}/${{ parameters.repoName}}#refs/heads/${{ parameters.branchRef}}
The test result:
Since you are talking about parameters I'm assuming you are using templates. I was able to achieve the desired result with the following code
# File: template.yml
parameters:
- name: project
type: string
- name: repo
type: string
- name: branch
type: string
stages:
- stage: A
displayName: Checkout
jobs:
- job: Checkout
steps:
- checkout: git://${{ parameters.project }}/${{ parameters.repo }}#${{ parameters.branch }}
# File: pipeline.yml
extends:
template: template.yml
parameters:
project: ProjectName
repo: RepoName
branch: BranchName
Related
I want to use reusable workflows and invoke two other actions, below is the snippet of both caller and called workflows:
az-dev-img-ops-reusable.yml (called workflow1)
on:
workflow_call:
inputs:
cloudProvider:
required: true
type: string
imgAction:
required: true
type: string
envName:
required: true
type: string
productImage:
required: true
type: string
jobs:
image-actions:
runs-on: self-hosted
steps:
- uses: actions/checkout#v2
- uses: ./.github/workflows/manage_images
with:
imgAction: ${{ inputs.imgAction }}
cloudProvider: ${{ inputs.cloudProvider }}
envName: ${{ inputs.envName }}
productImage: ${{ inputs.productImage }}
az-dev-tenant-ops-reusable.yml (called workflow2)
on:
workflow_call:
inputs:
cloudProvider:
required: true
type: string
envName:
required: true
type: string
action:
required: true
type: string
jobs:
tenant-actions:
runs-on: self-hosted
steps:
- uses: actions/checkout#v2
- uses: ./.github/workflows/refresh_tenant
with:
cloudProvider: azure
envName: az-dev
action: pod_refresh
az-dev-cicd.yml (caller workflow)
on:
workflow_dispatch:
jobs:
init:
runs-on: self-hosted
steps:
- uses: actions/checkout#v2
image-operations:
uses: ./.github/workflows/az-dev-img-ops-reusable.yml
with:
imgAction: check
cloudProvider: azure
envName: az-dev
productImage: my-server
tenant-operations:
needs: image-operations
uses: ./.github/workflows/az-dev-tenant-ops-reusable.yml
with:
cloudProvider: azure
envName: az-dev
action: pod_refresh
But while running, it throws error as below:
Can't find 'action.yml', 'action.yaml' or 'Dockerfile' under '/opt/my-runner/_work/my-images/my-images/.github/workflows/manage_images'. Did you forget to run actions/checkout before running your local action?
Checked over the web for solutions and tried all i.e.
Have a actions/checkout.
Make sure the caller function makes use of "uses" and not steps.
Tried both absolute path of repo/yaml and relative path.
But haven't been able to resolve it. Any hints/help would be highly appreciated.
Your issue is that you used the following syntax:
uses: ./.github/workflows/manage_images
... that refers to a directory.
When used that way, the Github Action interpreter expect a local action (using an action in the same repo).
However, to use local actions, you need to create a action.yaml file in a directory, but in your case, the image you shared in the comment shows that you actually have a manage_images.yaml file, that refers to a workflow, not a manage_images folder with an action.yml file in it.
To resolve your issue, if this manage_images.yaml is a reusable workflow (reusing the same workflow in different workflows), you need to add .yaml at the end of uses step, like this:
uses: ./.github/workflows/manage_images.yaml
Or, if you actually want to use a local action, convert this manage_images.yaml workflow to an action.yaml file, and put it inside a .github/workflows/manage_images directory.
Note that this answer could also give you more context about local actions and reusable workflows.
I have a project using multiple repositories:
resources:
repositories:
- repository: one
type: git
name: repo/one
ref: release
- repository: two
type: git
name: repo/two
ref: develop
- repository: three
type: git
name: repo/three
ref: master
How do I get the environment variables for each of these repositories in subsequent steps? For example, I'm interested in the variable $(Build.BuildNumber). Now I only get the variable for the branch from which the yaml project itself is launched.
The repository keyword in resources lets you specify external repositories.
When you check out multiple repositories, some details about the self repository are available as Predefined variables
When you use multi-repo triggers, some of those variables have information about the triggering repository instead. Details about all of the repositories consumed by the job are available as a template context object called resources.repositories
To get the environment variables for each of these repositories you need to define variable:
Example:
resources:
repositories:
- repository: other
type: git
name: MyProject/OtherTools
variables:
tools.ref: $[ resources.repositories['other'].ref ]
steps:
- checkout: self
- checkout: other
- bash: |
echo "Tools version: $(tools.ref)"
https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/multi-repo-checkout?view=azure-devops#repository-details
In my azure devops project, I have created a pipeline using template. This is the main yaml files for my build pipeline
name: Test-$(Date:yyyyMMdd)$(Rev:.r)
resources:
repositories:
- repository: api
type: git
name: porject/api
ref: master
- repository: front
type: git
name: project/front
ref: master
- repository: strapi
type: git
name: project/strapi
ref: master
trigger:
branches:
include:
- master
pool:
vmImage: 'ubuntu-latest'
workspace:
clean: all
variables:
- name: workingDir
value: project
- name: tfVersion
value: 0.12.29
- name: backendServiceGCP
value: test
- name: backendGCPBucketName
value: test
- name: tfpath
value: test
- name: env
value: dev
stages:
- stage: Terraform
jobs:
- job: Build
displayName: Build Terraform Infra
steps:
# Set and Export env var for api version to deploy
- template: templates/fetch-tag.yml
parameters:
repo: 'api'
envVar: TERRAFORM_API_TAG
# Set and Export env var for front version to deploy
- template: templates/fetch-tag.yml
parameters:
repo: 'front'
envVar: TERRAFORM_FRONT_TAG
# Set and Export env var for strapi version to deploy
- template: templates/fetch-tag.yml
parameters:
repo: 'strapi'
envVar: TERRAFORM_STRAPI_TAG
# Init Terraform providers
- template: templates/tf-init.yml
parameters:
backendServiceGCP: '$(backendServiceGCP)'
backendGCPBucketName: '$(backendGCPBucketName)'
workingDir: '$(workingDir)'
variableFilePath: $(buildSubscription)-common.tfvars
# Plan Terraform Infra to Deploy
- template: templates/tf-plan.yml
parameters:
backendServiceGCP: '$(backendServiceGCP)'
workingDir: '$(workingDir)'
variableFilePath: $(buildSubscription)-common.tfvars
# Publish Public Artifact with Terraform ressources to deploy
- template: templates/publish-artifact.yml
parameters:
tfpath: '$(tfpath)'
When I am trying to run the pipeline I have the following error:
Encountered error(s) while parsing pipeline YAML:
Job Build: The step name version appears more than once. Step names must be unique within a job.
Job Build: The step name version appears more than once. Step names must be unique within a job.
Job Build: The step name version appears more than once. Step names must be unique within a job.
Job Build: The step name version appears more than once. Step names must be unique within a job.
Job Build: The step name version appears more than once. Step names must be unique within a job.
I don't really understand why.
This is an example of templates I am using in the pipeline named publish-artifact.yml:
parameters:
tfPath: ''
steps:
- task: CopyFiles#2
inputs:
sourceFolder: ${{ parameters.tfpath }}
contents: |
tfplan
**/*.tf
**/*.tfvars
**/*.json
!**/.terraform
**/*.sh
targetFolder: $(Build.ArtifactStagingDirectory)
- task: PublishBuildArtifacts#1
inputs:
pathtoPublish: $(Build.ArtifactStagingDirectory)
artifactName: tf
What am I doing wrong with the job name ?
Azure Devops supports passing output variable from one step as next step's inputs. See this ticket:
We can name a step like this:
steps:
- script: echo test
name: ScriptName
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
Write-Host '##vso[task.setvariable variable=xxx;isOutput=true;]xxx'
name: PSName
The name must be unique so that we can use format $(referencename.variablename) to access output variable from specific step.
The error indicates that some steps in your templates have the same name version! And this is not supported. About why this issue occurs:
1.You called the same template several times, this is the main cause of your issue.
Devops expand templates first when processing the pipeline, so if your fetch-tag template has one step named version, the final expended azure-pipeline.yml would be:
stages:
- stage: Terraform
jobs:
- job: Build
displayName: Build Terraform Infra
steps:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
Write-Host "Hello World"
name: version
...
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
Write-Host "Hello World"
name: version
...
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
Write-Host "Hello World"
name: version
...
2.You may also need to check if there're steps with same name in your different templates.
We can call same template more than once in one pipeline, but we can't call same templates in which there's named step. Cause the pipeline will expand the templates more than once and the final pipeline would contain many steps with same name. And this is not supported since the name should be unique.
Solution:
1.Remove the name element of your steps if you don't need to use output variables mentioned above.
2.Or you can make several copies of fetch-tag.yml and name them fetch-tag-api.yml, fetch-tag-front.yml and fetch-tag-strapi.yml. Rename the referenceName version in these three files to version1, version2 or what. Then you can run the pipeline with:
steps:
# Set and Export env var for api version to deploy
- template: templates/fetch-tag-api.yml
parameters:
repo: 'api'
envVar: TERRAFORM_API_TAG
# Set and Export env var for front version to deploy
- template: templates/fetch-tag-front.yml
parameters:
repo: 'front'
envVar: TERRAFORM_FRONT_TAG
# Set and Export env var for strapi version to deploy
- template: templates/fetch-tag-strapi.yml
parameters:
repo: 'strapi'
envVar: TERRAFORM_STRAPI_TAG
Please consider the following:
- job: Backend
steps:
- template: $(ClassLibraryTemplate)
parameters:
projectName: 'Core'
solutionPath: 'Source/Core.sln'
ClassLibraryTemplate is defined as a pipeline variable. But when I run the build, it fails because the variable is not replaced by its value and the template is not found.
Is it not possible to store the template name in a variable ?
For Azure DevOps YAML pipeline, the template get processed at compile time. However, the $(ClassLibraryTemplate) get processed at the runtime. That's why it fails.
More information: Understand variable syntax
You could define variable or parameter in your YAML pipeline, then use template expression. For parameter, you could specify value when queue/run pipeline in pop-up window.
For example:
parameters:
- name: temName
displayName: template name
type: string
default: steps/test.yml
trigger:
- none
variables:
- name: tem
value: steps/build.yml
jobs:
- job: Linux
pool:
vmImage: 'ubuntu-16.04'
steps:
- template: ${{ variables.tem }}
- template: ${{ parameters.temName }}
I am trying to use extends as part of my pipeline. I am trying to get the first basic step working from Azure docs ie
# pythonparameter-template.yml
parameters:
- name: usersteps
type: stepList
default: []
steps:
- ${{ each step in parameters.usersteps }}
- ${{ step }}
# azure-pipelines.yml
trigger: none
resources:
repositories:
- repository: CI-CD-Templates
type: git
name: /CI-CD-Templates
ref: refs/heads/master
extends:
template: /pythonparameter-template.yml#CI-CD-Templates
parameters:
usersteps:
- script: echo This is my first step
- script: echo This is my second step
I keep getting the below error:
The directive 'each' is not allowed in this context. Directives are not supported for expressions that are embedded within a string. Directives are only supported when the entire value is an expression
Unexpected value '${{ each step in parameters.usersteps }} - ${{ step }}'
Also after I extend from a template can azure-pipelines.yml also have it's own custom steps ie
# azure-pipelines.yml
resources:
repositories:
- repository: templates
type: git
name: MyProject/MyTemplates
ref: tags/v1
extends:
template: template.yml#templates
parameters:
usersteps:
- script: echo This is my first step
- script: echo This is my second step
steps:
- template: /validation-template.yml#CI-CD-Templates
parameters:
commitMessage: $(commitMessage)
- template: /shared-template.yml#CI-CD-Templates
parameters:
buildArtifactDir: $(buildArtifactDir)
Update
Please refer the response in this DC link-- Yaml extends feature erroring out when looping through steps.
The YAML has a validation task in #pythonparameter-template.yml
Comment out these 2 lines and your YAML will succeed. The template shown here is preventing any tasks to be used. This could be a scenario for someone with specific security requirements.
${{ if eq(pair.key, 'task') }}:
'${{ pair.value }}': error
Are you trying to combine two yml pythonparameter-template.yml azure-pipelines.yml in the same file? It's not supported.
parameters:
- name: usersteps
type: stepList
default: []
steps:
- ${{ each step in parameters.usersteps }}
- ${{ step }}
According to the error Directives are not supported for expressions that are embedded within a string. Directives are only supported when the entire value is an expression Unexpected value '${{ each step in parameters.usersteps }} - ${{ step }}'
You maybe use the wrong format. About the formats you could refer our official doc here--
Template types & usage
Besides, you can make azure-pipelines.yml also have it's own custom steps. But you need to put them under parameters in the pipeline, not like the way you use.
azure-pipelines.yml
trigger:
- master
extends:
template: pythonparameter-template.yml
parameters:
buildSteps:
- bash: echo Test #Passes
displayName: succeed
- bash: echo "Test"
displayName: succeed
- script: echo "Script Test"
displayName: Fail