I have a project using multiple repositories:
resources:
repositories:
- repository: one
type: git
name: repo/one
ref: release
- repository: two
type: git
name: repo/two
ref: develop
- repository: three
type: git
name: repo/three
ref: master
How do I get the environment variables for each of these repositories in subsequent steps? For example, I'm interested in the variable $(Build.BuildNumber). Now I only get the variable for the branch from which the yaml project itself is launched.
The repository keyword in resources lets you specify external repositories.
When you check out multiple repositories, some details about the self repository are available as Predefined variables
When you use multi-repo triggers, some of those variables have information about the triggering repository instead. Details about all of the repositories consumed by the job are available as a template context object called resources.repositories
To get the environment variables for each of these repositories you need to define variable:
Example:
resources:
repositories:
- repository: other
type: git
name: MyProject/OtherTools
variables:
tools.ref: $[ resources.repositories['other'].ref ]
steps:
- checkout: self
- checkout: other
- bash: |
echo "Tools version: $(tools.ref)"
https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/multi-repo-checkout?view=azure-devops#repository-details
Related
How to authorize variable in a yaml template in another repo to be used in a different repo. IOW, how to declare variables in a template once and use in multiple repos in azure devops
I am trying to migrate from classic pipelines to yaml in azure devops. So i am trying to set up a repo to host all yaml templates so it can be referenced and reused by multiple repos for builds, etc.
I wrote this yaml pipeline to sample prototyping it:
`name: FirstPL
trigger:
- my_test_branch
pool: my-agent
resources:
repositories:
- repository: blah
type: git
name: foo/bar
ref: refs/heads/poc
variables:
- template: pipeline_vars.yml#blah
steps:
- script: echo $(variable_from_pipeline_vars)
`
However when i run this i get the follwoing error:
An error occurred while loading the YAML build pipeline. Variable group was not found or is not authorized for use. For authorization details, refer to https://aka.ms/yamlauthz.
How can i declare my variables and variables groups once in a template in a repo that is dedicated to host those templates and then use them over and again in multiple repos using the resourcs syntax above? Also, I tried to find a way authorize the variables template but couldn't find anything to enable this.
How to authorize variable in a yaml template in another repo to be
used in a different repo. IOW, how to declare variables in a template
once and use in multiple repos in azure devops. However when I run
this i get the follwoing error:
An error occurred while loading the YAML build pipeline. Variable group was not found or is not authorized for use. For authorization
details, refer to
https://aka.ms/yamlauthz.
You can directly add the variable group in your azure DevOps project in the Library tab and save all your variables from pipeline_vars.yml in the variable group like below:-
Now, You can access this variable group in your YAML pipeline of multiple repos like the below:-
# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml
pool:
vmImage: ubuntu-latest
workspace:
clean: all
resources:
repositories:
- repository: repo_a
type: git
name: InternalProjects/repo_a
trigger:
- main
- release
- repository: repo_b
type: git
name: InternalProjects/repo_b
trigger:
- main
variables:
- group: SharedVariables
steps:
- checkout: repo_a
- checkout: repo_b
- script: |
echo $(databaseServerName)
- task: AzureCLI#2
inputs:
azureSubscription: 'xxx subscription(xxxxxxxxx-f598-44d6-b4fd-xxxxxxxxxxxx)'
scriptType: 'bash'
scriptLocation: 'inlineScript'
inlineScript: 'az resource list --location uksouth'
Output:-
It asks for approving permission for the Variable group to run in the pipeline like below:-
Console:-
Tried the same with another repo repo_b in the project and it asks to approve access for repositories and variable groups like the below:-
Output:-
If you want this variable to be accessed in multiple stages/repos/pipelines within the project without authorization prompt. You can click on Security on top and allow it:-
I created one variables template and referenced it in the YAML pipeline to use across multiple repos by checking out another repo like below:-
# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml
pool:
vmImage: ubuntu-latest
workspace:
clean: all
resources:
repositories:
- repository: repo_a
type: git
name: InternalProjects/repo_a
trigger:
- main
- release
- repository: repo_b
type: git
name: InternalProjects/repo_b
trigger:
- main
variables:
- template: pipeline_vars.yml
steps:
- checkout: repo_a
- checkout: repo_b
- script: |
echo $(environmentName)
- task: AzureCLI#2
inputs:
azureSubscription: 'xxx subscription(xxxxxxxx-f598-44d6-b4fd-e2b6e97xxxxxx)'
scriptType: 'bash'
scriptLocation: 'inlineScript'
inlineScript: 'az resource list --location uksouth'
Output:-
I tried to reference the same template in another repo where it does not exist it could not read the pipeline_vars.yml file as it does not exist in the repo.
You can make use of variable groups like the above to reference the variables in this pipeline.
One of the possible reasons for this is that the project that hosts the repository with the variables does not allow access to it's repositories from yaml pipelines.
To verify, go to your project's settings -> Pipelines -> Settings -> Verify "Protect access to repositories in YAML pipelines" . This setting is enabled by default. You could set it to off or add a checkout step to your pipeline yaml. See here for more information.
I am trying to checkout repository in Azure pipeline which is different then self repo but in same Organization. Here repository name and project name will be passed as input parameter .
I have tried by following example in https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/multi-repo-checkout?view=azure-devops but not able to pass with parameter.
I have tried with syntex as below, but not get success.
resources:
repositories:
- repository: MyAzureReposGitRepository
type: git
name: $(project)/$(repo)
ref: $(branch)
Also have tried like
- checkout: git://${{ variables.repoName}}#${{ variables.branchRef }}
But here getting error at time of running pipeline
The String must have at least one character. Parameter name:repositoryName
Please help if you have any other way to make it success.
Checkout different Repository as per input in Azure Pipeline
According to this thread Pipeline resource Version property as a variable:
While we can’t allow variables in that field, this is an excellent use case for runtime parameters.
So, we could not use variable $(project)/$(repo) in the resources.
To resolve this issue, we could use the Checking out a specific ref:
parameters:
- name: ProjectName
displayName: Project Name
type: string
default: LeoTest
values:
- LeoTest
- name: repoName
displayName: repo Name
type: string
default: TestRepo
values:
- TestRepo
- name: branchRef
displayName: Branch Name
type: string
default: Dev
values:
- Dev
And
- checkout: git://${{ parameters.ProjectName}}/${{ parameters.repoName}}#refs/heads/${{ parameters.branchRef}}
The test result:
Since you are talking about parameters I'm assuming you are using templates. I was able to achieve the desired result with the following code
# File: template.yml
parameters:
- name: project
type: string
- name: repo
type: string
- name: branch
type: string
stages:
- stage: A
displayName: Checkout
jobs:
- job: Checkout
steps:
- checkout: git://${{ parameters.project }}/${{ parameters.repo }}#${{ parameters.branch }}
# File: pipeline.yml
extends:
template: template.yml
parameters:
project: ProjectName
repo: RepoName
branch: BranchName
I have one pipeline that is referencing several templates with resources and I know you can extend a template with resources by using the extend keyword. I've been looking at this documentation trying to make it work.
azure-pipeline.yaml
trigger: none
resources:
repositories:
- repository: repoName
type: git
name: project/repoName
- repository: 'Release Notes'
name: 'project/Release Notes'
type: git
pipelines:
- pipeline: ooi-adf-ci
source: ooi-adf-ci
extends:
- {template: '/cicd/pipelines/templates/stages-deploy-app-registration.yaml'}
- {template: '/cicd/pipelines/templates/stages-set-app-credentials.yaml'}
- {template: '/cicd/pipelines/templates/stages-buying-release-apps.yaml'}
- {template: '/cicd/pipelines/templates/stages-buying-adf.yaml'}
I also tried something like:
extends:
template:
[
'/cicd/pipelines/templates/stages-deploy-app-registration.yaml',
'/cicd/pipelines/templates/stages-set-app-credentials.yaml',
'/cicd/pipelines/templates/stages-buying-release-apps.yaml',
'/cicd/pipelines/templates/stages-buying-adf.yaml',
]
template.yaml where the resource is consumed:
...
resources:
pipelines:
- pipeline: ooi-adf-ci
source: ooi-adf-ci
...
I get an error in my azure-pipeline.yaml file saying "A sequence was not expected". I'm wondering if this is actually possible as I haven't been able to find any documentation on it or if I'm just incorrectly doing it.
I dont think you can infinitely extend templates at the root level, as in your example.
What you can do is compose a azure-pipeline.yaml file where each job / step is from a template. I do this a lot and works perfectly.
Here is the official docs:
enter link description here
I could not find any references to this topic: Is it possible to use some kind of wildcards in order to always get the latest (non-breaking) repository in a pipeline?
resources:
repositories:
- repository: my_repo
type: git
name: myProject/repo
ref: refs/tags/version-1.3.3 # something like version-1.3.* would be cool
I'm setting up several pipelines in Azure DevOps. To make my teams life easier, I'm using job templates.
These job templates are in a a proper repository, just for them.
For every pipeline I define the repository to get the templates from.
Some tasks in these templates run powershell code, and I want this code to be in a script file, to be reusable and stored in the same repo as the template.
When the pipelines runs, the template is embeded, it tries to locate the powershell script inside project repo actually being built/deployed.
How can i achieve this?
The workaround is to have inline code which I really don't want to have.
Any constructive answer will be very appreciated.
Thanks
After some digging I couldn't find any way to specify a script file as source to powershell task in a template.
Inside pipeline definition:
resources:
repositories:
- repository: templates
type: git
name: deploy-templates
variables:
artifactName: 'Trade Data ETL - $(Build.SourceBranchName)'
stages:
- stage: Build
displayName: Build
variables:
- group: DEV-Credential-Group
- group: COMMON-Settings-Group
jobs:
- template: ssis/pipelines/stage-build.yml#templates # Template reference
parameters:
artifactName: '$(artifactName)'
Inside template file:
- task: PowerShell#2
inputs:
filePath: ssis/pipelines/scripts/build-ssis-project.ps1
arguments: '-ProjectToBuild "tradedata-ldz-ssis/tradedata-ldz-ssis.dtproj'
pwsh: true
Update 2021
According to learn.microsoft.com, you can now also check out multiple repositories without custom scripting.
If you check out more than one repository, a separate folder containing the repository is created below $(Build.SourcesDirectory).
You can define multiple repositories like this:
resources:
repositories:
- repository: devops
type: git
name: DevOps
ref: main
- repository: infrastructure
type: git
name: Infrastructure
ref: main
And in the steps simply check them out as follows:
steps:
- checkout: self
- checkout: devops
- checkout: infrastructure
# List all available repositories
- script: ls
Original Answer
Currently the resources command only supports yml files in other repositories. However, you could simply checkout the repository in a task and then run the desired powershell script.
steps:
- task: PowerShell#2
inputs:
targetType: inline
script: |
git clone -b <your-desired-branch> https://azuredevops:$($env:token)#dev.azure.com/<your-organization>/<your-project>/_git/<your-repository> <target-folder-name>
./<target-folder-name>/foo.ps1
env:
token: $(System.AccessToken)
This script would checkout an arbitrary branch and execute a script foo.ps1 in the root of the target repository.
Call - checkout: templates inside the template file. This might only work when you insert a template but it successfully sees the repository resource and pulls it down.
You can copy the script files from source directory. Currently, you have not mentioned the root folder -
ssis/pipelines/scripts/build-ssis-project.ps1
Assuming, you are building on a repo where the powershell script resides -
Try -
- task: PowerShell#1
inputs:
scriptName: '$(ScriptsDir)/ssis/pipelines/scripts/build-ssis-project.ps1'
Pass the value of ScriptsDir where it could be the build source directory or build working directory