I'm trying to edit an ADO YAML file down to the bare minimum in order to isolate another issue.
When I run Validate, it comes back with the following error:
No repository found by name templates
Here's the general gist of my YAML:
#resources:
# repositories:
# - repository: templates
# type: git
# name: TemplateProject/TemplateRepo
name: $(VersionName)
trigger:
branches:
include:
- main
batch: true
paths:
exclude: $(ListOfExclusions)
stages:
- template: core/setVersion.yml#templates
- stage: Build
pool: linux
jobs:
- job: BuildDocker
displayName: Build and Push Docker Image
pool: linux
steps:
- task: Docker#2
displayName: Build and push an image to container registry
inputs:
command: buildAndPush
repository: $(RepoName)
dockerfile: $(Build.SourcesDirectory)/Dockerfile
containerRegistry: $(dockerRegistryServiceConnection)
tags: |
$(Tag)
What could be going wrong? The error message makes me think the YAML isn't clean.
It turns out I caused a simple typo when commenting out the resources section of the YAML. I had a template part of the stage that also needed to be commented out, and I neglected to do this.
Once I updated the code to read:
stages:
# - template: core/setVersion.yml#templates
- stage: Build
pool: linux
jobs:
- job: BuildDocker
# etc...
Now my YAML validates with OK.
Related
I want to use a remote repo template in my azure pipeline. But I want to skip some of the steps included.
Note: I dont have access to configure the repote pipeline steps.
The remote yml looks like this
build.yml
steps:
- download: none
- checkout: ${{ parameters.checkoutRepo }}
- task: Cache#2
displayName: Cache Maven local repo
- task: Maven#3
displayName: 'Maven: validate'
- task: SonarQubePrepare#4
displayName: 'Prepare analysis on SonarQube'
My yml my-build.yml
resources:
repositories:
- repository: my-remote-repo-above
type: git
name: my-remote-repo-above
.
.
.
stages:
- template: build-stage.yml
parameters:
So my question is, can I somehow specify steps from remote to skip OR there is a way to pick the ones I want to execute only?
Basically, we can define a parameter to your template, and use it in a task condition so that we can skip the specific steps based on the condition.
For example, in build-stage.yml
parameters:
enableSonarCloud: false
steps:
- download: none
- checkout: ${{ parameters.checkoutRepo }}
- task: Cache#2
displayName: Cache Maven local repo
- task: Maven#3
displayName: 'Maven: validate'
- task: SonarQubePrepare#4
condition: and(succeeded(), ${{ parameters.enableSonarCloud }} )
displayName: 'Prepare analysis on SonarQube'
And your build.yml
resources:
repositories:
- repository: my-remote-repo-above
type: git
name: my-remote-repo-above
.
.
.
stages:
- template: build-stage.yml
parameters:
enableSonarCloud: true
But in your scenario, you don't have access to configure the remote template steps. In this case, I don't think you can achieve that if the parameter and task condition is not defined in the template.
Ultimately, I'm trying to do this:
Move azure-pipelines.yaml and associated templates out of the code repository (code-repo).
Move them into a separate dedicated repository (pipeline-repo).
Have the pipeline look at the config for the pipeline in pipeline-repo, but run the pipeline on the code in the code-repo.
I'm referring the following documentation:
Use other repositories: this one refers to "templates in other repositories," but I'm trying to remove any pipeline configs so the code-repo is just purely application code... and the Dockerfile.
Define a repositories resource
For testing, I have this simple test.yaml:
# Triggers when PR is created due to branch policies
trigger: none
resources:
repositories:
- repository: code-repo
type: git
name: code-repo
pool:
vmImage: 'ubuntu-latest'
stages:
- stage: Testing
displayName: Test stage
jobs:
- job: ParallelA
steps:
- bash: echo Hello from parallel job A
displayName: 'Run a one-line script'
When I create a PR on code-repo, it is triggering the pipeline, which is to say branch policies are configured to refer to that pipeline. I do get the print out the Hello from parallel job A print out.
But I don't see in the run logs it pulling code-repo.
I do see the following, however:
My actual PR pipeline would look something like this:
trigger: none
resources:
repositories:
- repository: code-repo
type: git
name: code-repo
variables:
- template: templates/variables.yaml
pool:
vmIMage: $(vmImageName)
stages:
- template: templates/build/buildStage.yaml
...
Testing that, it confirms that it isn't running on the code-repo PR, but the pipeline-repo so everything fails.
So it is unclear to me what I need to do from here to get the pipeline to run on the PR code from code-repo.
Suggestions?
Ok, I think I have it sorted out, at least some of my stages are now succeeding.
I came across this documentation which informed me of checkout.
So in addition to doing something like:
resources:
repositories:
- repository: code-repo
type: git
name: code-repo
Then you need to add a step called checkout like the following:
# Triggers when PR is created due to branch policies
trigger: none
resources:
repositories:
- repository: code-repo
type: git
name: code-repo
pool:
vmImage: 'ubuntu-latest'
stages:
- stage: Testing
displayName: Test stage
jobs:
- job: ParallelA
steps:
- checkout: code-repo
- task: task1
- task: task2
The checkout should set the context for the subsequent steps.
When I locally build Dockerfile it finishes in few seconds while on the Azure it takes few minutes.
I know that there is new Cache#2 task, but I'm not sure how to implement it into my build.yml file and I didn't found any similar example using it.
Here is my .yml file without caching:
trigger:
branches:
include:
- main
pool:
vmImage: 'ubuntu-latest'
steps:
- checkout: self
clean: true
submodules: false
- task: Docker#2
inputs:
containerRegistry: 'MyDockerHub'
repository: 'myRepo/myservice'
command: 'buildAndPush'
Dockerfile: './MyService/Dockerfile'
buildContext: './'
tags: |
$(Build.BuildId)
latest
Can this be cached in order to improve build speed ?
With the below setup, when my pipeline corresponding to my repo runs, it runs the template (template.yml) file belonging to 'anotherRepo'. But when it checks out, it checks out my repo instead of 'anotherRepo'.
Is there any issue in my setup?
Looks like checkout:self does not have any impact and it does not work
My current Repo:
azurepipeline.yml file:
variables:
acceptanceTestsRepoName: 'anotherRepo'
resources:
repositories:
- repository: 'anotherRepo'
name: ProjectName/anotherRepo
type: git
ref: master
stages:
- stage: acceptance_tests
displayName: 'Run Acceptance Tests in Dev'
jobs:
- template: 'azure-pipelines-templates/template.yml#${{variables.acceptanceTestsRepoName}}'
Repo:anotherRepo
template.yml
jobs:
- job: AcceptanceTest
displayName: Run Acceptance Test in Dev
pool:
vmImage: 'ubuntu-latest'
steps:
- checkout: self
- task: UsePythonVersion#0
inputs:
versionSpec: '$(python.version)'
self always refers to a repo associated with the build pipeline. In your case, you need to checkout anotherRepo manually:
# Azure Repos Git repository in the same organization
- checkout: git://anotherRepo
This assumes that anotherRepo is in the same Azure DevOps organization. If it's not or stored somewhere else (GitHub, Bitbucket, etc...) you also need to add it as a resource to the pipeline defintion. See Check out multiple repositories in your pipeline for details.
According to the document about checkout: self represents the repo where the initial Azure Pipelines YAML file was found.
So your pipeline checkout the repo where your azurepipeline.yml file is located.
If you want to checkout your anotherRepo, the checkout step in your template.yml should be - checkout: anotherRepo:
jobs:
- job: AcceptanceTest
displayName: Run Acceptance Test in Dev
pool:
vmImage: 'ubuntu-latest'
steps:
- checkout: anotherRepo
- task: UsePythonVersion#0
inputs:
versionSpec: '$(python.version)'
You can also use Inline syntax checkout to directly check out another repo in azure-pipelines.yml file:
stages:
- stage: acceptance_tests
displayName: 'Run Acceptance Tests in Dev'
jobs:
- job: checkout
steps:
- checkout: git://ProjectName/anotherRepo
Thanks all for your response expecially #beatcracker
By replacing the checkout step from self , I was able to run successfully
No change to My Current Repo.
Below change done to another repo
jobs:
- job: AcceptanceTest
displayName: Run Acceptance Test in Dev
pool:
vmImage: 'ubuntu-latest'
steps:
- checkout: git://ProjectName/anotherRepo
- task: UsePythonVersion#0
inputs:
versionSpec: '$(python.version)'
I have two .yml files in my repo. One for build, one for deployment. The main reason why I would like to keep build separate from the deployment is that I also would like to store variables for environments in my repo, e.i. in variables-dev.yml and variables-prod.yml files. So there is no need to create a new build every time (which includes running tests, docker image build etc.).
The file build.yml:
trigger:
paths:
exclude:
- build.yml
- deploy.yml
stages:
- stage: build
jobs:
...
And the deploy.yml, which I want to be triggered only on the completion of the build pipeline. That's why I add the first exclusion of all paths, but add one on pipeline resource.
trigger:
paths:
exclude:
- '*'
resources:
pipelines:
- pipeline: build
source: build
trigger:
branches:
include:
- '*'
stages:
- stage: dev
variables:
- template: variables-dev.yml
jobs:
- deployment: deploy_dev
environment: 'dev'
strategy:
runOnce:
deploy:
steps:
...
- stage: prod
dependsOn: dev
variables:
- template: variables-prod.yml
jobs:
- deployment: deploy_prod
environment: 'prod'
strategy:
runOnce:
deploy:
steps:
...
Unfortunately it does not seem to work. The top trigger blocks lower trigger. And if I remove the top trigger than the deploy pipeline is triggered at the same time with the build one.
you have to start your deploy.yml with trigger: none
trigger: none
resources:
pipelines:
- pipeline: ci-pipeline
source: my-build-pipeline
trigger:
enabled: true
branches:
include:
- master
Set your triggers for the second yml to none, then add this setting in the "Triggers" section of the UI. It will stage your builds as you describe