Azure Devops: how configure my pipeline script to trigger another pipeline with different repo? - azure

I have my own pipeline p-A with repository r-A, and pipeline p-B with another repo r-B.
I want to update the pipeline script for p-A only, to trigger the p-B actively, without any modification in p-B.
Below is the yaml pipeline script for p-B, which is already set up to run with schedule
pool:
name: 'workflow_test_pool'
schedules:
- cron: "0 19 * * *"
displayName: run test every day at 8PM CET
branches:
include:
- main
always: true
trigger: none
jobs:
- job:
timeoutInMinutes: 30
steps:
- script: |
python -m pytest tests/ -s
displayName: 'Run the test'
below is the pipeline script main.yaml for p-A
pool:
name: 'workflow_test_pool'
stages:
#########################
- template: pipeline2/p1.yaml
############################
- template: pipeline2/p2.yaml
parameters:
dependsOn:
- FirstPipeline
so the question is, how to trigger the pipeline p-B in pipeline2/p2.yaml(from p-A)?

You can create PowerShell script task as the last step of the pipeline to trigger pipeline p-B through REST API. You will have to maintain Personal Access Token, ideally as secret variable.
REST API call you will use:
https://learn.microsoft.com/en-us/rest/api/azure/devops/pipelines/runs/run-pipeline?view=azure-devops-rest-7.1
Detailed step-by-step guide:
https://blog.geralexgr.com/cloud/trigger-azure-devops-build-pipelines-using-rest-api

Azure DevOps supports multiple repositories checkout, you can just reference resources function in your YAML script and call another repository to trigger from the pipeline.
YAML code :-
# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml
pool:
vmImage: ubuntu-latest
workspace:
clean: all
resources:
repositories:
- repository: repo_a
type: git
name: InternalProjects/repo_a
trigger:
- main
- release
- repository: repo_b
type: git
name: InternalProjects/repo_b
trigger:
- main
steps:
- checkout: repo_a
- checkout: repo_b
- script: dir $(Build.SourcesDirectory)
I am running this pipeline from repo_a and the repo_a and repo_b both ran successfully like below:-
Output :-
You can directly run any task from pipeline with multiple repositories like below:-
# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml
pool:
vmImage: ubuntu-latest
workspace:
clean: all
resources:
repositories:
- repository: repo_a
type: git
name: InternalProjects/repo_a
trigger:
- main
- release
- repository: repo_b
type: git
name: InternalProjects/repo_b
trigger:
- main
steps:
- checkout: repo_a
- checkout: repo_b
- task: AzureCLI#2
inputs:
azureSubscription: 'Subscription-name(sub-id)'
scriptType: 'bash'
scriptLocation: 'inlineScript'
inlineScript: 'az resource list --location uksouth'
Output:-
References :-
Check out multiple repositories in your pipeline - Azure Pipelines | Microsoft Learn
Trigger azure Devops pipeline from another repository - GeralexGR
Multiple Repositories in a Single Azure Pipeline - DEV Community 👩‍💻👨‍💻

Related

Run two stages in Azure DevOps Pipeline "partially" parallel

I have two stages in my Azure DevOps pipeline. One with Pulumi Preview (let's call it Preview) and one with Pulumi Up (Up) in order to run my infrastructure as code.
Both run from the same container and it takes a while to pull it. I want to manually approve the Preview before the implementation.
Can I pull and run the container for both stages simultaneously but wait for the last job of the UP-Stage until the Preview-Stage is approved?
Currently they depend on eachother as follows:
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
stages:
- stage: Pulumi_Preview
jobs:
- job: Preview
container:
image: REGISTRY.azurecr.io/REPOSITORY:latest
endpoint: ServiceConnection
steps:
- task: Pulumi#1
displayName: pulumi preview
inputs:
azureSubscription: 'Something'
command: 'preview'
args: '--diff --show-config --show-reads --show-replacement-steps'
stack: $(pulumiStackShort)
cwd: "./"
- stage: Pulumi_Up
displayName: "Pulumi (Up)"
dependsOn: Pulumi_Preview
jobs:
- job: Up
container:
image: REGISTRY.azurecr.io/REPOSITORY:latest
endpoint: ServiceConnection
steps:
- task: Pulumi#1
inputs:
azureSubscription: 'Something'
command: 'up'
args: "--yes --skip-preview"
stack: $(pulumiStackShort)
cwd: "./"
You could use the Manual Validation Task.
Use this task in a YAML pipeline to pause a run within a stage, typically to perform some manual actions or validations, and then resume/reject the run.
jobs:
- job: waitForValidation
displayName: Wait for external validation
pool: server
timeoutInMinutes: 4320 # job times out in 3 days
steps:
- task: ManualValidation#0
timeoutInMinutes: 1440 # task times out in 1 day
inputs:
notifyUsers: |
test#test.com
example#example.com
instructions: 'Please validate the build configuration and resume'
onTimeout: 'resume'

Azure Devops Trigger a Pipeline from Multiple Stages of Other Pipeline

I'm trying to integrate a QA pipeline in Azure DevOps that triggers during a dev pipeline. The dev pipeline has 4 environments that the build is deploying to, where each environment is a stage in the dev pipeline
Currently, I'm able to trigger the QA builds for each environment using 4 separate QA pipelines using syntax similar to this in each pipeline:
resources:
pipelines:
- pipeline: Dev_Env_1
source: Dev
trigger:
stages:
- Env_1
My goal is to only have one QA pipeline that is triggered multiple times by the dev pipeline when it completes each stage. It feels like syntax in the yml file like this should work:
resources:
pipelines:
- pipeline: Dev_Env_1
source: Dev
trigger:
stages:
- Env_1
- pipeline: Dev_Env_2
source: Dev
trigger:
stages:
- Env_2
However, this only triggers after Env_1 is completed, when I would like a build triggered for the completion of the Env_1 and Env_2 stages in the Dev pipeline.
Is there a way to do this without drastically changing the way either pipeline currently works?
The below steps can help you achieve your requirements.
For example,
Dev Pipeline
#Dev Pipeline
trigger:
- none
pool:
vmImage: 'windows-latest'
stages:
- stage: Env_1
displayName: Env_1
jobs:
- job:
steps:
- task: CmdLine#2
inputs:
script: |
echo Stage1
- stage: Env_2
displayName: Env_2
jobs:
- job:
steps:
- task: CmdLine#2
inputs:
script: |
echo Stage2
- stage: Env_3
displayName: Env_3
jobs:
- job:
steps:
- task: CmdLine#2
inputs:
script: |
echo Stage3
- stage: Env_4
displayName: Env_4
jobs:
- job:
steps:
- task: CmdLine#2
inputs:
script: |
echo Stage4
1, Create an incoming webhook service connection, and an incoming service webhook that is triggered by the complete and success of stages of the Dev Pipeline.
incoming service hook URI: https://dev.azure.com/<ADO Organization>/_apis/public/distributedtask/webhooks/<WebHook Name>?api-version=6.0-preview
Official document:
https://learn.microsoft.com/en-us/azure/devops/release-notes/2020/pipelines/sprint-172-update#generic-webhook-based-triggers-for-yaml-pipelines
After that, write QA pipeline as below:
QA Pipeline
# QA pipeline
trigger:
- none
resources:
webhooks:
- webhook: bowmantest
connection: BowmanIncommingWebHook
pool:
vmImage: 'windows-latest'
steps:
- task: CmdLine#2
inputs:
script: |
echo QA pipeline
Everything works fine:

Migrating azure-pipelines.yaml to separate repo, but running on code of other repo

Ultimately, I'm trying to do this:
Move azure-pipelines.yaml and associated templates out of the code repository (code-repo).
Move them into a separate dedicated repository (pipeline-repo).
Have the pipeline look at the config for the pipeline in pipeline-repo, but run the pipeline on the code in the code-repo.
I'm referring the following documentation:
Use other repositories: this one refers to "templates in other repositories," but I'm trying to remove any pipeline configs so the code-repo is just purely application code... and the Dockerfile.
Define a repositories resource
For testing, I have this simple test.yaml:
# Triggers when PR is created due to branch policies
trigger: none
resources:
repositories:
- repository: code-repo
type: git
name: code-repo
pool:
vmImage: 'ubuntu-latest'
stages:
- stage: Testing
displayName: Test stage
jobs:
- job: ParallelA
steps:
- bash: echo Hello from parallel job A
displayName: 'Run a one-line script'
When I create a PR on code-repo, it is triggering the pipeline, which is to say branch policies are configured to refer to that pipeline. I do get the print out the Hello from parallel job A print out.
But I don't see in the run logs it pulling code-repo.
I do see the following, however:
My actual PR pipeline would look something like this:
trigger: none
resources:
repositories:
- repository: code-repo
type: git
name: code-repo
variables:
- template: templates/variables.yaml
pool:
vmIMage: $(vmImageName)
stages:
- template: templates/build/buildStage.yaml
...
Testing that, it confirms that it isn't running on the code-repo PR, but the pipeline-repo so everything fails.
So it is unclear to me what I need to do from here to get the pipeline to run on the PR code from code-repo.
Suggestions?
Ok, I think I have it sorted out, at least some of my stages are now succeeding.
I came across this documentation which informed me of checkout.
So in addition to doing something like:
resources:
repositories:
- repository: code-repo
type: git
name: code-repo
Then you need to add a step called checkout like the following:
# Triggers when PR is created due to branch policies
trigger: none
resources:
repositories:
- repository: code-repo
type: git
name: code-repo
pool:
vmImage: 'ubuntu-latest'
stages:
- stage: Testing
displayName: Test stage
jobs:
- job: ParallelA
steps:
- checkout: code-repo
- task: task1
- task: task2
The checkout should set the context for the subsequent steps.

How to run a template(.yml) belonging to another repository from my current repository in azure devops?

With the below setup, when my pipeline corresponding to my repo runs, it runs the template (template.yml) file belonging to 'anotherRepo'. But when it checks out, it checks out my repo instead of 'anotherRepo'.
Is there any issue in my setup?
Looks like checkout:self does not have any impact and it does not work
My current Repo:
azurepipeline.yml file:
variables:
acceptanceTestsRepoName: 'anotherRepo'
resources:
repositories:
- repository: 'anotherRepo'
name: ProjectName/anotherRepo
type: git
ref: master
stages:
- stage: acceptance_tests
displayName: 'Run Acceptance Tests in Dev'
jobs:
- template: 'azure-pipelines-templates/template.yml#${{variables.acceptanceTestsRepoName}}'
Repo:anotherRepo
template.yml
jobs:
- job: AcceptanceTest
displayName: Run Acceptance Test in Dev
pool:
vmImage: 'ubuntu-latest'
steps:
- checkout: self
- task: UsePythonVersion#0
inputs:
versionSpec: '$(python.version)'
self always refers to a repo associated with the build pipeline. In your case, you need to checkout anotherRepo manually:
# Azure Repos Git repository in the same organization
- checkout: git://anotherRepo
This assumes that anotherRepo is in the same Azure DevOps organization. If it's not or stored somewhere else (GitHub, Bitbucket, etc...) you also need to add it as a resource to the pipeline defintion. See Check out multiple repositories in your pipeline for details.
According to the document about checkout: self represents the repo where the initial Azure Pipelines YAML file was found.
So your pipeline checkout the repo where your azurepipeline.yml file is located.
If you want to checkout your anotherRepo, the checkout step in your template.yml should be - checkout: anotherRepo:
jobs:
- job: AcceptanceTest
displayName: Run Acceptance Test in Dev
pool:
vmImage: 'ubuntu-latest'
steps:
- checkout: anotherRepo
- task: UsePythonVersion#0
inputs:
versionSpec: '$(python.version)'
You can also use Inline syntax checkout to directly check out another repo in azure-pipelines.yml file:
stages:
- stage: acceptance_tests
displayName: 'Run Acceptance Tests in Dev'
jobs:
- job: checkout
steps:
- checkout: git://ProjectName/anotherRepo
Thanks all for your response expecially #beatcracker
By replacing the checkout step from self , I was able to run successfully
No change to My Current Repo.
Below change done to another repo
jobs:
- job: AcceptanceTest
displayName: Run Acceptance Test in Dev
pool:
vmImage: 'ubuntu-latest'
steps:
- checkout: git://ProjectName/anotherRepo
- task: UsePythonVersion#0
inputs:
versionSpec: '$(python.version)'

Project files not available in deploy job azure dev ops pipelines

The files in the related repo are available when running the Build stage of the Azure Dev Ops pipeline, but not when running the deploy stage. Any ideas as to why this would be the case?
Here is a simplified version of the yaml file:
# Deploy to Azure Kubernetes Service
# Build and push image to Azure Container Registry; Deploy to Azure Kubernetes Service
# https://learn.microsoft.com/azure/devops/pipelines/languages/docker
trigger:
- master
resources:
- repo: self
variables:
# Agent VM image name
vmImageName: 'ubuntu-latest'
# Name of the new namespace being created to deploy the PR changes.
k8sNamespaceForPR: 'review-app-$(System.PullRequest.PullRequestId)'
stages:
- stage: Build
displayName: Build stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- task: Bash#3
inputs:
targetType: 'inline'
script: |
pwd
ls -la
- stage: Deploy
displayName: Deploy stage
dependsOn: Build
jobs:
- deployment: Deploy
condition: and(succeeded(), not(startsWith(variables['Build.SourceBranch'], 'refs/pull/')))
displayName: Deploy
pool:
vmImage: $(vmImageName)
environment: 'test.development'
strategy:
runOnce:
deploy:
steps:
- task: Bash#3
inputs:
targetType: 'inline'
script: |
pwd
ls -la
Additional notes:
If the deploy stage is run first (the build stage is removed) the working directory is also empty.
The job in your Deploy stage is a deployment job rather than a standard job, deployment jobs don't automatically checkout the repo that the pipeline is based on but they do have access to any published pipeline artifacts.
You can either publish a pipeline artifact in the Build stage or add a task to your Deploy stage to explicitly checkout the repo.
To publish a pipeline artifact add the Publish Pipeline Artifact as task in your Build stage. In your Deploy stage you can then reference files in that artifact with the path $(Pipeline.Workspace)/<artifactName>/<rest-of-path>
To checkout out the whole repo add this to your Deploy stage:
steps:
- checkout: self
path: 'myrepo/'
Then reference the files in the repo using $(System.DefaultWorkingDirectory)\<rest-of-path>

Resources