I have a GitLab project pipeline that triggers a downstream pipeline, GitLab multi-project pipelines.
image: docker
trigger-docs:
trigger:
project: my-group/docs
branch: feat/my-feature-branch
Is there a way for the triggered pipeline in my-group/docs to find out where it was triggered from? I checked the predefined CI variables but none seems to carry this information.
Could it be that my only option is to pass a dedicated variable from the upstream project as documented at https://docs.gitlab.com/ee/ci/pipelines/multi_project_pipelines.html#pass-cicd-variables-to-a-downstream-pipeline-by-using-the-variables-keyword?
Here's the workaround we have been using for months now; send along the custom UPSTREAM_PROJECT variable.
# Trigger a downstream build https://docs.gitlab.com/ee/ci/pipelines/multi_project_pipelines.html
docs-build:
stage: .post
variables:
UPSTREAM_PROJECT: $CI_PROJECT_PATH
# Variable expansion for 'trigger' or 'trigger:project' does not seem to be supported. If we wanted this we would have
# to work around it like so: https://gitlab.com/gitlab-org/gitlab/-/issues/10126#note_380343695
trigger: my-group/docs
Related
I am trying to write a pipeline to build an image and deploy to the test environment which is hosted on Azure. My codebase lies on GitHub. While trying to trigger the pipeline on a pull request from the source branch against the target branch, I am facing an issue where the pipeline doesn't trigger for the PR but runs fine for my other conditions, such as, push to develop or master.
The condition used for the PR trigger is as follows:
and(succeeded(), eq(variables['Build.Reason'], 'PullRequest'), startsWith(variables['System.PullRequest.SourceBranch'], 'release/'), eq(variables['System.PullRequest.TargetBranch'], 'master'))
The triggers in the yaml file can be seen below:
trigger:
branches:
include:
- develop
- master
paths:
exclude:
- k8s/*
- src/VERSION
- src/package.json
pr:
- master
Am I missing something here?
There are two scenarios:
Scenario 1: The pipeline was triggered when the pull request is created, but the stages/jobs/tasks with the condition you showed don't run.
Then the issue should be related to condition, not trigger.
I have tested and confirmed that your condition is right. So, it's probably not the condition notation but something else that's causing your task not to run.
Here is a troubleshooting advice:
Go to the build log, click on the stages/jobs/tasks that were skipped. You will find a comparison between the condition and the real value. From here, you can tell which part of the condition is keeping your tasks from running.
Scenario 2: The pipeline wasn't triggered when the pull request is created.
Then the issue should be related to trigger, not condition.
Please select documents below for detailed troubleshooting advice based on your case:
I just created a new YAML pipeline with CI/PR triggers, but the pipeline is not being triggered.
My CI or PR triggers have been working fine. But, they stopped working now.
I'm not sure if this will fix it but according to the documentation the System.PullRequest.TargetBranch variable has format refs/heads/main which would mean your condition needs updating to add refs/heads in front of the variables.
As such I would add a step to echo these variables just to confirm if they have the refs/head prefix and if so adjust your logic accordingly
I have an Azure DevOps Pipeline for a Git repository. I currently have a script to validate the PR comments in the Azure Pipeline.
When the code is merged into the main branch I want to trigger a build. I am not sure how to achieve this with a Azure DevOps pipeline.
#Trigger for Development
trigger:
branches:
include:
- development
- master
#Trigger checks for PR
pr:
branches:
include:
- development
- master
- feature
- main
paths:
exclude:
- README/*
When the code is merged into the main branch I wanted to trigger build
If you want to verify the comments after the code is merged into the main branch, we need to trigger the build after the PR completed instead of when PR is created.
So, the PR triggers could not meet our requirement in this case.
To resolve this issue, we could enable CI triggers for the main branch with ** condition** eq(variables['Commitcomment'], 'Merge pull request') for the task of script to validate the PR comments.
With this condition, the pipeline will execute the job only when the Commitcomment is Merge pull request, this can filter out modifications not done by PR.
To get the value of the variable Commitcomment, we could to check the commits message on our github by the variable Build.SourceVersionMessage:
If the commit comes from PR, it will given a default comment, starting with: Merge pull request xxx, we could add a bash\powershell script to get the first few fields.
Then use Logging Command to set the variable Commitcomment to true if the first few fields is Merge pull request:
- task: CmdLine#2
displayName: get the first few fields
inputs:
script: >-
echo $(Build.SourceVersionMessage)
set TempVar=$(Build.SourceVersionMessage)
set Commitcomment=%TempVar:~0,18%
echo %Commitcomment%
echo ##vso[task.setvariable variable=Commitcomment]%Commitcomment%
Reference link: Is there a short 7-digit version of $(SourceVersion) in Azure Devops?
Then add this variable as condition condition: and(succeeded(), eq(variables['Commitcomment'], 'Merge pull request')) for your task to verify the PR comments:
- task: CmdLine#2
displayName: script to validate the PR comments
condition: and(succeeded(), eq(variables['Commitcomment'], 'Merge pull request'))
inputs:
script: >
echo To validate the PR comments
In this case, if the commit not comes from PR, it will skip the PR comments verify task:
If you just want to launch a build when the merge is done (pull request validated) in a specific branch, your code is good.
If you want to run a validation build currently it is not integrated into the Yaml pippeline configuration (https://learn.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema%2Cparameter-schema#pr-trigger)
To do this, it must be done via the graphical interface:
Project Settings -> Repositories -> Select your repo -> Policies -> Branch Policies -> Select your branch -> Build Validation -> + -> add build information
(https://learn.microsoft.com/en-us/azure/devops/repos/git/branch-policies?view=azure-devops#build-validation)
I'm able to include the latest version of the artifact published by another pipeline (AppCIPipline) into my YAML pipeline using Conditional insertion:
name: '$(Build.SourceBranchName)-$(date:yyyyMMdd)$(rev:.r)'
resources:
pipelines:
- pipeline: AppBuildToDeploy # Required when source == Specific
source: App_Master_CI
branch: master
# buildToDeploy is a pipeline variable
${{ if ne(variables['buildToDeploy'], '') }}:
version: $(buildToDeploy) #let's leave it blank from the pipeline
project: NewHorizon
trigger: none
pool: 'Matrix' # Self hosted agent on a windows server
steps:
- download: 'AppBuildToDeploy'
patterns: '*_BuildScripts.zip'
displayName: 'Download Specified Artifacts'
I'm getting the following error: " A template expression is not allowed in this context"
Is there a way to get the version number from the user at run time and use the version, if supplied, else default to the current version?
For now the user experience is not supported yet. For now, we have to use hard-code way.
Someone has posted this feature request in DC before. You can vote for this open issue and follow it to track the request there. If it gets enough votes, the team would consider it seriously.
I have a triggers set up in our azure-pipelines.yml like so below:
the scriptsconn represents a connection to the default/self repo that contains the deployment pipeline yaml.
the serviceconn represents a microservice repo we are building and deploying using the template and publish tasks.
We have multiple microservices with similar build pipelines, so this approach is an attempt to lessen the amount of work needed to update these steps.
Right now the issue we're running into is two fold:
no matter what branch we specify in the scriptsconn resources -> repositories section the build triggers for every commit to every branch in the repo.
no matter how we configure the trigger for serviceconn we cannot get the build to trigger for any commit, PR created, or PR merged.
According to the link below this configuration should be pretty straighforward. Can someone point out what mistake we're making?
https://github.com/microsoft/azure-pipelines-yaml/blob/master/design/pipeline-triggers.md#repositories
resources:
repositories:
- repository: scriptsconn
type: bitbucket
endpoint: BitbucketAzurePipelines
name: $(scripts.name)
ref: $(scripts.branch)
trigger:
- develop
- repository: serviceconn
type: bitbucket
endpoint: BitbucketAzurePipelines
name: (service.name)
ref: $(service.branch)
trigger:
- develop
pr:
branches:
- develop
variables:
- name: service.path
value: $(Agent.BuildDirectory)/s/$(service.name)
- name: scripts.path
value: $(Agent.BuildDirectory)/s/$(scripts.name)
- name: scripts.output
value: $(scripts.path)/$(release.folder)/$(release.filename)
- group: DeploymentScriptVariables.Dev
stages:
- stage: Build
displayName: Build and push an image
jobs:
- job: Build
displayName: Build
pool:
name: 'Self Hosted 1804'
steps:
- checkout: scriptsconn
- checkout: serviceconn
The document you linked to is actually a design document. So it's possible\likely that not everything on that page is implemented. In the design document I also see this line:
However, triggers are not enabled on repository resource today. So, we will keep the current behavior and in the next version of YAML we will enable the triggers by default.
The current docs on the YAML schema seem to indicate that triggers are not supported on Repository Resources yet.
Just as an FYI you can see the current supported YAML schema at this url.
https://dev.azure.com/{organization}/_apis/distributedtask/yamlschema?api-version=5.1
I am not 100% sure on what you are after template wise. General suggestion, if you are going with the reusable content template workflow, you could trigger from an azure-pipelines.yml file in each of your microservice repos, consuming the reusable steps from your template. Hope that helps!
Attempting to trigger an Azure pipeline when another pipeline has been completed using a YAML. There's documentation indicating that you can add a pipeline resource with:
resources: # types: pipelines | builds | repositories | containers | packages
pipelines:
- pipeline: string # identifier for the pipeline resource
connection: string # service connection for pipelines from other Azure DevOps organizations
project: string # project for the source; optional for current project
source: string # source defintion of the pipeline
version: string # the pipeline run number to pick the artifact, defaults to Latest pipeline successful across all stages
branch: string # branch to pick the artiafct, optional; defaults to master branch
tags: string # picks the artifacts on from the pipeline with given tag, optional; defaults to no tags
However, I've been unable to figure out what the "source" means. For example, I have a pipeline called myproject.myprogram:
resources:
pipelines:
- pipeline: myproject.myprogram
source: XXXXXXXX
Moreover, it's unclear how you'd build based a trigger based on this.
I know that this can be done from the web-GUI, but it should be possible to do this from a YAML.
For trigger of one pipeline from another azure official docs suggest this below solution. i.e. use pipeline triggers
resources:
pipelines:
- pipeline: RELEASE_PIPELINE // any arbitrary name
source: PIPELINE_NAME. // name of the pipeline shown on azure UI portal
trigger:
branches:
include:
- dummy_branch // name of branch on which pipeline need to trigger
But actually what happens, is that it triggers two pipelines. Take an example, let suppose we have two pipelines A and B and we want to trigger B when A finishes. So in this scenario B runs 2 times, once when you do a commit (parallel with A) and second after A finishes.
To avoid this two times pipeline run problem follow the below solution
trigger: none // add this trigger value to none
resources:
pipelines:
- pipeline: RELEASE_PIPELINE // any arbitrary name
source: PIPELINE_NAME. // name of the pipeline shown on azure UI portal
trigger:
branches:
include:
- dummy_branch // name of branch on which pipeline need to trigger
By adding trigger:none second pipeline will not trigger at start commit and only trigger when first finish its job.
Hope it will help.
Microsoft documentation says that YAML is the preferred approach. So, instead of going for the build-trigger option let's understand the, little bit confusing, YAML trigger. The following tags will work from the original question and now with a bit easier documentation:
resources:
pipelines:
- pipeline: aUniqueNameHereForLocalReferenceCanBeAnything
project: projectNameNOTtheGUID
source: nameOfTheOtherPipelineNotTheDefinitionId
trigger:
branches:
include:
- master
- AnyOtherBranch
The documentation from Microsoft is confusing and the IDs are numerous. At times they want the Project GUID at times the project name. At times they want the pipeline name and at times the pipeline definition Id. But they use the same name for the variable (project and pipeline). And on top of that they write documentation that does not make it easy to guess which one to use the best way is to trial and error.
I think to avoid the confusion in other places I'm giving example of another place in the pipeline you refer to the same variables with different values. In the DownloadArtifact task, you need to use the project GUID and the pipeline definition Id as shown below:
- task: DownloadPipelineArtifact#2
inputs:
source: specific (a literal constant value not the pipeline name)
project: projectGUIDNOTtheProjectName
pipeline: numericDefinitionIdOfPipelineNotPipelineNameOrUniqueRef
runVersion: 'latest'
Just look at how they used the same variables in a different way, but both referring to a pipeline and in my case the same exact pipeline. That could create confusion and to avoid stumbling into the next issue I give it here for clarification.
The resources are not for the Build Completion trigger. according to the docs the build completion trigger not yet supported in YAML syntax.
After you create the YAML pipeline you can go to the classic editor (click on settings or variables) and there create the trigger.
Edit:
Now you need to click on the "Triggers":
And then:
Second Edit:
Microsoft added this feature also the YAML :) see here:
# this is being defined in app-ci pipeline
resources:
pipelines:
- pipeline: security-lib
source: security-lib-ci
trigger:
branches:
- releases/*
- master
In the above example, we have two pipelines - app-ci and security-lib-ci. We want the app-ci pipeline to run automatically every time a new version of the security library is built in master or a release branch.
If you're not publishing an artifact from the triggering pipeline, it won't trigger the triggered pipeline.
Also, if the defaultBranch for manual and scheduled builds in the triggered pipeline is not the same as your working branch, the triggered pipeline won't kick in at the end of the triggering pipeline execution.
I have created a minimum viable product for a pipeline trigger, and I explain better the two issues I just mentioned in this answer.