While creating a pipeline, I want to enable batching for specific branches and not so for other branches, is there a way of doing so in microsoft azure pipelines ?
If I enable batch:true it runs for all branches and if I remove it, it doesn't work for any branches
I have it like
trigger:
batch: true
branches:
include:
- master
- A*
- B*
- C*
- D*
- TEST
- Develop
I don't want batching only for master branch but want batching for other branches, can this be achieved without creating seperate pipeline yml just to enable batch:true in the same pipeline ?
In the Azure DevOps Service, the yaml build just trigger and batch for current branch.
For example:
If we create YAML build in the master branch, it will create azure-pipelines.yml file in the master branch. Edit the YAML build definition, such as:
trigger:
batch: true
branches:
include:
- master
- TEST
Switch to TEST branch and edit the azure-pipelines.yml file, such as:
trigger:
batch: true
branches:
include:
- master
Then push code in the TEST branch, it will not trigger the build and queue a new build.
I want to enable batching for specific branches and not so for other branches,
As a work around, we could switch the branch, edit the YAML file and enable batching for specific branch. This will not affect other branches.
Note: Since the file is different in the different branches, the file azure-pipelines.yml will conflict when we create PR
Update
Thanks for #Hiteshdua1 sharing.
The solution is create different yaml build definitions, then we could enable batching for specific branches and not so for other branches.
Related
I have a pipeline in Azure DevOps that is triggering from more than just the branch I have specified in the yaml file. I am using bitbucket cloud for my repository.
At first I had the trigger set like this:
trigger:
- development
Then I changed it to this:
trigger:
branches:
include:
- development
It is still triggering from 2 other branches besides development. When I look at the branches tab on the pipeline it has development and the other 2 branches it's triggering from. I also don't have any triggers set in the UI.
Any ideas? Thanks.
Have you updated the YAML file in the correct branch? The branch the pipeline definition is derived from?
As a workaround you could try to add the excludes clause in your YAML trigger to ignore those other two branches.
trigger:
branches:
include:
- development
exclude:
- PBI123
- PBI456
A note on the Branches tab, this is a summary of the pipeline runs that have already been executed, it is not a listing of the branches what will be executed by the pipeline.
I hope this helps resolve your issue.
Ok after looking into it some more I believe this solves it:
pr: none
As long as the PR has that in it. It shouldn't trigger.
I have CI/CD related question
I created a release pipeline that copies one file to Azure blob and it's working fine.
It's getting triggered every time a change is merged to the main/master branch.
Question is, is there a way to only trigger it if the merged changes contain a specific file? just to avoid over triggering
For a YAML-based pipeline, the documentation (currently at https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/azure-repos-git?view=azure-devops&tabs=yaml) suggests that "you can specify file paths to include or exclude", with the example
# specific path build
trigger:
branches:
include:
- master
- releases/*
paths:
include:
- docs
exclude:
- docs/README.md
It used to be the case that Azure DevOps would run a new CI build when you pushed a new branch to Origin. My company relied on this, as we would issue releases by creating a release/* branch off of develop which would trigger a build and a subsequent release into our testing environment. If everything passed UAT/QA, we would deploy the that same build into production.
Our Build Pipelines are both Classic UI and have two Branch filters, develop and release/*. The one product in question has two build pipelines - one for the Webjob and one for the API, and as such each pipeline has a Path filter (to, or not to, include the Webjob folder). Creating a release/* branch does not trigger either CI pipeline.
Our Release Pipeline looks like the following, where DEV is triggerd on artifacts produced from develop and TEST->PROD is triggered on artifacts built from release/* -
This allows us to have a release environment where we iteratively make changes to get it ready, while we simultaneously make changes to develop for the next release. This is a common Gitflow model.
The Problem
Now that Azure no longer triggers a build on branch creation, I'm forced to manually run a build on Release in order to trigger the build and subsequent TEST deployment depicted in the image above.
What can I do to have automated builds and deployments in Azure while maintaining Gitflow?
I was also stuck not knowing why my TEST build wouldn't trigger. This post helped explain why. More searching found the way to resolve it. Simple as removing any path checks from your TEST build.
From the linked documentation:
If your pipeline has path filters, it will be triggered only if the new branch has changes to files that match that path filter.
If your pipeline does not have path filters, it will be triggered even if there are no changes in the new branch.
See:
Behavior of triggers when new branches are created
Not sure if you are using YAML or not but using the GUI you can add a 'path' filter for Continuous Integration and set the build to run anytime a path contains 'release'.
YAML pipelines are configured by default with a CI trigger on all branches.
YAML pipelines are configured by default with a CI trigger on all branches.
When defining pipelines using YAML and defining a repository resource (repo for shorthand). A new pipeline run gets triggered when a commit has occurred. You can further configure the trigger to ONLY act on certain branches.
See:
CI triggers - MSDN
Repo triggers
Example (All branches by default):
resources:
- repo: self
clean: true
pool:
vmImage: 'ubuntu-20.04'
steps:
- task: CMake#1
displayName: 'Generate Makefile'
inputs:
workingDirectory: $(Build.SourcesDirectory)
cmakeArgs: '-B$(Agent.BuildDirectory) -H.'
- task: Bash#3
displayName: 'Build project'
inputs:
workingDirectory: $(Agent.BuildDirectory)
targetType: 'inline'
script: 'make -j4'
Limit to release branches (add trigger to repository resource):
trigger: # Optional; Triggers are enabled by default
branches: # branch conditions to filter the events, optional; Defaults to all branches.
include:
- release/*
A build pipeline can be tied to only 1 source branch.
In release pipeline, we configure an artifact by selecting the source build pipeline. So a release artifact can be tied to only 1 build pipeline.
What is the purpose of Build branch filters in Continuous deployment trigger?
Let say that your code base was updated. You want deploy only if build was made over specific branch (example develop).
In that scenario Build Branch Filter looks fairly redundant... but what if:
You want to trigger new deployment for every latest artifact, built from each feature branch to get them tested. You have to filter for feature/*.
You want to trigger new deployments if your release branches are updated, but some of them are deprecated and you need to filter them out. I that scenario specify one include filter release/* and second exclude filter release/old*.
Anyway.. in most of the cases the filter and the branch name (in artifact) will match each other. Still sometimes it can be heady to trigger deployment from multiple branches or filter something out.
In regards to your comment, I uploaded part of the yaml build. In fact one build can create artifacts from all branches in repository if you want.
trigger:
branches:
include:
- feature/*
- bugfix/*
- release/*
- develop
- master
exclude:
- experimental/*
I'm having problems triggering a pipeline from another Pipeline in Azure DevOps. I have a CI pipeline and I want to trigger a Deploy Pipeline whenever CI passes on a master branch. This seems to be technically possible, but the documentation is unclear.
I see the following:
# this is being defined in app-ci pipeline
resources:
pipelines:
- pipeline: securitylib
source: security-lib-ci
trigger:
branches:
- releases/*
- master
But it's unclear as to a) whether this goes in the triggering pipeline (in my case the CI pipeline) or the triggered pipeline (in my case, the deploy pipeline).
It's also unclear as to what the pipeline and source refer to, and how I find out these variables? Are they both the name of the pipeline? I've tried various different permutations and nothing seems to be working.
EDIT 2
Finally Microsoft has improved their documentation with regards to the pipeline triggers in YAML! Here's the link.
EDIT
After having written my answer, Microsoft has come up with another solution to solve this problem, by using a build completion trigger via a classic pipeline. Their solution can be found here.
If you're not publishing an artifact from the triggering pipeline, it won't trigger the triggered pipeline.
Also, there is a very big restriction on the use of these types of triggers. It is necessary to change the defaultBranch for manual and scheduled builds in the depends pipeline, to the working branch. Otherwise it won't kick in at the end of the source pipeline execution. So, let's say you're working on feature branch, and defaultBranch is set to feature. You commit your code, and everything will run as expected: the source pipeline kicks in, and at its end, the depends pipeline will be triggered. All good! But when you will merge into master, if you do not change the defaultBranch, the depends pipeline won't be triggered at the end of the source pipeline. I explain how to change the defaultBranch at the end of the answer.
How to setup a pipeline trigger
I managed to get this up and running on a minimalistic project. Here you can have the code and here the project on Azure DevOps. I will try to guide you through how I did it, and answer the questions you've asked in your post.
I will be calling the triggered pipeline as depends pipeline and the triggering pipeline as source pipeline.
On the source pipeline, there's no need to do anything except publishing an artifact. If you don't publish an artifact from the source pipeline, it won't work. Below you can find the code I am using for my dummy source pipeline. I want it to be triggered for master branch, and at the end I want to be sure to publish an artifact.
trigger:
branches:
include: # branch names which will trigger a build
- master
pr: none
steps:
# required to cause pipeline triggering downstream
- task: CopyFiles#2
inputs:
contents: $(System.DefaultWorkingDirectory)/**/*.yml
targetFolder: $(Build.ArtifactStagingDirectory)
- task: PublishBuildArtifacts#1
inputs:
pathtoPublish: $(Build.ArtifactStagingDirectory)
artifactName: dummy-$(Build.BuildId)
On the depends pipeline (code shown below), I have to disable CI and PR triggers, otherwise when I commit to this repo, this pipeline will be triggered by the CI trigger, and then by the end of the execution of the source pipeline. This is done by the two first lines of my code. Then I want that the pipeline named source (this is the source property in the YAML below), within the project named Pipelining (project property in the YAML) will trigger the current (depends) pipeline when this updates master branch.
trigger: none
pr: none
resources:
pipelines:
- pipeline: source
project: Pipelining
source: source
trigger:
branches:
include:
- master
steps:
- checkout: none
- script: echo 'triggered depends'
Does it make sense? It is important for your project name on Azure DevOps to match the property in the YAML depends pipeline code.For me it is Pipelining
As well as the source property, again in the YAML depends pipeline code.
Change the default branch
In order to change the defaultBranch, because of the issue mentioned above, you should edit the pipeline (in this case, the depends pipeline), then on the three dots on the top right corner pick Triggers. Then choose the YAML tab, and you will get to the screen shown in the image below, where you can set the working branch.
Above yaml pipeline trigger should be defined in the triggered pipeline(deploy pipeline).
- pipeline: string the string here is identifier you give to this pipeline resource. It can any string.
source: string the string here is the definition name of the triggering pipeline(the name of your CI pipeline).
Below yaml is from the document pipeline resource.
resources:
pipelines:
- pipeline: string # identifier for the pipeline resource
project: string # project for the build pipeline; optional input for current project
source: string # source pipeline definition name
branch: string # branch to pick the artifact, optional; defaults to all branches
version: string # pipeline run number to pick artifact, optional; defaults to last successfully completed run
trigger: # optional; triggers are not enabled by default.
branches:
include: [string] # branches to consider the trigger events, optional; defaults to all branches.
exclude: [string] # branches to discard the trigger events, optional; defaults to none.
Option: You can also set the pipeline triggers from Ui page. Go the edit page of the triggered yaml pipeline(Deploy pipeline), Click the 3dots and choose Triggers
Go to Triggers--> Build completion and click add--> Select your triggering pipeline(CI pipeline)
Update:
I saw the pipeline resource in azure-deploy.yml is defined as below.
resources:
pipelines:
- pipeline: 'Deploy to Development'
source: 'DFE-Digital.dfe-teachers-payment-service'
trigger:
branches:
include:
- "master"
- "release-stuff"
please try changing the indentation of trigger element the same as source element. Check below example:
resources:
pipelines:
- pipeline: 'Deploy to Development'
source: 'DFE-Digital.dfe-teachers-payment-service'
trigger:
branches:
include:
- "master"
- "release-stuff"
I found the following:
In source pipeline I didn't need to create an artifact
In depends pipeline if I wanted to build after any commit to the source branch I could get it to work with this:
trigger: none
pr: none
resources:
pipelines:
- pipeline: 'depends'
source: 'common-gulp-trigger'
trigger: true
I may assume you are not working on the master branch, right? I have the same issue previously. But after I read the section Default branch for triggers of MS's doc. I understand why. The trigger only examine master's branch's yaml file by default. This means the pipeline will only be triggered by the definition of triggers in master branch's yaml file.
Therefore, whatever branches you add in the trigger section of yaml file in other branches(not master), tirgger is not active. You need to change the pipeline to look the yaml file in your current branch, not master. Just follow the doc's instruction, change the default trigger branch. You will get it working.
Once you merge your work into master, you probably need to change the dedault trigger branch back to master.