Updated 26-Feb-2020
In our Project, I have a pipeline "MyPipeline" which restores the NuGet packages, builds the solutions and runs the tests.
On the master branch, I have a policy which does things like add code reviewers, and it has a "build validation" step which executes "MyPipeline". All well and good.
However, I created another branch from Master called NewBranch and synced (pushed) this up to Azure. After doing some minor changes in Visual Studio, I did a merge from master, committed and sync'd to Azure.
I was a bit surprised then to see "MyPipeline" executing. It seems to have been triggered when I pushed my changes to NewBranch to Azure. I don't have a branch policy on "NewBranch". The trigger in the YAML file is:
trigger:
- master
What kicked this off? I'll soon burn through my free agent minutes if this continues...
Update 26-Feb-2020
As per the comments below:
The history of commits on the master branch are:
Tue 9:10 PM
Tue 7:47 PM
Mon 2:46 PM
The history of the pipeline execution show:
Wed 9:14 AM "PR Automated"
So, nothing new has been committed to the Master branch. However, I think I know what's causing this. Just that I'm not convinced on the timing....
So, we have two branches, master and NewBranch.
Master has a policy that requires two code reviewers to Approve, and it requires the build to succeed. Because of this policy, a developer can't therefore merge directly into master - they have to generate a Pull Request.
So, Developer A creates a Pull Request to merge NewBranch to Master. There is then the rather lengthy code review process which may take multiple additional commits to the "NewBranch"'s Pull Request before it's deemed acceptable.
What seems to be happening is that everytime one of these new Commits is sync'd to the Pull Request, a build is triggered. Is this a good thing? Maybe, Maybe not. If the build is going to trigger just once, then the compilation should occur when all the code in the Pull Request has been approved, not before hand. Why trigger a build at such an early stage; the master branch may be updated by multiple other Pull Requests before this is ready to be merged in. However, with unlimited resources, then I guess there's no harm in building as often as possible, but a) this can delay other builds (representing an impediment to other developers) and b) this uses up the free limit on agent time.
The detailed info you share in question is much help for me to understand the whole workflow of yours.
Though you did not express too clear, but, yeah, what you are guessing is correct. The action you faced is expected and by designed. The root cause is you are using branch policy and the Build validation also included into this policy.
The nutshell is you are using Pull request trigger.
Explanation:
Let's pay attention to its definition:
Pull request (PR) triggers cause a build to run whenever a pull request is opened with one of the specified target branches, or when
changes are pushed to such a pull request.
Based on your added contents, your developers are pushing changes (new commit) into the opening Pull request (Note: The key point is Pull request is still be opening.) This belongs to the work scope of the PR trigger because of above definition. That is why the build triggered every y new changes pushed into NewBranch branch.
Work around:
I agree with the logic of #devpro's answer. But its script does not available for your scenario. Because the pr in YAML only work for GitHub and Bitbucket Cloud repos.
Here you are using VSTS repo, and configured the policy for it. So, you can only via branch policy configuration to avoid such burning trouble.
In your Build validation panel, you are setting build policy with Automatic, right?
Please change the Automatic to Manual, also keep the Policy requirement value as Required.
Now, the corresponding build pipeline will not be ran automatically once new commit pushed, it can only built when someone run it manually.
For the delay timing you noticed which make you unsure, I guess it would because of conflict.
For sample, the Pull request P1 which merge from NewBranch to Master is opening. I commit a new change C1 into NewBranch. BUT, it causes the conflict. At this time, the build will not be ran because the changes does not actually pushed into PR.
Note: The commit is true to NewBranch. BUT changes does not accepted by PR yet, because PR detect out there has conflict here and you must tell PR which changes you want to keep. Only the changes pushed into PR can work with PR trigger.
Only the conflict solved, the changes, maybe I can say commit, can truly be accepted and pushed into PR. Then the triggered build ran.
This would be the time period delay you noticed.
I've seen the same behaviour today.
Can you try rewriting the trigger part in your pipeline like this:
trigger:
branches:
include:
- master
Does that help ? I can't verify it myself yet, since I've done that change but the PullRequest isn't approved yet :)
I've also another repository where I do not see that behaviour, but there, my pipeline trigger looks like this:
trigger:
- master
(Notice the 2 spaces before - master)
I think you need to add pr: none to your pipeline definition to disable the automated run.
If pr is not set I think the default is to run on every PR.
That would give something like this:
trigger:
batch: true
branches:
include:
- master
paths:
exclude:
- README.md
pr: none
Related
I am using Azure Devops in an IT environment with many different development teams and git repositories. Each development team owns one or more repositories. It's my job to work on various application components contained in said repositories. Because I do not own those repositories, I should not make any changes in build/release pipelines, build policies etc, all by myself because that can impact other people's work.
Now let's say I have a feature branch named UpgradedFeature in the repository FeatureRepository, containing my changes. Said changes also introduce a breaking change in the build pipeline used for that repository for the master branch. Let's say that pipeline is named MasterBuildPipeline.
So in order for my build based on the branch UpgradedFeature to succeed and not impact other people's work, I make a clone of the MasterBuildPipeline, name it UpgradedFeatureBuildPipeline and configure the breaking changes. This new build pipeline is used exclusively by me for the UpgradedFeature branch only.
The build, now using the new UpgradedFeatureBuildPipeline pipeline succeeds and now I want to merge into master , so I make a pull request to merge the changes contained in UpgradedFeature into master. The master branch has a branch policy in place named MasterBranchPolicy like described on https://learn.microsoft.com/en-us/azure/devops/repos/git/branch-policies?view=azure-devops&tabs=browser. This branch policy contains the MasterBuildPipeline and prevents completion of that pull request when the build using that pipeline does not succeed.
So my problem is that my pull request triggers the MasterBranchPolicy containing the MasterBuildPipeline and not the UpgradedFeatureBuildPipeline containing the necessary breaking changes for the build to succeed. So the build fails and I cannot complete the pull request.
Of course I could edit the MasterBuildPipeline for a short time, introduce my breaking changes, run the build, then discard the breaking changes again. But there's a chance I may impact other people's work with that and somehow I have a feeling that's not the right approach. I could also edit or disable the MasterBranchPolicy for a short time but again, I may impact other people's work and I feel it's not the right approach.
How should I do this?
So the build fails and I cannot complete the pull request.
To complete the pull request even through the build fails, you could grant yourself Bypass permissions. Bypass permissions let you complete pull requests that don't satisfy branch policies. You can grant bypass permissions to yourself then complete the pull request. Here is Bypass branch policies for reference.
Please navigate to Project setting >> Repositories >> The repo >> Security >> user (yourself) >> Bypass policies when completing pull requests.
Then, you can Override branch policies and enable merge even the MasterBuildPipeline faild.
Please also note that use caution when granting the ability to bypass policies, especially at the repo and project levels. Policies are a cornerstone of secure and compliant source code management. In your scenario, it's suggested to edit the MasterBuildPipeline and the MasterBranchPolicy or disable the MasterBranchPolicy as you mentioned.
Generally, workflow of DevOps Branching Strategy as follows
Developer will create a feature or bugfix branch out of develop. One feature or bugfix branch usually stands for one JIRA bug or feature item. These branches are personal
The changes will be pushed into the developer's feature or bugfix branch.
When the new feature or bugfix is complete.A developer will create a pull request. Pull requests open a code review phase.
Once a pull request has been approved, the team lead or development team will move it into development.
When the development branch has all the epics and bug fixes, i.e., the content planned for the next release, the development team or team lead will create a release branch. This initiates the release regression testing phase.At this stage, only bug fixes are accepted for release, and the workflow is similar to that of the development branch.
Having a separate release branch will enable future development towards the next release in the development branch. Features for the next release are not included in this release. However, bug fixes for this release will be incorporated into the development of the next release as well.
When release content meets the criteria, the release branch will be frozen, which means that it ends. Content from releases will be merged to master and tagged there. For the next release, a new release branch is created when needed.
As per my experience, I would suggest creating branching policies like
A pull request is requested to merge the develop, release, and master branches.
Pull request approvers should be leads.
All developers can create feature branches.
All developers can push to hotfix and feature branches. Commit messages must include the JIRA issue id.
I need to set up a pipeline for a branch that will do the following,
On push, it will trigger a pipeline (which happens as per GitLab)
1st Job to trigger a build (.net5) for the branch (new code pushed to the branch). If the build succeeds, it will begin the next job. However, I am to trigger build (MSBuild) on the runner location check out when the pipeline runs. But is there any way to directly run a build on the branch?
If the above build fails, the push should be reverted on a branch. I applied git revert and reset commands, but it only gets used to the build location of the runner. I need to revert commit at the branch level.
When GitLab starts a pipeline, it is a "detached HEAD" which basically means it creates a temporary branch that points to your specific commit. It does this because there might be a commit after it which the current build doesn't know about.
To get out of the detached head, you need to switch to the branch directly.
git checkout branch-name
To get the branch name, you find an appropriate predefined variable which works for your.
Once you have that, you need to get the latest version so you can revert. I usually needed the reset here to make things work.
git reset --hard
git pull
From there, you can revert. Now, the CI process doesn't have write permissions, so you need to have a GitLab access token with write permissions, which means you have to set up a new origin or parent.
git remote add changes https://oauth2:$PUSH_GITLAB_TOKEN#gitlab.com/group-name/project-1
git push -f changes branch-name
Now... that said, this is probably not what you want to do. Since you said you were new to CI, I'm going to say I'm 99.99999% sure you don't want to go down this path because it only leaves to heartache and frustration.
The reason is the branch may have a second or additional commit. It could be you missed something at the last minute and threw up a new one, an automated process starts up, or just your typical race conditions. Reverting a commit means you either will blow up with those changes or you end up erasing every other commit as you have to force push to get it up.
This also means that this is a very fragile process and will break many times, usually at the worst times (when you have a rapid series of commits), and then you have to tell everyone to stop working while you fix it.
(A good sign is how much work you have to do to fight the CI process.)
Instead, I recommend you create a merge request (MR) against your branch and set up the rules to make sure that is valid before you merge it into your branch. That way, the process doesn't have to worry about future or past commits, it just says what you have in your merge request (which is another branch) can't be applied after merging it with branch-name.
In our team, we have many branches (main, next, release/2.3.4, release/2.4.0, etc.) and doing MRs against each one has worked out well. We do the merge requests against each of the release branches, and the CI process says it can merge and pass tests before it tries to commit, but then we can commit knowing that 98+% of the time, it will be fine.
And it doesn't require jumping through hoops.
I have created a pipeline in my repository which is used to validate code by executing unit tests for code that is being pushed to features/* branches. The same pipeline is used as Build validation pipeline set as Branch Policy on the develop branch to validate incoming PRs. This is the trigger of the pipeline.
# pipeline.yml
trigger:
batch: false
branches:
include:
- features/*
However we have come across the following condition: Given an open PR from refs/heads/features/azure-pipelines -> refs/heads/develop we push a commit on the features/azure-pipelines branch.
This causes the pipeline to trigger twice. To my understanding one of the runs is due to the trigger of the pipeline (The one marked as Individual CI on the screenshot) and the second run is due to branch policy trying to validate code being pushed onto the open PR to develop. (The PR Automated)
Is there any way to disable one of the executions since it's essentially a duplicate? I was maybe looking for a way to retrieve open PRs and abort execution of a pipeline for Individual CI if there is an open PR for a branch but I am not sure that's the best way around that and I am looking for options.
You can set
trigger: none
This way only the branch policy will trigger the pipeline.
Is there any way to disable one of the executions since it's essentially a duplicate?
As we know, we could not disable the Build validation pipeline set as Branch Policy on the develop branch to validate incoming PRs unless we cancel the Build validation.
For your situation, you could try to include [skip ci] in the commit message or description of the HEAD commit to make the Azure Pipelines skip running CI when you plan to merge the features branch to the develop branch.
You could check the document Skipping CI for individual commits for some more details.
Here it depends if they does the same. You can have conditional checks in the pipeline which does a different things for PR and CI runs. However, I'm pretty sure that this is not possible, because one is defined on the YAML and the second on the Azure DevOps portal. So even if you disnle PR trigger here in YAML, a branch policy still runs a PR. And you can specify antyhing in YAML to block branch policy.
I know this is a basic question but can someone explain with a scenario based on VCS like Git
A "PR" pipeline is just a pipeline that is triggered by a new pull request. It will usually run tests, static analysis, linting, or other checks against the incoming branch as a way to ensure coding standards are met before merging the PR. Azure DevOps will show the progress and result of the build on the PR page as well.
A CI pipeline is usually triggered when a specific branch is changed (perhaps master or develop). A build is automatically kicked off to run build/tests/static analysis/archiving/etc to ensure the new commits do not break things.
Here is more information from Microsoft that may help:
https://learn.microsoft.com/en-us/azure/devops/learn/what-is-continuous-integration
Hope this helps!
I've been using Azure Pipelines for a while now and haven't changed my azure-pipelines.yml file here in 2 months. Previously, when there was a new PR, the pipeline would trigger and cause the environment to be built and the tests would be run.
Today, there was a new PR but I noticed that the pipeline was not being triggered. Then, to further test this, I forked, cloned, and branched the repository myself and created another new PR and, again, the pipeline was not triggered.
It's not clear to me where things are getting stuck and it's not clear how one would debug this. I've gone through this Azure DevOps documentation but it wasn't useful. I can manually trigger the pipeline to execute and test the master branch but I don't know how to manually trigger the same thing for a PR. Here's my Azure DevOps page for reference.
As normal, you do not need config pr in YAML script if there's no any special demand, we would do pull request trigger for all branches. But, it start broken from 03-13 21:02 (UTC), which caused by us, you do not do anything wrong.
The fix is preparing with our best.
As Alex said, this is the implicit trigger which YAML support only, if you do not configure pr in YAML explicitly.
To avoid such stuck later, except the method that Alex mentioned: add pr into YAML. You can also make use of UI configuration which performance is very stable until now.
Just go Pipeline definition page => Click on three dots of right corner => Select Trigger:
Then you will see Triggers tab which has Continues integration and Pull request validation display below. Open Pull request validation and enable Override the YAML pull request trigger from here:
Additional, Our team has noticed this broken issue, will update whether it is fixed here once we have any fixed release in progress.
Update 3/18/2020:
Fixed has released to all region. Every one can work github pr trigger as the document shows now.