I am using Azure Devops in an IT environment with many different development teams and git repositories. Each development team owns one or more repositories. It's my job to work on various application components contained in said repositories. Because I do not own those repositories, I should not make any changes in build/release pipelines, build policies etc, all by myself because that can impact other people's work.
Now let's say I have a feature branch named UpgradedFeature in the repository FeatureRepository, containing my changes. Said changes also introduce a breaking change in the build pipeline used for that repository for the master branch. Let's say that pipeline is named MasterBuildPipeline.
So in order for my build based on the branch UpgradedFeature to succeed and not impact other people's work, I make a clone of the MasterBuildPipeline, name it UpgradedFeatureBuildPipeline and configure the breaking changes. This new build pipeline is used exclusively by me for the UpgradedFeature branch only.
The build, now using the new UpgradedFeatureBuildPipeline pipeline succeeds and now I want to merge into master , so I make a pull request to merge the changes contained in UpgradedFeature into master. The master branch has a branch policy in place named MasterBranchPolicy like described on https://learn.microsoft.com/en-us/azure/devops/repos/git/branch-policies?view=azure-devops&tabs=browser. This branch policy contains the MasterBuildPipeline and prevents completion of that pull request when the build using that pipeline does not succeed.
So my problem is that my pull request triggers the MasterBranchPolicy containing the MasterBuildPipeline and not the UpgradedFeatureBuildPipeline containing the necessary breaking changes for the build to succeed. So the build fails and I cannot complete the pull request.
Of course I could edit the MasterBuildPipeline for a short time, introduce my breaking changes, run the build, then discard the breaking changes again. But there's a chance I may impact other people's work with that and somehow I have a feeling that's not the right approach. I could also edit or disable the MasterBranchPolicy for a short time but again, I may impact other people's work and I feel it's not the right approach.
How should I do this?
So the build fails and I cannot complete the pull request.
To complete the pull request even through the build fails, you could grant yourself Bypass permissions. Bypass permissions let you complete pull requests that don't satisfy branch policies. You can grant bypass permissions to yourself then complete the pull request. Here is Bypass branch policies for reference.
Please navigate to Project setting >> Repositories >> The repo >> Security >> user (yourself) >> Bypass policies when completing pull requests.
Then, you can Override branch policies and enable merge even the MasterBuildPipeline faild.
Please also note that use caution when granting the ability to bypass policies, especially at the repo and project levels. Policies are a cornerstone of secure and compliant source code management. In your scenario, it's suggested to edit the MasterBuildPipeline and the MasterBranchPolicy or disable the MasterBranchPolicy as you mentioned.
Generally, workflow of DevOps Branching Strategy as follows
Developer will create a feature or bugfix branch out of develop. One feature or bugfix branch usually stands for one JIRA bug or feature item. These branches are personal
The changes will be pushed into the developer's feature or bugfix branch.
When the new feature or bugfix is complete.A developer will create a pull request. Pull requests open a code review phase.
Once a pull request has been approved, the team lead or development team will move it into development.
When the development branch has all the epics and bug fixes, i.e., the content planned for the next release, the development team or team lead will create a release branch. This initiates the release regression testing phase.At this stage, only bug fixes are accepted for release, and the workflow is similar to that of the development branch.
Having a separate release branch will enable future development towards the next release in the development branch. Features for the next release are not included in this release. However, bug fixes for this release will be incorporated into the development of the next release as well.
When release content meets the criteria, the release branch will be frozen, which means that it ends. Content from releases will be merged to master and tagged there. For the next release, a new release branch is created when needed.
As per my experience, I would suggest creating branching policies like
A pull request is requested to merge the develop, release, and master branches.
Pull request approvers should be leads.
All developers can create feature branches.
All developers can push to hotfix and feature branches. Commit messages must include the JIRA issue id.
Related
I'm looking to limit how merges are allowed to be done in Azure DevOps. I'm looking to replicate a setup that I had in Atlassian Bitbutcket.
So far I cannot see an easy or even a hard way to accomplish this in Azure DevOps. Right now the devs are merging directly into Master, and there are a number of internal devs and external contractors. I want to protect the main branch via hard limit policies. I know someone might suggest a solution to "just tell the devs to stop doing that", but unfortunately that never works. People are fallible, habits are hard to break, and "policies" go out the window when under a crunch or pressure to get something out quickly.
What I need to accomplish in DevOps:
Master cannot be deleted or rewritten
Nobody can push or merge directly to Master
Master can only be changed by a PR, preferably only from the Dev branch
Dev cannot be deleted or rewritten
Nobody can push or merge directly to Dev
Dev can only by change by a PR from an existing branch
Tagged branches cannot be deleted or rewritten
If someone has a suggestion how to accomplish this I'm all ears, and very thankful.
You need to setup a Branch Policy for the branches you want to do this on, this note is on that page
Branch Policies
Note: If any required policy is enabled, this branch cannot be deleted and changes must be made via pull request.
To get to Branch Policies -> in Project Settings choose Repositories then the particular Repository then Branch Policies are at the bottom - choose the branch.
As per reading from https://learn.microsoft.com/en-us/azure/devops/repos/git/branch-policies?view=azure-devops&tabs=browser
I have tested out the guide and it works for just 1 branch.
My question is that is there a way for me to do it for all of my branches?
Example branches name:
feature/test-1
feature/test-2
feature/test-2-1
rel/test-1
master
I can PR feature/test-2-1 to feature/test-2
I can PR feature/test-2 to rel/test-1
But based on the example, I can only set to 1 branch and if I wanted the 2nd branch to have I need to set it manually.
Is there a way I can set that ALL PR must pass the validation before accepting it? No matter what is the branch name.
In an single Git repository on Azure DevOps, you can only set protection policy for each branch individually.
However, on the project-level, you can set the Cross-repo branch policies that can be applied to all the matched branches in all repositories in current project.
[UPDATE]
Since the the Cross-repo branch policies will be applied to all the matched branches in all repositories in a project.
If you want the policies only can be applied to all the branches in a particular repository, you can only set the policies for each branch individually.
If you really need this feature, I recommend that you can report a feature request on Developer Community. That will make it possible for you to interact with the appropriate engineering team, and make it more convenient for the engineering team to collect and categorize your suggestions.
We have two developers using the same ADF. Each developer creates a git branch and starts working on it. Each developer can save the changes to their own git branch but there can only be one collaboration branch and this branch decides the publishing branch. This is causing a blockade (for one of the developer. How can we solve this ?
ADF publish branch can be set using a publish_config.json but now there is an option to set this in the adf itself. which one takes precedence? What is the best practice here?
You need to manage the work of each developer with standard git branch/merge processes. When one dev is done with work in their feature branch, then they will create a pull request to merge changes into your collaboration branch.
If the second dev has not created a feature branch yet, they can just do so after the pull request from the first dev is complete and then continue work from there. If the second dev has already created a feature branch, then they will need to merge the new changes from the collaboration branch into their feature branch to continue work before later committing to git and creating a pull request to merge changes from their feature branch back into the collaboration branch. From there, you can publish as needed.
This git work can be done through the ADF editor as well as through any other git interface you have. It's up to you.
This article discusses the process in specific detail using the ADF editor.
EDIT:
I believe you now have answers for this from 3 of the other 5 questions you posted about this same topic in the past day.
ADF publish confusion in git mode
Azure data factory working-branch confusion
When ADF publish branch is git protected how to publish?
Here is another article which describes the fundamental git process for ADF to help bring you up to speed with the fundamentals of how the different branches work, and how you can switch publish branches on the fly if needed.
Updated 26-Feb-2020
In our Project, I have a pipeline "MyPipeline" which restores the NuGet packages, builds the solutions and runs the tests.
On the master branch, I have a policy which does things like add code reviewers, and it has a "build validation" step which executes "MyPipeline". All well and good.
However, I created another branch from Master called NewBranch and synced (pushed) this up to Azure. After doing some minor changes in Visual Studio, I did a merge from master, committed and sync'd to Azure.
I was a bit surprised then to see "MyPipeline" executing. It seems to have been triggered when I pushed my changes to NewBranch to Azure. I don't have a branch policy on "NewBranch". The trigger in the YAML file is:
trigger:
- master
What kicked this off? I'll soon burn through my free agent minutes if this continues...
Update 26-Feb-2020
As per the comments below:
The history of commits on the master branch are:
Tue 9:10 PM
Tue 7:47 PM
Mon 2:46 PM
The history of the pipeline execution show:
Wed 9:14 AM "PR Automated"
So, nothing new has been committed to the Master branch. However, I think I know what's causing this. Just that I'm not convinced on the timing....
So, we have two branches, master and NewBranch.
Master has a policy that requires two code reviewers to Approve, and it requires the build to succeed. Because of this policy, a developer can't therefore merge directly into master - they have to generate a Pull Request.
So, Developer A creates a Pull Request to merge NewBranch to Master. There is then the rather lengthy code review process which may take multiple additional commits to the "NewBranch"'s Pull Request before it's deemed acceptable.
What seems to be happening is that everytime one of these new Commits is sync'd to the Pull Request, a build is triggered. Is this a good thing? Maybe, Maybe not. If the build is going to trigger just once, then the compilation should occur when all the code in the Pull Request has been approved, not before hand. Why trigger a build at such an early stage; the master branch may be updated by multiple other Pull Requests before this is ready to be merged in. However, with unlimited resources, then I guess there's no harm in building as often as possible, but a) this can delay other builds (representing an impediment to other developers) and b) this uses up the free limit on agent time.
The detailed info you share in question is much help for me to understand the whole workflow of yours.
Though you did not express too clear, but, yeah, what you are guessing is correct. The action you faced is expected and by designed. The root cause is you are using branch policy and the Build validation also included into this policy.
The nutshell is you are using Pull request trigger.
Explanation:
Let's pay attention to its definition:
Pull request (PR) triggers cause a build to run whenever a pull request is opened with one of the specified target branches, or when
changes are pushed to such a pull request.
Based on your added contents, your developers are pushing changes (new commit) into the opening Pull request (Note: The key point is Pull request is still be opening.) This belongs to the work scope of the PR trigger because of above definition. That is why the build triggered every y new changes pushed into NewBranch branch.
Work around:
I agree with the logic of #devpro's answer. But its script does not available for your scenario. Because the pr in YAML only work for GitHub and Bitbucket Cloud repos.
Here you are using VSTS repo, and configured the policy for it. So, you can only via branch policy configuration to avoid such burning trouble.
In your Build validation panel, you are setting build policy with Automatic, right?
Please change the Automatic to Manual, also keep the Policy requirement value as Required.
Now, the corresponding build pipeline will not be ran automatically once new commit pushed, it can only built when someone run it manually.
For the delay timing you noticed which make you unsure, I guess it would because of conflict.
For sample, the Pull request P1 which merge from NewBranch to Master is opening. I commit a new change C1 into NewBranch. BUT, it causes the conflict. At this time, the build will not be ran because the changes does not actually pushed into PR.
Note: The commit is true to NewBranch. BUT changes does not accepted by PR yet, because PR detect out there has conflict here and you must tell PR which changes you want to keep. Only the changes pushed into PR can work with PR trigger.
Only the conflict solved, the changes, maybe I can say commit, can truly be accepted and pushed into PR. Then the triggered build ran.
This would be the time period delay you noticed.
I've seen the same behaviour today.
Can you try rewriting the trigger part in your pipeline like this:
trigger:
branches:
include:
- master
Does that help ? I can't verify it myself yet, since I've done that change but the PullRequest isn't approved yet :)
I've also another repository where I do not see that behaviour, but there, my pipeline trigger looks like this:
trigger:
- master
(Notice the 2 spaces before - master)
I think you need to add pr: none to your pipeline definition to disable the automated run.
If pr is not set I think the default is to run on every PR.
That would give something like this:
trigger:
batch: true
branches:
include:
- master
paths:
exclude:
- README.md
pr: none
My requirement is whenever developer try to do check-in existing GitLab repository then before doing check-in in repository,build should trigger (Jenkins build) and Junit test case should run on new check-in and if passes then it should go forward and will allow developer to do check-in in main repository.
I am not sure but is pre-hook commit can achieve this requirement?
While you could achieve this with pre-commit hooks, it's more common to do so with post-commit hooks on the server-side.
You can achieve this by operating a branch based workflow, there are multiple to choose from - I would recommend reading through this guidance by Atlassian.
Developers will create branches from a 'main' branch (often master, but can be a 'dev' branch working towards a release for instance), then develop code on that branch. They will then push their branch and commits to the remote repository (GitLab). When ready to merge into the main branch, your developers can open a merge request onto the main branch.
On GitLab you can setup a webhook to trigger Jenkins builds when a push event occurs. I would recommend this guide to guide you through it.
In the GitLab project settings you can require a passing build before merge requests are allowed to merge.
Furthermore, your understanding of Git seems incorrect - check in is not a term used in Git. Please take a look at the Git documentation. In Git a developer creates commits against a local copy of the repository, then pushes these to a remote repository (GitLab/GitHub etc.). There is no direct equivalent of the 'check in' used in various centralised version control systems e.g. SVN.