Our team members are not all familiar with git, some members may take a mistake to force push the local branch on to server especially if they are using Windows GUI tools.
I'm wondering if there is any way to control the force push permission for different roles. I googled, but found no answer. I'm using latest Gitlab.
Not yet (should be in GitLab 6.8+): there is a pull request in progress: "pull 6190", which stemmed from this suggestion.
GitLab already has protected branches that prevent push for all except masters. However, there's still the problem of accidental history rewrites and force pushes by masters that can wipe out the whole repository (has happened in real life).
Therefore it is useful to additionally have protection against branch deletion and history rewriting.
The code changes are visible here and are based on a pull request on gitlab-shell, with a definition in lib/gitlab_update.rb:
def forced_push?
missed_refs = IO.popen(%W(git rev-list #{#newrev}..#{#oldrev} --)).read
missed_refs.split("\n").size > 0
end
Related
I am using Azure Devops in an IT environment with many different development teams and git repositories. Each development team owns one or more repositories. It's my job to work on various application components contained in said repositories. Because I do not own those repositories, I should not make any changes in build/release pipelines, build policies etc, all by myself because that can impact other people's work.
Now let's say I have a feature branch named UpgradedFeature in the repository FeatureRepository, containing my changes. Said changes also introduce a breaking change in the build pipeline used for that repository for the master branch. Let's say that pipeline is named MasterBuildPipeline.
So in order for my build based on the branch UpgradedFeature to succeed and not impact other people's work, I make a clone of the MasterBuildPipeline, name it UpgradedFeatureBuildPipeline and configure the breaking changes. This new build pipeline is used exclusively by me for the UpgradedFeature branch only.
The build, now using the new UpgradedFeatureBuildPipeline pipeline succeeds and now I want to merge into master , so I make a pull request to merge the changes contained in UpgradedFeature into master. The master branch has a branch policy in place named MasterBranchPolicy like described on https://learn.microsoft.com/en-us/azure/devops/repos/git/branch-policies?view=azure-devops&tabs=browser. This branch policy contains the MasterBuildPipeline and prevents completion of that pull request when the build using that pipeline does not succeed.
So my problem is that my pull request triggers the MasterBranchPolicy containing the MasterBuildPipeline and not the UpgradedFeatureBuildPipeline containing the necessary breaking changes for the build to succeed. So the build fails and I cannot complete the pull request.
Of course I could edit the MasterBuildPipeline for a short time, introduce my breaking changes, run the build, then discard the breaking changes again. But there's a chance I may impact other people's work with that and somehow I have a feeling that's not the right approach. I could also edit or disable the MasterBranchPolicy for a short time but again, I may impact other people's work and I feel it's not the right approach.
How should I do this?
So the build fails and I cannot complete the pull request.
To complete the pull request even through the build fails, you could grant yourself Bypass permissions. Bypass permissions let you complete pull requests that don't satisfy branch policies. You can grant bypass permissions to yourself then complete the pull request. Here is Bypass branch policies for reference.
Please navigate to Project setting >> Repositories >> The repo >> Security >> user (yourself) >> Bypass policies when completing pull requests.
Then, you can Override branch policies and enable merge even the MasterBuildPipeline faild.
Please also note that use caution when granting the ability to bypass policies, especially at the repo and project levels. Policies are a cornerstone of secure and compliant source code management. In your scenario, it's suggested to edit the MasterBuildPipeline and the MasterBranchPolicy or disable the MasterBranchPolicy as you mentioned.
Generally, workflow of DevOps Branching Strategy as follows
Developer will create a feature or bugfix branch out of develop. One feature or bugfix branch usually stands for one JIRA bug or feature item. These branches are personal
The changes will be pushed into the developer's feature or bugfix branch.
When the new feature or bugfix is complete.A developer will create a pull request. Pull requests open a code review phase.
Once a pull request has been approved, the team lead or development team will move it into development.
When the development branch has all the epics and bug fixes, i.e., the content planned for the next release, the development team or team lead will create a release branch. This initiates the release regression testing phase.At this stage, only bug fixes are accepted for release, and the workflow is similar to that of the development branch.
Having a separate release branch will enable future development towards the next release in the development branch. Features for the next release are not included in this release. However, bug fixes for this release will be incorporated into the development of the next release as well.
When release content meets the criteria, the release branch will be frozen, which means that it ends. Content from releases will be merged to master and tagged there. For the next release, a new release branch is created when needed.
As per my experience, I would suggest creating branching policies like
A pull request is requested to merge the develop, release, and master branches.
Pull request approvers should be leads.
All developers can create feature branches.
All developers can push to hotfix and feature branches. Commit messages must include the JIRA issue id.
We are using Git for a website project where the develop branch will be the source of the test server, and the master branch will serve as the source for the live, production site. The reason being to keep the git-related steps (switching branches, pushing and pulling) to a minimum for the intended user population. It should be possible for these (not extremely technical) users to run a script that will merge develop into master, after being alerted that this would be pushed to live. master cannot be modified by normal users, only one special user can do the merge.
This is where I'm not sure how to integrate this identity change into my code below:
https://gist.github.com/jfix/9fb7d9e2510d112e83ee49af0fb9e27f
I'm using the simple-git npm library. But more generally, I'm not sure whether what I want to do is actually possible as I can't seem to find information about this anywhere.
My intention would be of course to use a Github personal token instead of a password.
Git itself doesn't do anything about user or permission management. So, the short answer is, don't try to do anything sneaky. Rather, use Github's user accounts they way they were intended.
What I suggest is to give this special user their own Github account, with their own copy of the repo. Let's say the main repo is at https://github.com/yourteam/repo, and the special repo is at https://github.com/special/repo.
The script will pull changes from the team repo's develop branch, and merge this into it's own master branch and push to https://github.com/special/repo.
Then, it will push its changes to the team's master branch. This step can optionally be a forced push, since no one else is supposed to mess with master, anyway. (In case someone does, using a forced push here means they have to fix their local repo to match the team repo later on, rather than having the script fail until someone fixes the team repo.)
At the same time, your CI software will notice that master has changed at https://github.com/special/repo, and will publish as you normally would. This is the linchpin: the CI doesn't pay attention to the team repo, so although your team has permission to change it, those changes don't make it into production.
This special user will need commit access to the team repo, in addition to its own GitHub repo. The easiest way is probably to use an SSH key, and run the git push command from the script, rather than trying to use the GitHub API.
Background
I'm working with a large team using git for version control. The normal flow is:
People selecting a ticket from the "backlog queue".
Working on the issue via a local branch (i.e. git checkout -b my_feature_branch).
Making several commits as they go (i.e. git commit).
Pushing local changes to a remote branch in order to "backup" their work so it lives on more than one machine, in case the laptop is damaged or stolen (i.e. git push -u origin my_feature_branch).
Eventually creating a code review on our private github page, and doing a squashed merge from the feature branch to master.
In addition to the remote feature branches created by employees on an as-needed basis, we have several dozen release branches that are used to create the "gold builds" we ship to customers, i.e. 1.00, 1.01, 2.00, 2.01, 2.02, etc.
Problem
Some developers have begun to complain that there are too many branches, and I tend to agree. Some developers haven't been diligent about cleaning up old branches when they are no longer needed (even though github provides a one-button delete feature for this once the code review is complete).
Question
Is there a way to configure our company github deployment so that, when people use git branch via the CLI:
Only our "important/release/gold" branches appear.
The one-off developer (temporary) branches only appear via git branch -a?
The main goal of this is to reduce clutter.
Edit: I found a similar question, but the only answer is not at all applicable (don't use remote branches), which violates my key constraint of allowing people to push to remote branches as a form of data backup. The concept of private namespaces, as hinted by #Mort, seems to be exactly what I'm looking for. Now, how do I accomplish that?
Long story short: you can - but it may be a bit tricky.
You should use the namespace concept (give a look here: gitnamespaces)
Quoting from the docs:
Git supports dividing the refs of a single repository into multiple namespaces, each of which has its own branches, tags, and HEAD. Git can expose each namespace as an independent repository to pull from and push to, while sharing the object store
and
Storing multiple repositories as namespaces of a single repository avoids storing duplicate copies of the same objects, such as when storing multiple branches of the same source.
To activate a namespace you can simply:
export GIT_NAMESPACE=foo
or
git --namespace=foo clone/pull/push
When a namespace is active, through git remote show origin you can see only the remote branches created in the current namespace. If you deactivate it (unset GIT_NAMESPACE), you will see again the main remote branches.
A possible workflow in your situation may be:
Create a feature branch and work on it
export GIT_NAMESPACE=foo
git checkout -b feature_branch
# ... do the work ...
git commit -a -m "Fixed my ticket from backlog"
git push origin feature_branch # (will push into the namespace and create the branch there)
Merging upstream
unset GIT_NAMESPACE
git checkout master
git pull (just to have the latest version)
git merge --squash --allow-unrelated-histories feature_branch
git commit -a -m "Merged feature from backlog"
git push # (will push into the main refs)
The tricky part
Namespace provides a complete isolation of branches, but you need to activate and to deactivate namespace each time
Pay attention
Pay attention when pushing. Git will push in the current namespace. If you are working in the feature branch and you forgot to activate the namespace, when pushing, you will create the feature branch in the main refs.
It seems as if the simplest solution here, since you're using GitHub and a pull-request workflow, is that developers should be pushing to their own fork of the repository rather than to a shared repository. This way their remote feature branches aren't visible to anybody else, so any "clutter" they see will be entirely their own responsibility.
If everything lives in a single repository, another option would be to set up a simple web service that receives notifications from github when you close a pull request (responding to the PullRequest event). You could then have the service delete the source branch corresponding to the pull request.
This is substantially less simple than the previous solution, because it involves (a) writing code and (b) running a persistent service that is (c) accessible to github webooks and that (d) has appropriate permissions on the remote repository.
The first answers are good. If you can fork repositories and use pull-requests or just keep the branches for yourself, do it.
I will however put my 2 cents in case you are in my situation : a lot of WIP branches that you have to push to a single repository since you work on multiple workstations, don't have fork possibilities, and don't want to annoy your fellow developers.
Make branches starting with a specific prefix, i.e. wip/myuser/, fetch from / push to a custom refspec, i.e. refs/x-wip/myuser/*.
Here is a standard remote configuration after a clone:
[remote "origin"]
url = file:///c/temp/remote.git
fetch = +refs/heads/*:refs/remotes/origin/*
To push branches starting with wip/myuser/ to refs/x-wip/myuser/, you will add:
push = refs/heads/wip/myuser/*:refs/x-wip/myuser/*
This will however override the default push rule for the normal branches. To restore it, you will add:
push = refs/heads/*:refs/heads/*
Finally, to fetch you WIP branches that are now outside the conventional refs/heads/* refspec, you will add:
fetch = +refs/x-wip/myuser/*:refs/remotes/origin/wip/myuser/*
You will end up with this 2nd remote configuration:
[remote "origin"]
url = file:///c/temp/remote.git
fetch = +refs/x-wip/myuser/*:refs/remotes/origin/wip/myuser/*
fetch = +refs/heads/*:refs/remotes/origin/*
push = refs/heads/wip/myuser/*:refs/x-wip/myuser/*
push = refs/heads/*:refs/heads/*
(Git evaluates fetch / push rules from the top to the bottom, and stops as soon as one matches; this means you want to order your rules from the most to the less specific rule.)
People using the standard remote configuration will only fetch branches from refs/heads/*, while you will fetch branches from both refs/heads/* and refs/x-wip/myuser/* with the 2nd configuration.
When your branch is ready to be "public", remove the wip/myuser/ prefix.
The refspec internal documentation was useful to make it.
Please note that once you have push rules in your remote configuration, running the command...
git push
... with no arguments will no longer only push your current branch, nor use any strategy defined with the push.default configuration. It will push everything according to your remote push rules.
You will either need to always specify the remote and the branch you want to push, or use an alias as suggested in this answer.
In our office we are using Stash without server access (maximum access level is create branch) so we need to restrict a particular Git branch from the pushes. So simply, what we need to do is we need to freeze our branch after the release.
Currently what we are doing is we replicate the feature branch and remove that branch from the remote, but that is bit risky.
Are there any other possible solutions?
You should be able to associate to that branch a permission which suits your need.
See "Stash: Using branch permissions".
By limiting write access to a specific owner, for instance, you remove push for all other contributors.
Technically, you could set up a hook that forbids pushing to a certain branch. This is discussed in Is there a way to lock a branch in GIT.
However, your statement
what we need to do is we need to freeze our branch after the release
makes me think that you actually want to use git tag instead of making a branch read-only.
Are gitlab deployment keys read only?
I need to clone on ci server using a deployment key and then push the tag created by the ci process.
is that possible using a deployment key?
Edit2: This change currently has a sideeffect, as there are no users on deployment keys. So you will find some ugly messages like ERROR -> POST-RECEIVE: Triggered hook for non-existing user. The cache-invalidation (and possibly other things) are therefor not handled on write pushes through deployment-keys, which is a bit ugly. bundle exec rake cache:clear RAILS_ENV=production is your friend until I can find a way to fix that (see link to GitHub below) - and please remember, that clearing the Redis-cache logs out all users, too.
Here is a short workaround to archive following on a per-key basis:
Make My SSH keys readonly. This allows to add machines with readonly access to all repos of an account. Good if you work with trainloads of git subprojects.
Make Deploy-Keys read-write on developer level.
Make Deploy-Keys read-write on master level.
This is archived by prepending the key's title with some special characters:
* to make a My SSH keys readonly
! to make Deploy-Keys read-write
!! to make Deploy-Keys master (write to protected branches)
So instead naming the key "My global key" you name it "* my global readonly key" etc.
diff --git a/lib/api/internal.rb b/lib/api/internal.rb
index ed6b50c..ce350b1 100644
--- a/lib/api/internal.rb
+++ b/lib/api/internal.rb
## -30,7 +30,11 ## module API
if key.is_a? DeployKey
- key.projects.include?(project) && DOWNLOAD_COMMANDS.include?(git_cmd)
+return false unless key.projects.include?(project)
+return true if DOWNLOAD_COMMANDS.include?(git_cmd)
+return true if key.title.start_with? '!!'
+return false if project.protected_branch?(params[:ref])
+key.title.start_with? '!'
else
user = key.user
## -42,6 +46,7 ## module API
then :download_code
when *PUSH_COMMANDS
then
+return false if key.title.start_with? '*' # VAHI 2014-02-09
if project.protected_branch?(params[:ref])
:push_code_to_protected_branches
else
Edit1: This patch now is available as cherry-pick on GitHub, see https://github.com/hilbix/gitlabhq/wiki
Edit3: You probably want following workaround, too, in case you push through a deployment key. This is not perfect, as it does not trigger all those hooks, but it invalidates the caches such that the web pages no more show stale data:
diff --git a/app/workers/post_receive.rb b/app/workers/post_receive.rb
index 6416aa6..2fe98d4 100644
--- a/app/workers/post_receive.rb
+++ b/app/workers/post_receive.rb
## -26,6 +26,8 ## class PostReceive
unless user
log("Triggered hook for non-existing user \"#{identifier} \"")
+ project.ensure_satellite_exists
+ project.repository.expire_cache
return false
end
There still is are some problems left:
A minor one, you cannot use the same key on the account level and on the deploy level at the same time. So for keys which only have read-only access on a global basis (which probably is the default key used), you need a second special "push only" key, which allows push access to the repos.
Edit3: And the major one that deployment keys do not have a user attached, so that all the convenience things do not work. If this is a problem for you the only way is, for each SSH key create a dummy-user, and add it to the group/project and give this dummy-users the correct permissions.
Complete example under Linux for a test repo at git#gitlab.example.com:root/test.git
Apply above patch to GitLab
Restart GitLab to read in the new code
Add your ~/.ssh/id_rsa.pub to GitLab Admin under My SSH keys and name it * my readonly key (or something different, starting with *).
Verify, that following works: git clone git#gitlab.example.com:root/test.git
Verify, that following fails on the git push step:
cd test
date > DATE.tmp
git add DATE.tmp
git commit -m testing
git push
Create a second SSH key ~/.ssh/id_push: ssh-keygen -b 4096 -f ~/.ssh/id_push
Add ~/.ssh/id_push.pub as Deploy-Key to repo root/test.git. Name it ! my push key (or something different, starting with !)
Add following to ~/.ssh/config
Host gitlab-push
Hostname gitlab.example.com
User git
IdentityFile ~/.ssh/id_push
Add this target to your cloned repo:
git remote add to gitlab-push:root/test.git
Verify following works: git push to master
Notes:
I used copy+paste to insert the patch. It is likely it will not apply cleanly. Sorry.
Yes, this is really a very crude hack which should not make it into the mainline. The correct implementation would be to have a flag in the database which does this, such that you can edit it through the GUI.
For deploy keys this "key-level"-flag should be in the interesection table between key and project. And in the non-deploy-key variant it should be on the key itself. Perhaps this field can then act as a default value when adding a deploy key to another project and record the last usage of such key.
Unfortunately I am not able to implement this properly myself as I lack the knowledge how to add the necessary elements to the GUI, sorry. ;(
Update 2017
GitLab 8.16 (January 2017) does introduce deploy keys with write-access!
Merge Request 5807
Now with ability to add write-access on deploy key, we can build packages like releases, makes the tag (through CI) and prepare the next release, and of course push both of them commits to git repository
Update Feb. 2021 and GitLab 13.9
Allow Deploy Keys to push to protected branches
Prior to GitLab 12.0, Deploy Keys with write access could push commits to protected branches.
Support for this was removed due to security concerns, but many users still requested it, as they were using it to ensure that only users with Deploy Keys could push to their repositories.
It also eliminates the need to use a service user or machine user, which ties up a license for any team that wants to allow Deploy Keys to push to protected branches just for this use case.
We are excited to announce that we resolved this issue and now Deploy Keys can push to protected branches once more while abiding by security best practices. By moving towards an isolated permission model for Deploy Keys, users can now select Deploy Keys to link to protected branches directly from the settings page on protected branches.
See Documentation and Issue.
Original answer 2013:
Last time I checked (in "Push to GitLab repository within CI server (deploy keys)", no, you don't have the right to use a deploy key to push to a repo.
I think giving deploy keys push access is misguided. It solves the problem on the wrong end.
When you have to hot patch production systems (while running?) and push changes back you are probably doing it wrong.
Changes should always flow from the development to the production system (this should be automated!).
Make your dev env as similar to your production env as possible (use VMs or dedicated dev/staging servers) and write tests (really do!).
Litmus adds in the comments, and I agree with him:
not just in GitLab, even on Bitbucket, Github etc.: deploy keys are readonly.
Given that deploy keys are used to deploy on production, it should be a oneway flow. Code should go from DVCS to production but never the other way.
Also production servers should have as less privilege as possible... that is a security best practice.
CI runs in test environment.
Never use same keys for production and test. That will be a disaster
Curt J. Sampson mentions in the comments:
There are other uses for deploy keys that don't relate to deployment.
For example, if you need to mirror a repo from elsewhere into your own GitLab, you likely want your mirror script to push to your GitLab using a deploy key rather than someone's personal key.
Note, from GitLab 13.5 (October 2020):
Configuration option to allow Deploy Keys to push to protected branches
In release 12.0, we updated Deploy Keys so that keys with write access could no longer push commits to protected branches. As a workaround for this limitation, some users removed access restrictions to the master branch, leaving it unprotected and allowing all developers to push to master.
This increases security risks, so in order to provide a better option we have decided to re-enable the previous behavior through a configuration setting.
See Documentation and Issue.
Not just in GitLab, even on Bitbucket, Github etc. deploy keys are readonly.
Given that deploy keys are used to deploy on production, it should be a oneway flow. Code should flow from DVCS to production but never the other way.
Also production servers should have as less privilege as possible... that is a security best practice.
And most often there is a need to share deploy keys with non-developers, or automation tools. Making them readonly (at least) ensures that an unauthorized person does not screw up with the code base (of course git lets you recover almost everything, but isn't prevention better than cure?)
CI runs in test environment. Never use same keys for production and test. That will be a disaster.