I'm wondering what is the best practice in strengthening and enforcing good security on a circleci pipeline. I'd like to ensure that no one should be able to deploy to prod without having their PR approved by another user in the organization.
Circleci offers one functionality which is contexts. These can be used to ensure that only people within a security group are allowed to run certain jobs and therefore access certain env variables. That works mostly fine, except we would like anyone to be able to deploy prod changes given that their changes have been approved by someone else in a PR.
We've setup so merges to master can only be done by approving the PR, but now we're faced with two options:
Only people with access to the context can merge the change (not what we want, it slows us down too much)
We remove contexts (insecure, anyone with access to the repo could change the CI job to print the credentials and steal them). We could give every user with push access also access to the contexts, but then it becomes equally insecure.
What is the best way to tackle this? Are there other best practices for securing the pipelines?
Related
My requirement is to design a framework to enforce custom coding standards (per Gitlab project) pre-push or while raising an MR. We are looking at Gitlab CI/ Webhooks / Server-side githooks.
We faced resistance from Gitlab admins that Server-side githooks are not the right way as it may result in a server overload and lead to Gitlab performance issues.
Also, we want to give the developer the choice to commit with or without fixing the issues that crop up after the check (in case of emergency situations). The check should happen on the server side and not at the client-side (developer). The applications may be backend/front-end with technologies like Java/ReactJS/NodeJS etc.
What would be the best way to implement this?
The best way to do this is to have a build stage that runs a linter for your language, and have it fail if the code doesn't meet the standards.
Then enforce rules so that all merges to master must go through a merge request, and all build stages must pass before a request can be approved.
we are using Azure DevOps as our ALM system. When a user story or bug fix is resolved, it shows in a public query - like a stack - where our QA team members subsequentially pull tickets independently for verification. As this is part of a pull request review, a PR can not be merged unless QA finished testing. So we aim for fast response times and parallelization of testing to minimise the potential of merge conflicts. Often times, we find that multiple work items are self-assigned to the same people, while other team members do not have work items assigned, increasing the potential response times for our devs (unless people change assignments) and leading to a rather subsequential then parallel verification of work items
So we are looking for a way in Azure Dev Ops that allows us to make sure that members of a certain user group can only be assigned one work item of certain work item type and state at the time. We looked into Custom Rules in detail but failed to get anything like this out of it. I'm thankful for any ideas and hints on how this can be accomplished (extensions also welcome)
There is no such rule or policy in Azure DevOps.
And it won't prevent someone from working on it anyway to be honest... I assume testing multiple changes in a single go isn't an option? It would simplefy things tremendously...
I'm trying to setup a git server and I want to allow only a specific user to push commits to master branch.
I have tried to use the Linux group permission setting to meet the requirement above but it seems not a correct way.
And I even don't know what are the key words for searching the answer for this.
Any help would be appreciated.
Git does not allow you to have private branches, but you can achieve this functionality by implementing your own server-side pre-receive hook. Github enterprise specific pre-receive hook example is here, as a reference.
However, if you are using Git hosting services (like Github) they might have an option for this. Github, in particular, has an option called branch restrictions, but it requires you to have a paid subscription, unless your project is public.
You have two options:
By far the easiest solution is to use hosting software that already provides this functionality. You might want to look at GitLab, which has free options for both SaaS (hosted at gitlab.com) and self-managed instances (running your own gitlab instance). Or github. Or bitbucket. Or I'm sure there are others I'm not thinking of.
If you really don't want to use any of those, you can implement access control on a simple git server, but it's not so simple. The short (or rather, glib) answer is "hooks" - but a hook is just a script that runs when something happens - like in this case you'd use the prereceive hook, which runs when someone's trying to push and decides whether to accept the push. Now, how does your hook know who is pushing? (The commit metadata does not indicate who's pushing. What you need is authentication around the actual connection, and visibility of the authentication in your script so that the script may implement your authorization rules. That very quickly breaks down into "it depends on your environment".)
Since it's not really possible to exhaustively cover every scenario for doing this manually, hopefully either you'll find a pre-packaged solution you like, or you'll find the above to be enough to get you pointed in the right direction to do it manually.
Trying to understanding the full workflow of a git-crypt based secret keeping solution.
The tool itself works pretty nicely when on a dev machine, even scaling to multiple developers seems to work fine.
However, it is not clear to me how will this work when deployed to a multiple servers on a cloud, some are created on-demand:
Challenge of unattended creation of GPG key on the new server (someone needs to create the passphrase, or is it in a source control, and than, what is all this even worth?)
Once a GPG is created, how is it being added to the ring?
Say we decide to skip #1 and just share a key across servers, how is the passphrase being supplied as part of the "git-crypt unlock" process?
I've really tried to search, and just couldn't find a good end-to-end workflow.
Like many Linux tools, git-crypt is an example of doing only one thing and doing it well. This philosophy dictates that any one utility doesn't try to provide a whole suite of tools or an ecosystem, just one function that can be chained with others however you like. In this case git-crypt doesn't bill itself as a deployment tool or have any particular integrations into a workflow. Its job is just to allow the git repository to store sensitive data that can be used in some checkouts but not others. The use cases can vary, as will how you chain it with other tools.
Based on the wording of your question I would also clarify that git-crypt is not a "secret keeping solution". In fact it doesn't keep your secrets at all, it just allows you to shuffle around where you do keep them. In this case it enables you to keep secret data in a repository along side non-secret information, but it only does so at the expense of putting the secret keeping burden on another tool. It exchanges one secret for another: your project's version controlled secret component(s) for a GPG key. How you manage the secret is still up to you, but now the secret you need to handle is a GPG key.
Holding the secrets is still up to you. In the case of you and other developers that likely means having a GPG private key file kicking around in your home directory, hopefully protected by a passphrase that is entered into an agent before being dispensed to other programs like git-crypt that call for it.
In the case of being able to automatically deploy software to a server, something somewhere has to be trusted with real secrets. This is often the top-level tool like Ansible or Puppet, or perhaps a CI environment like Gitlab, Travis, or Circle. Usually you wouldn't trust anything but your top level deployment tool with knowing when to inject secrets in an environment and when not to (or in the case of development / staging / production environments, which secrets to inject).
I am not familiar with Circle, but I know with Travis under your projects Settings tab there is an Environment Variables section that you can use to pass private information into the virtual machine. There is some documentation for how to use this. Gitlab's build in CI system has something similar, and can pass different secrets to test vs. deploy environments, etc.
I would suggest the most likely use case for your work flow is to:
Create a special secret variable for use on your production machines that has the passphrase for a GPG key used only for deployments. Whatever you use to create your machines should drop a copy of this key into the system and use this variable to unlock it and add it to an agent.
The deploy script for your project would checkout your git project code, then check for a GPG agent. If an agent is loaded it can try to decrypted the the checkout.
In the case of a developer's personal machine this will find their key, in the case of the auto-created machines it will find the deploy key. Either way you can manage access to the secrets in the deployment environment like one more developer on the project.
Whatever tool you use to create the machines becomes responsible for holding and injecting the secrets, probably in the form of a private key file and a passphrase in an environment variable that is used to load the key file into an agent.
Say I've got a \\Repo\... repo. Currently devs generally tend to do all their work directly in there, which normally isn't a problem for small pieces of work. Every so often, this approach fails for various reasons, mainly because they're unable to submit the incomplete change to Live.
So, I was wondering, is there a way to enforce on the server that:
1) no files can be directly checked out from \\Repo\...
2) users then branch to a private area (\\Projects\...)
3) dev, test, submit, dev, test, submit, ...
4) on dev complete, they can re-integrate back into \\Repo\...
I guess the last part is the problem, as files need to be checked out! Has anyone implemented something similar? Any suggestions are much appreciated.
There is no way (that I know of) to enforce this type workflow in P4. You could try to enforce it by setting commit triggers, restricting permissions, or locking files however I believe it would only result in more work (micro-management) and frustrate you and your team.
The best way to establish and enforce any SCM workflow is to set as company/studio policy. Your team should be responsible/able to follow the set procedure and determine (by themselves or through discussion) if an issue is able to be fixed in the main line.
One note about the proposed workflow; creating a new branch for every issue will eventually cause issues and at some point you will need to perform maintenance on the server to conserve disk space and depot browsing speed.
For more information (over) branching on Perforce read this Perforce blog entry from 2009: Perforce Anti-Patterns Part 2: Overuse of branching.
In many studios using Perforce, most developers have their own "working" branch which they continually re-use whenever there are changes that are not safe or able to be performed in the main line.
if i understand your questions properly, you should try with shelving features and working offline features of Perforce. Process is main thing to achieve success in this senario. So you might need to setup a right process to execute this.
For more Info about shelving and working offline with perforce, you can try following links...
http://www.perforce.com/perforce/doc.current/manuals/cmdref/shelve.html