We are currently facing a conundrum with our multi-tenant project which contains various configuration files for each of our tenants and their associated environment. Our CI/CD Pipeline is split into two parts
An Upstream pipeline which analyses the new commit to master to
determine which tenants/environments have been changed. This triggers
the downstream pipeline with the correct environment variables via
the API
A downstream pipeline which executes scripts to deploy
changes to the tenants' environment based on the environment variables
passed through. This works well, however we have a Gitlab Runner per
environment to access the customers environment. We use this to avoid
hard-coding multiple credentials within our scripts or CI environment
variables.
Is there a way we can trigger this downstream pipeline with the specific Gitlab Runner? Our Gitlab Runners are tagged per environment so that we can use the passed environment variables to detect which runner it should be ran on.
I’ve had a look around the Gitlab CI, specific runners and shared runners (which ours are currently) but doesn’t seem to be supported.
tags: supports variable expansion. So, it is possible to pass variables in the API call when creating your downstream pipelines in order to control which runners are used.
Related
I am using bitbucket pipeline to run test cases. In order for test cases to succeed, I need secrets which is stored in google secret manager. Is there any way I can access those secrets within bitbucket pipeline environment ?
There are a couple of options.
In case if these secrets are static, the easiest solution would be adding them to your Repository or Deployment variables. Make sure that they're marked as Secured, so that they will be masked, i.e hidden, in the logs.
Alternatively, if your secrets are rotated and must be fetched from the secrets manager on every build in order to stay up-to-date, you'll need to use corresponding CLI commands in the build script. In order for this to work you will have to give Bitbucket Pipelines access to the secrets in your cloud. For details, check out, for example, this page.
I have setup the Gitlab CI/CD for Sandbox environment. Now, I want to setup the same CI/CD for UAT environment. So I have created a different branch for UAT and in the gitlab-ci.yml file made the changes like Runner tag, variables and all.
Our developer merge the code from Sandbox environment to UAT whenever they want to do the deployment in UAT but the issues comes like while merging it through merge conflict error as the Runner tag, variables, env file names are different.
How to overcome this issue?
I have following image configuration in my gitlab-ci.yml:
default:
image: registry.gitlab.com/gitlab-org/terraform-images/stable:latest
#image: curlimages/curl:latest
...
this works fine when I am deploying on https://gitlab.com/, however when you try to deploy my code along with above ci configuration I get following errors in my CICD:
This job is stuck because the project doesn't have any runners online assigned to it.
Go to project CI settings
My question is while using https://gitlab.com/ I didn't specifically assigned any runners to my project through settings. But now it seems I have supposed to do that.
Why is that?
And if this is necessary how can I do this?
When using gitlab.com -- the gitlab instance has shared runners configured available to run all untagged CI jobs.
On your own self-hosted gitlab you must either configure your own shared runners for your instance or register runners to your projects/groups.
You cannot use the gitlab.com shared runners on a self-hosted gitlab instance.
From scope of runners:
Shared Runners
Shared runners are available to every project in a GitLab instance.
Use shared runners when you have multiple jobs with similar requirements. Rather than having multiple runners idling for many projects, you can have a few runners that handle multiple projects.
If you are using a self-managed instance of GitLab:
Your administrator can install and register shared runners by going to your project’s Settings > CI/CD, expanding the Runners section, and clicking Show runner installation instructions. These instructions are also available in the documentation.
The administrator can also configure a maximum number of shared runner pipeline minutes for each group.
If you are using GitLab.com:
You can select from a list of shared runners that GitLab maintains.
The shared runners consume the pipelines minutes included with your account.
I suppose you technically would be able to use public GitLab runners for your self-hosted instance if you create an account on gitlab.com and setup CICD for external repos pointing to your self-hosted instance -- but your minutes would be a separate entitlement from your self-hosted license, among other serious limitations.
Gitlab has a notion of environments. I'd like to define a deployment stage for production, but only allow to run it, if the same commit has been deployed successfully in staging. Is this best done in one pipeline with prod jobs after staging or is there some way to define a dependency for environments?
Since you can:
view environments and their associated deployments
query a specific environment
I would make as a first step of my deployment stage for production one that queries the staging environment, get the deployable/commit id associated to that environment (part of the JSON answer), and compare it with the commit being deployed to production.
If that does not match, the pipeline would stop immediately in error.
I am learning CI/CD concepts and trying to set up a pipe line in Azure to deploy my sample api to dev, int,qa and prod environments. I was able to deployment the build successfully to all environments. But what is the correct way of doing CI/CD in the companies? We cannot have build on every check-in to all environments even though we have the unit testing run in the build. The usual flow should be Unit test then deploy to Stage/QA and once QA signs off then we promote the build to PROD right? How does this fit into the CI/CD pipeline? Also we can have multiple builds in Dev, can we select which build to deploy to stg and prod?
The ideal way would be to link each of your check-ins to some bug/task so that it will be linked to appropriate test case .After the check in automated test cases are run simultaneously and test whether the check in is valid. If it is valid you can line it up to deployment which only will be deployed after getting the necessary approvals.It is recommended to run continuous integration pipeline to each check in so that you will have a history of build success and failures which is a big relief when you have to find/track errors.
To the prod environment you have multiple approvals as requirement to deploy and provide all the approvals when the sign off is provided.
For the build to map to stg and prod .yes you can do this .If you are using web app in azure and you have prod and stage slots then you can point the branch to respective slots in the deployment configuration.
Please let me know if you need anything else.
Hope this helps.