We have multiple environments (staging, production...) and I don't want to put sentitive informations like database passwords inside the codebase. For this, I want to use environment variables provided by GitLab CI/CD.
However I don't know how to tell GitLab to run a different set of variables depending on my environment.
What I've done so far:
1- Create environments : Via UI (Project => Operations => Environments : Here I created 2 environments, STAGING and PRODUCTION
2- Create variables Via UI (Project => Settings => CI/CD => Variables : Here I created the variable DB_PASSWORD twice (with of course different values assigned) , one with environment scope set to STAGING, the other one to PRODUCTION.
Now what I want to do is run my project's pipeline. So I go to Project => CI/CD => Pipelines => Run Pipeline and here I expect GitLab CI to ask me if I would like to run my pipeline with the set of variables set for STAGING or PRODUCTION but it doesn't.
How I am supposed to tell GitLab that I want to run my pipeline using DB_PASSWORD variable with the value corresponding to the environment I want to target?
You need to specify the environment in your gitlab-ci.yml file, see here
Example from official gitlab docs:
stages:
- test
- build
- deploy
test:
stage: test
script: echo "Running tests"
build:
stage: build
script: echo "Building the app"
deploy_staging:
stage: deploy
script:
- echo "Deploy to staging server"
environment:
name: staging
url: https://staging.example.com
only:
- master
In this example when running deploy_staging the environment is set to staging and thus you can access the defined Variables for the environment, like so:
deploy_staging:
stage: deploy
script:
- echo "Deploy to staging server"
environment:
name: staging
url: https://staging.example.com
variables:
DB_PASS: ${DB_PASSWORD} # which is your defined variable within Gitlab CI
only:
- master
Related
Going over this tutorial:
https://docs.gitlab.com/ee/ci/cloud_deployment/ecs/deploy_to_aws_ecs.html
I couldn't find anywhere an explanation how to configure different env variable values for different branches.
Specifically, I am trying to have different ECS cluster names for main and dev. I tried to create these from the CI/CD menu with different values for the main/dev environments but it doesn't work.
This is my gitlab-ci.yml:
include:
- template: AWS/Deploy-ECS.gitlab-ci.yml
When I try printing the env variables with this yml, the variables are printed as expected:
`stages:
debug
print-all-env-vars-job:
stage: debug
script:
- echo "GitLab CI/CD | Print all environment variables"
- env
`
I tried passing variables through the CI/CD menu on Gitlab, and I also tried passing variables to the gitlab-ci.yml like this:
variables: CI_AWS_ECS_CLUSTER: ${CI_AWS_ECS_CLUSTER} CI_AWS_ECS_TASK_DEFINITION: ${CI_AWS_ECS_TASK_DEFINITION}
Any idea how to solve this?
I am working with gitlab version 14.10.5.
In the left panel, there is a "Deployments" icone that allow me to create environments.
In a gitlab repository I want to set up pipeline for dev, stage and main branches relating to 3 environments.
In these environments, I want to setup specific variables that will chaneg over environments.
For example, I want the variable NINJA to be setup with the value:
"1" for the branch dev
"2" for the branch stage
"3" for the branch main
So, in my .gitlab-ci.yml I have:
build-testing:
stage: build
script:
- echo "Hello team"
- echo "$NINJA"
environment:
name: testing
only:
refs:
- dev
build-staging:
stage: build
script:
- echo "Hello team"
- echo "$NINJA"
environment:
name: staging
only:
refs:
- stage
build-production:
stage: build
script:
- echo "Hello team"
- echo "$NINJA"
environment:
name: production
only:
refs:
- main
How to setup the variable NINJA in an environment, in gitlab ?
You can set variable scope, so a variable with same name will return different values in in different envs.
Navigate to Settings -> CI/CD
In Variables section press "Add Variable"
Set key as NINJA
Set "Environment scope" with proper env, i.e testing
Repeat 2,3 and 4 with other envs (staging, production)
See also: https://docs.gitlab.com/ee/ci/environments/#scope-environments-with-specs
I am new at gitlab CI/CD Settings. I don't want to put sensitive credentials (like API-Keys, passwords...) into my branch. For this, GitLab (and other CI/CD-Services) are able to set environment variables.
What I have done so far:
Via UI (Project ⇒ Settings ⇒ CI/CD ⇒ Variables)
First go to Project ⇒ Settings ⇒ CI/CD ⇒ Variables and add them like this:
enter image description here
Now here trying to get the File with all your config-values(e.g. with dotenv).
require("dotenv");
module.exports = process.env.NODE_ENV.trim() === "production" ? _config.production : _config.development;
Current .gitlab-ci.yaml file is:
image: node:8.9.0
cache:
paths:
- node_modules/
stages:
- ver
- init
- test
- build
- deploy
ver:
stage: ver
script:
- node -v
init:
stage: init-dev
script:
- npm install
tags:
- dev_server
only:
- dev
variables:
ENV_PRODUCTION: "/builds/AkdiD/8/abcde/projectName/ENV_PRODUCTION"
test:
stage: test
script:
- npm test
build:
stage: build
script:
- echo "BUILD_VERSION=production" >> build.env
artifacts:
reports:
dotenv: build.env
deploy:
stage: deploy-dev
script:
- npm run killcurrent
- echo $ENV_PRODUCTION
- echo $BUILD_VERSION
- npm run staging
tags:
- dev_server
only:
- dev
Question: where do I need to keep this ENV_PRODUCTION file name (yaml file or other place) so that server take that value ??
Edited variable like this- still server it not fetching these variables. Should I change/put something in .gitlab-ci.yml file?
Settings up a Custom environment variables of type File (GItLab 11.11+) does not seem the way to reference/set a list of variables, including ones with sensitive information.
A variable of type file is generally there to represent, for instance, a certificate.
You should define variables, possibly Group-level environment variables
You can define per-project or per-group variables that are set in the pipeline environment.
Group-level variables are stored out of the repository (not in .gitlab-ci.yml) and are securely passed to GitLab Runner, which makes them available during a pipeline run.
For Premium users who do not use an external key store or who use GitLab’s integration with HashiCorp Vault, we recommend using group environment variables to store secrets like passwords, SSH keys, and credentials.
I run an end2end test in gitlab-CI , see https://docs.cypress.io/guides/guides/continuous-integration.html.
I run it after I deploy my app.
It works well but I want to change the base url in order to run it against my prod or my staging env. It is possible via an environment var passed to the test.
I don’t want to write a test job per environment, then I would like to get the environment URL via env var, but the $CI_ENVIRONMENT_URL is only available on the deploy job, not in the next one.
deploy-prod:
stage: deploy
script:
- some commands
environment:
name: prod
url: http://myprod.com
only:
- master
deploy-staging:
stage: deploy
script:
- some other commands
environment:
name: staging
url: http://mystaging.com
only:
- staging
test:
stage: after-deploy
script:
- CYPRESS_baseUrl=$CI_ENVIRONMENT_URL cypress run
I expect $CI_ENVIRONMENT_URL equal http://mystaging.com or http://myprod.com depending the previous deploy job has been run. But it is empty, seems $CI_ENVIRONMENT_URL is only available in deploy job.
Is it possible to pass a variable from on job to a next job?
You can use artifacts feature: write the $CI_ENVIRONMENT_URL in a file:
echo $CI_ENVIRONMENT_URL > environmentUrl.txt
save it as artifact, and then read it in the next job:
$CI_ENVIRONMENT_URL=`cat environmentUrl.txt`
I use Gitlab runner and works fine for a single server. The gitlab-ci.yml is simple:
stages:
- test
- deploy
test:
stage: test
image: php
tags:
- docker
script:
- echo "Run tests..."
deploy:
stage: deploy
tags:
- shell
script:
- sh deploy.sh
As i said this is fine for a single server but to deploy same app on another server? I tried with same gitlab-runner config (same conf.toml) but then it was only updating one of them randomly.
Is there somehow gitlab Ci to be triggered by more than 1 runner and deploy all of them according gitlab-ci.yml?
You can register several runners (e.g. tagged serverA and serverB) from different servers and have multiple deployment jobs, each of them performed by a different runner. This is because you can set more than one tag in a job and only a runner having all the tags will be used.
stages:
- test
- deploy
test:
stage: test
image: php
tags:
- docker
script:
- echo "Run tests..."
deployA:
stage: deploy
tags:
- shell
- serverA
script:
- sh deploy.sh
deployB:
stage: deploy
tags:
- shell
- serverB
script:
- sh deploy.sh
However, take into account a situation when one of the deployment jobs fails - this would end up in you having two different versions of the code on the servers. Depending on your situation this might or might not be a problem.
Yes there is, just set up two jobs for the same stage:
stages:
- deploy
deploy:one:
stage: deploy
script:
- echo "Hello CI one"
deploy:two:
stage: deploy
script:
- echo "Hello CI two"
If necessary you can use tags on your runners to choose which one to use.
Since 2016, you now have Environments and deployments
Environments describe where code is deployed.
Each time GitLab CI/CD deploys a version of code to an environment, a deployment is created.
GitLab:
Provides a full history of deployments to each environment.
Tracks your deployments, so you always know what is deployed on your servers.
It does integrates well with Prometheis, and, with GitLab 13.11 (April 2021), you even have:
Update a deploy freeze period in the UI
In GitLab 13.2, we added the ability to create a deploy freeze period in the project’s CI/CD settings.
This capability helps teams avoid unintentional deployments, reduce uncertainty, and mitigate deployment risks.
However, it was not possible to update deploy freezes.
In GitLab 13.11, we are adding the ability to edit an existing deploy freeze. This way, you can update the freeze period to match your business needs.
See Documentation and Issue.
As shown in "gitlab-ci.yml deployment on multiple hosts", you can use YAML anchors to trigger parallel deployment on multiple environments, which means "multiple servers".