I am working with gitlab version 14.10.5.
In the left panel, there is a "Deployments" icone that allow me to create environments.
In a gitlab repository I want to set up pipeline for dev, stage and main branches relating to 3 environments.
In these environments, I want to setup specific variables that will chaneg over environments.
For example, I want the variable NINJA to be setup with the value:
"1" for the branch dev
"2" for the branch stage
"3" for the branch main
So, in my .gitlab-ci.yml I have:
build-testing:
stage: build
script:
- echo "Hello team"
- echo "$NINJA"
environment:
name: testing
only:
refs:
- dev
build-staging:
stage: build
script:
- echo "Hello team"
- echo "$NINJA"
environment:
name: staging
only:
refs:
- stage
build-production:
stage: build
script:
- echo "Hello team"
- echo "$NINJA"
environment:
name: production
only:
refs:
- main
How to setup the variable NINJA in an environment, in gitlab ?
You can set variable scope, so a variable with same name will return different values in in different envs.
Navigate to Settings -> CI/CD
In Variables section press "Add Variable"
Set key as NINJA
Set "Environment scope" with proper env, i.e testing
Repeat 2,3 and 4 with other envs (staging, production)
See also: https://docs.gitlab.com/ee/ci/environments/#scope-environments-with-specs
Related
We have multiple environments (staging, production...) and I don't want to put sentitive informations like database passwords inside the codebase. For this, I want to use environment variables provided by GitLab CI/CD.
However I don't know how to tell GitLab to run a different set of variables depending on my environment.
What I've done so far:
1- Create environments : Via UI (Project => Operations => Environments : Here I created 2 environments, STAGING and PRODUCTION
2- Create variables Via UI (Project => Settings => CI/CD => Variables : Here I created the variable DB_PASSWORD twice (with of course different values assigned) , one with environment scope set to STAGING, the other one to PRODUCTION.
Now what I want to do is run my project's pipeline. So I go to Project => CI/CD => Pipelines => Run Pipeline and here I expect GitLab CI to ask me if I would like to run my pipeline with the set of variables set for STAGING or PRODUCTION but it doesn't.
How I am supposed to tell GitLab that I want to run my pipeline using DB_PASSWORD variable with the value corresponding to the environment I want to target?
You need to specify the environment in your gitlab-ci.yml file, see here
Example from official gitlab docs:
stages:
- test
- build
- deploy
test:
stage: test
script: echo "Running tests"
build:
stage: build
script: echo "Building the app"
deploy_staging:
stage: deploy
script:
- echo "Deploy to staging server"
environment:
name: staging
url: https://staging.example.com
only:
- master
In this example when running deploy_staging the environment is set to staging and thus you can access the defined Variables for the environment, like so:
deploy_staging:
stage: deploy
script:
- echo "Deploy to staging server"
environment:
name: staging
url: https://staging.example.com
variables:
DB_PASS: ${DB_PASSWORD} # which is your defined variable within Gitlab CI
only:
- master
I have set up my GitLab pipeline and I'm using GitLab CI variables to generate my configuration file during the build phase. Now we've set up a couple new environments, with each having its own database and other credentials, so I need to generate my configuration file using each environment's variables based on branch. I've already seen:
https://gitlab.com/gitlab-org/gitlab/-/issues/14223
https://gitlab.com/gitlab-org/gitlab-foss/-/issues/13379
https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/8858
https://medium.com/spacepilots/sourcing-environment-variables-in-gitlab-ci-or-a-poor-mans-dotenv-dfc33ca231df
Some users suggested sourcing environment variables from files but that isn't really a solution because we want to limit access to these variables.
Is there a solution or workaround to this problem in .gitlab-ci.yaml?
Yes. The following example will echo ${TEST_VAR} depending on the branch (dev or main).
.test-job:
stage: test
script:
- echo "${TEST_VAR}"
test-dev-job:
variables:
TEST_VAR: "dev-value"
rules:
- if: $CI_COMMIT_BRANCH == "dev"
extends: .test-job
test-main-job:
variables:
TEST_VAR: "main-value"
rules:
- if: $CI_COMMIT_BRANCH == "main"
extends: .test-job
I am new at gitlab CI/CD Settings. I don't want to put sensitive credentials (like API-Keys, passwords...) into my branch. For this, GitLab (and other CI/CD-Services) are able to set environment variables.
What I have done so far:
Via UI (Project ⇒ Settings ⇒ CI/CD ⇒ Variables)
First go to Project ⇒ Settings ⇒ CI/CD ⇒ Variables and add them like this:
enter image description here
Now here trying to get the File with all your config-values(e.g. with dotenv).
require("dotenv");
module.exports = process.env.NODE_ENV.trim() === "production" ? _config.production : _config.development;
Current .gitlab-ci.yaml file is:
image: node:8.9.0
cache:
paths:
- node_modules/
stages:
- ver
- init
- test
- build
- deploy
ver:
stage: ver
script:
- node -v
init:
stage: init-dev
script:
- npm install
tags:
- dev_server
only:
- dev
variables:
ENV_PRODUCTION: "/builds/AkdiD/8/abcde/projectName/ENV_PRODUCTION"
test:
stage: test
script:
- npm test
build:
stage: build
script:
- echo "BUILD_VERSION=production" >> build.env
artifacts:
reports:
dotenv: build.env
deploy:
stage: deploy-dev
script:
- npm run killcurrent
- echo $ENV_PRODUCTION
- echo $BUILD_VERSION
- npm run staging
tags:
- dev_server
only:
- dev
Question: where do I need to keep this ENV_PRODUCTION file name (yaml file or other place) so that server take that value ??
Edited variable like this- still server it not fetching these variables. Should I change/put something in .gitlab-ci.yml file?
Settings up a Custom environment variables of type File (GItLab 11.11+) does not seem the way to reference/set a list of variables, including ones with sensitive information.
A variable of type file is generally there to represent, for instance, a certificate.
You should define variables, possibly Group-level environment variables
You can define per-project or per-group variables that are set in the pipeline environment.
Group-level variables are stored out of the repository (not in .gitlab-ci.yml) and are securely passed to GitLab Runner, which makes them available during a pipeline run.
For Premium users who do not use an external key store or who use GitLab’s integration with HashiCorp Vault, we recommend using group environment variables to store secrets like passwords, SSH keys, and credentials.
I currently have two jobs in my CI file which are nearly identical.
The first is for manually compiling a release build from any git branch.
deploy_internal:
stage: deploy
script: ....<deploy code>
when: manual
The second is to be used by the scheduler to release a daily build from develop branch.
scheduled_deploy_internal:
stage: deploy
script: ....<deploy code from deploy_internal copy/pasted>
only:
variables:
- $MY_DEPLOY_INTERNAL != null
This feels wrong to have all that deploy code repeated in two places. It gets worse. There are also deploy_external, deploy_release, and scheduled variants.
My question:
Is there a way that I can combine deploy_internal and scheduled_deploy_internal such that the manual/scheduled behaviour is retained (DRY basically)?
Alternatively: Is there is a better way that I should structure my jobs?
Edit:
Original title: Deploy job. Execute manually except when scheduled
You can use YAML anchors and aliases to reuse the script.
deploy_internal:
stage: deploy
script:
- &deployment_scripts |
echo "Deployment Started"
bash command 1
bash command 2
when: manual
scheduled_deploy_internal:
stage: deploy
script:
- *deployment_scripts
only:
variables:
- $MY_DEPLOY_INTERNAL != null
Or you can use extends keyword.
.deployment_script:
script:
- echo "Deployment started"
- bash command 1
- bash command 2
deploy_internal:
extends: .deployment_script
stage: deploy
when: manual
scheduled_deploy_internal:
extends: .deployment_script
stage: deploy
only:
variables:
- $MY_DEPLOY_INTERNAL != null
Use GitLab's default section containing a before_script:
default:
before_script:
- ....<deploy code>
job1:
stage: deploy
script: ....<code after than deploy>
job2:
stage: deploy
script: ....<code after than deploy>
Note: the default section fails to function as such if you try to execute a job locally with the gitlab-runner exec command - use YAML anchors instead.
I need some direction here. I'm reading whatever documentation I can find online but it's not hitting the right synapses or I haven't found the right link yet. On a merge request to a deployable environment, I want to kick off a build on two separate machines. Both machines are IBM Is, running different versions of the OS. I'd like for these builds and subsequent deploys to happen independently of each other.
My .yml file has the entries for the build for the two machines (QQDEV & BNADEV), but the builds occur sequentially, not in parallel. The picture below is what Gitlab draws.
To me, from the above picture, it looks like both build_BNADEV and build_QQDEV are going to run the deploy jobs DEV_BNADEV and DEV_QQDEV. I want build_BNADEV to run DEV_BNADEV, et al, and that is a separate issue aside from the parallel builds.
What do I need here? Another runner? Another pipeline? Just looking for general pointers and direction here.
Here is my YAML.
stages:
- build
- deploy
build_QQDEV:
variables:
THING: "This is a THING for build for QQDEV"
script:
- "bash ./GitLabCI/GitLabCI.Build.sh qqdev"
stage: build
only:
- DEV
- QA
- UAT
- PROD
build_BNADEV:
variables:
THING: "This is a THING for build for BNADEV"
script:
- "bash ./GitLabCI/GitLabCI.Build.sh bnadev"
stage: build
only:
- DEV
- QA
DEV_QQDEV:
variables:
THING: "This is a THING for deploy_DEV_QQDEV"
ASPGRP: "*NONE"
script:
- "bash ./GitLabCI/GitLabCI.Deploy.sh QQDEV EPDEV1_5 /home/quikq/1.5/dev"
stage: deploy
environment:
name: DEV
only:
- DEV
DEV_BNADEV:
variables:
THING: "This is a THING for deploy_DEV_BNADEV"
REBUILD_DEPLOYMENT: "0"
ASPGRP: "DATADEV"
script:
- "bash ./GitLabCI/GitLabCI.Deploy.sh BNADEV EPDEV1_5 /home/quikq/1.5/dev"
stage: deploy
environment:
name: DEV
only:
- DEV
QA_QQDEV:
variables:
THING: "This is a THING for deploy_QA_QQDEV"
ASPGRP: "*NONE"
script:
- "bash ./GitLabCI/GitLabCI.Deploy.sh QQDEV EPQA1_5 /home/quikq/1.5/qa"
stage: deploy
environment:
name: QA
only:
- QA
QA_BNADEV:
variables:
THING: "This is a THING for deploy_QA_BNADEV"
REBUILD_DEPLOYMENT: "0"
ASPGRP: "DATADEV"
script:
- "bash ./GitLabCI/GitLabCI.Deploy.sh BNADEV EPQA1_5 /home/quikq/1.5/qa"
stage: deploy
environment:
name: QA
only:
- QA
UAT_QQ:
variables:
THING: "This is a THING for deploy_UAT_QQ"
ASPGRP: "*NONE"
script:
- "bash ./GitLabCI/GitLabCI.Deploy.sh QQ EPUAT1_5 /home/quikq/1.5/uat"
stage: deploy
environment:
name: UAT
only:
- UAT
UAT_QQBNA:
variables:
THING: "This is a THING for deploy_UAT_QQBNA"
ASPGRP: "*NONE"
script:
- "bash ./GitLabCI/GitLabCI.Deploy.sh QQBNA EPUAT1_5 /home/quikq/1.5/uat"
stage: deploy
environment:
name: UAT
only:
- UAT
PROD_QQ:
variables:
THING: "This is a THING for deploy_PROD_QQ"
ASPGRP: "*NONE"
script:
- "bash ./GitLabCI/GitLabCI.Deploy.sh QQ EPPROD1_5 /home/quikq/1.5/prod"
stage: deploy
environment:
name: PROD
only:
- PROD
PROD_QQBNA:
variables:
THING: "This is a THING for deploy_PROD_QQBNA"
ASPGRP: "*NONE"
script:
- "bash ./GitLabCI/GitLabCI.Deploy.sh QQBNA EPPROD1_5 /home/quikq/1.5/prod"
stage: deploy
environment:
name: PROD
only:
- PROD