Going over this tutorial:
https://docs.gitlab.com/ee/ci/cloud_deployment/ecs/deploy_to_aws_ecs.html
I couldn't find anywhere an explanation how to configure different env variable values for different branches.
Specifically, I am trying to have different ECS cluster names for main and dev. I tried to create these from the CI/CD menu with different values for the main/dev environments but it doesn't work.
This is my gitlab-ci.yml:
include:
- template: AWS/Deploy-ECS.gitlab-ci.yml
When I try printing the env variables with this yml, the variables are printed as expected:
`stages:
debug
print-all-env-vars-job:
stage: debug
script:
- echo "GitLab CI/CD | Print all environment variables"
- env
`
I tried passing variables through the CI/CD menu on Gitlab, and I also tried passing variables to the gitlab-ci.yml like this:
variables: CI_AWS_ECS_CLUSTER: ${CI_AWS_ECS_CLUSTER} CI_AWS_ECS_TASK_DEFINITION: ${CI_AWS_ECS_TASK_DEFINITION}
Any idea how to solve this?
Related
I am running the below code section in gitlab-ci.yml file:
script:
- pip install --upgrade pip
- cd ./TestAutomation
- pip install -r ./requirements.txt
Below are the keys and values. So I have to pass any values to the pipeline with key as a variable
ENV : dev
I have added all the above three variables in the GitLab CI CD variables sections by expanding them. just added a single value along with key
I also found like we can add variables in the .yml file itself as below. I am not sure how we can add multiple values for one key
variables:
TEST:
value: "some value" # this would be the default value
description: "This variable makes cakes delicious"
When I run the pipeline I am getting errors as looks like these variables and values are not injected properly.
More details:
And the same error I am getting while running the pipeline. Hence my suspect is like Category variable is not injected properly when I am running through the pipeline
If needed I will show it on the share screen
please find attached an image snippet of my gitlab-ci.yml file- [![enter image description here][1]][1]
I am passing the below parameter while running pipeline -
[![enter image description here][2]][2]
What I have observed is --the values associated with keys which I am passing as parameter or variables , those are not injected or replaced instead of key. So ideally ${Category} should be replaced with value smoke etc
Variables set in the GitLab UI are not passed down to service containers. To set them, assign them to variables in the UI, then re-assign them in your .gitlab-ci.yml:
stages:
- Test
# Added this to your yml file
variables:
ENV: $ENV
BROWSER: $BROWSER
Category: $Category
ui_tests:
stage: Test
image:
name: joyzourky/python-chromedriver:3.8
entrypoint: [""]
tags:
- micro
only:
- develop
when: manual
script:
- pip install --upgrade pip
- cd ./src/Tests/UIAutomation
- pip install -r ./requirements.txt
- pytest -s -v --env=${ENV} --browser=${BROWSER} --alluredir=./reports ./tests -m ${Category}
artifacts:
when: always
path:
- ./src/Tests/UIAutomation/reports/
- ./src/Tests/UIAutomation/logs/
expire_in: 1 day
Please refer attachment it's working with any issue.
When Gitlab CI CD variables are not getting injected into your pipelines as environment variables, please follow the following steps to verify.
Check whether the variable is defined. You need to have at least the Maintainer role setup for your user. Go to Settings --> CI/CD --> Variables. You can see all project variables, and group variables (inherited).
Next, check whether these variables are defined as Protected variables. If they are marked as Protected, then they are only exposed to protected branches or protected tags. I would suggest to uncheck this, if your current branch is not a protected branch. If not you can always make your current branch a protected one.
Next, check whether your code is accessing the environment variables correctly. Based on your scripting language, just access as if you are accessing a regular environment variable.
You don't really need to define these variables in the .gitlab-ci.yaml file. (Even though their documentation says so)
Hope this helps.
As #Keet Sugathadasa mentioned, the branch that triggers the CI must be protected; this was my case so I have to protect it by going to Settings > Repository > Protected branch and then protect the branch from there
We have multiple environments (staging, production...) and I don't want to put sentitive informations like database passwords inside the codebase. For this, I want to use environment variables provided by GitLab CI/CD.
However I don't know how to tell GitLab to run a different set of variables depending on my environment.
What I've done so far:
1- Create environments : Via UI (Project => Operations => Environments : Here I created 2 environments, STAGING and PRODUCTION
2- Create variables Via UI (Project => Settings => CI/CD => Variables : Here I created the variable DB_PASSWORD twice (with of course different values assigned) , one with environment scope set to STAGING, the other one to PRODUCTION.
Now what I want to do is run my project's pipeline. So I go to Project => CI/CD => Pipelines => Run Pipeline and here I expect GitLab CI to ask me if I would like to run my pipeline with the set of variables set for STAGING or PRODUCTION but it doesn't.
How I am supposed to tell GitLab that I want to run my pipeline using DB_PASSWORD variable with the value corresponding to the environment I want to target?
You need to specify the environment in your gitlab-ci.yml file, see here
Example from official gitlab docs:
stages:
- test
- build
- deploy
test:
stage: test
script: echo "Running tests"
build:
stage: build
script: echo "Building the app"
deploy_staging:
stage: deploy
script:
- echo "Deploy to staging server"
environment:
name: staging
url: https://staging.example.com
only:
- master
In this example when running deploy_staging the environment is set to staging and thus you can access the defined Variables for the environment, like so:
deploy_staging:
stage: deploy
script:
- echo "Deploy to staging server"
environment:
name: staging
url: https://staging.example.com
variables:
DB_PASS: ${DB_PASSWORD} # which is your defined variable within Gitlab CI
only:
- master
I am writing a CI/CD pipeline for terraform to deploy GCP resources.
In terraform code, I got many functionality and Network is one of them. The folder structure of the Network is
Network
VPC
LoadBalancer
DNS
VPN
So, I want to loop terraform init, plan and apply commands for all the sub-folder of Network folder. The yml file looks like
image:
name: registry.gitlab.com/gitlab-org/terraform-images/stable:latest
variables:
TF_ROOT: ${CI_PROJECT_DIR}
env: 'prod'
network_services: ""
stages:
- init
init:
stage: init
script:
- |
network_services = ['vpc' 'vpn']
for service in $network_services[#]
do
echo The path is var/$env/terraform.tfvars
done
The above gives me the error:
$ network_services = ['vpc' 'vpn'] # collapsed multi-line command
/bin/sh: eval: line 103: network_services: not found
Please suggest a way to declare array variable in gitlab CI/CD yml.
Try changing your job to something like this:
init:
stage: init
script:
- network_services = ('vpc' 'vpn')
- for service in $network_services[#]
do
echo "The path is var/$env/terraform.tfvars
done
I don't believe you can do multi-line commands like that due to the way the the gitlab-runner eval's the entries in the script array, though combining them with ; has worked for me if I want them on one line.
I am new at gitlab CI/CD Settings. I don't want to put sensitive credentials (like API-Keys, passwords...) into my branch. For this, GitLab (and other CI/CD-Services) are able to set environment variables.
What I have done so far:
Via UI (Project ⇒ Settings ⇒ CI/CD ⇒ Variables)
First go to Project ⇒ Settings ⇒ CI/CD ⇒ Variables and add them like this:
enter image description here
Now here trying to get the File with all your config-values(e.g. with dotenv).
require("dotenv");
module.exports = process.env.NODE_ENV.trim() === "production" ? _config.production : _config.development;
Current .gitlab-ci.yaml file is:
image: node:8.9.0
cache:
paths:
- node_modules/
stages:
- ver
- init
- test
- build
- deploy
ver:
stage: ver
script:
- node -v
init:
stage: init-dev
script:
- npm install
tags:
- dev_server
only:
- dev
variables:
ENV_PRODUCTION: "/builds/AkdiD/8/abcde/projectName/ENV_PRODUCTION"
test:
stage: test
script:
- npm test
build:
stage: build
script:
- echo "BUILD_VERSION=production" >> build.env
artifacts:
reports:
dotenv: build.env
deploy:
stage: deploy-dev
script:
- npm run killcurrent
- echo $ENV_PRODUCTION
- echo $BUILD_VERSION
- npm run staging
tags:
- dev_server
only:
- dev
Question: where do I need to keep this ENV_PRODUCTION file name (yaml file or other place) so that server take that value ??
Edited variable like this- still server it not fetching these variables. Should I change/put something in .gitlab-ci.yml file?
Settings up a Custom environment variables of type File (GItLab 11.11+) does not seem the way to reference/set a list of variables, including ones with sensitive information.
A variable of type file is generally there to represent, for instance, a certificate.
You should define variables, possibly Group-level environment variables
You can define per-project or per-group variables that are set in the pipeline environment.
Group-level variables are stored out of the repository (not in .gitlab-ci.yml) and are securely passed to GitLab Runner, which makes them available during a pipeline run.
For Premium users who do not use an external key store or who use GitLab’s integration with HashiCorp Vault, we recommend using group environment variables to store secrets like passwords, SSH keys, and credentials.
I am importing some secrets from Azure Key Vault to Variable Group to CI / CD pipeline.
I am able to map the required secrets in VariableGroup from KeyVault using Azure Devops UI.
In my pipeline YAML i am able to read and print those VariableGroup variables which are AzureKeyVault secrets.
trigger:
- dev
# define the VM image
pool:
vmImage: "Ubuntu 16.04"
# define variables to use during the build
variables:
- group: SecretVarGroup # it has keyvault variable 'KV_API_KEY'
- group: PublicVarGroup # it has a variable 'API_CLIENTID'
# define the step to export key to env varaiable
steps:
- script: echo $MYSECRETAPIKEY
env:
MYSECRETAPIKEY: $(KV_API_KEY)
## Run the npm build
- script: |
npm run build
displayName: "npm build"
I am able to see value for 'KV_API_KEY' secret printed as *** value in the build output log which i assume its able to consume. I also see value for API_CLIENTID printed in build log as well as node js process.env object.
I was assuming the variable "MYSECRETAPIKEY" will be available in my node js process.env object. But it's not avaialble.
The way i tested it is in my node js project build config i have a print statement which prints process.env object. It printed all the environment variables of pipeline build agent including my PUBLICVARGROUP variable 'API_CLIENTID'. But i don't see my secret variable 'MYSECRETAPIKEY' in the process.env object.
env:
MYSECRETAPIKEY: $(KV_API_KEY)
I thought above line would export variable to specific language process environment. But it is not. How can i fix this?
# define the step to export key to env varaiable
steps:
## Run the npm build
- script: |
npm run build
displayName: "npm build"
env:
MYSECRETAPIKEY: $(KV_API_KEY)
Looks like secrets are scoped on the agent for individual tasks and scripts to use. The issue was i had env: declaraion in a separate adhoc task.Moving it to the same place of my script declaration in the above code has fixed the issue.