If I add variables(build stage), the stages does not work. Where exactly am I going wrong. If remove the variables under only variables section, it runs.
build:
stage: build
script:
- echo "Build is running"
only:
changes:
- Dockerfile
- requirements.txt
- ./configs/*
variables:
- $BUILD == "True"
development:
stage: development
script:
- echo "development"
except:
variables:
- $BUILD == "False"
development_build:
stage: development_build
script:
- echo "BUILD OK"
rules:
- if: $BUILD == "True"
when: always
The documentation mentions
only:variables / except:variables
Use the only:variables or except:variables keywords to control when to add jobs to a pipeline, based on the status of CI/CD variables.
Keyword type: Job keyword. You can use it only as part of a job.
Possible inputs: An array of CI/CD variable expressions.
First: try and use rules:if to see if the issue persists.
job1:
variables:
VAR1: "variable1"
script:
- echo "Test variable comparison"
rules:
- if: $VAR1 == "variable1"
Second, try with a different variable name, in case BUILD is somehow reserved.
Check also where you have BUILD defined (is there a workflow:rules:variables section?)
Related
I'm following the gitlab docs for multi-project pipelines. I'm running on gitlab.com (not enterprise/self-hosted).
I have successfully set up a multi-project pipeline. My question is - is there a way to pass some but not all variables between stages?
Here's a very simple build script for two projects:
Main project:
variables:
THIS_PROJECT_NAME: trigger-source
SHARED_ARGUMENT: "hello world!"
stages:
- build
- downstream
build-code-job:
stage: build
script:
- echo "${THIS_PROJECT_NAME}"
- echo "${SHARED_ARGUMENT}"
run-trigger-job:
stage: downstream
inherit:
variables: false
variables:
SHARED_ARGUMENT: $SHARED_ARGUMENT
trigger: my-org/triggers_dest
Triggered project:
variables:
THIS_PROJECT_NAME: trigger-dest
SHARED_ARGUMENT: "overwrite me"
stages:
- test
triggered-job:
stage: test
script:
- echo "${THIS_PROJECT_NAME}"
- echo "${SHARED_ARGUMENT}"
only:
- pipelines
when I run this with inherit: variables: false, the output in the triggered project's builds just show the default variables (no variables are passed):
$ echo "${THIS_PROJECT_NAME}"
trigger-dest
$ echo "${SHARED_ARGUMENT}"
overwrite me
However, when I use inherit: variables: true, all variables are passed, except the value of SHARED_ARGUMENT is actually written as the literal "$SHARED_ARGUMENT, which then gets expanded to "overwrite me":
$ echo "${THIS_PROJECT_NAME}"
trigger-source
$ echo "${SHARED_ARGUMENT}"
overwrite me
This is the opposite of what I want! Essentially I want to whitelist variables to pass through, rather than blacklisting them as above. Any way to do this?
Found the answer buried in the docs on the inherit: variables keyword. In addition to true/false, you can specify a list of variables to inherit.
Changing the source project's .gitlab-ci.yml to the following:
variables:
THIS_PROJECT_NAME: trigger-source
SHARED_ARGUMENT: "hello world!"
stages:
- build
- downstream
build-code-job:
stage: build
script:
- echo "${THIS_PROJECT_NAME}"
- echo "${SHARED_ARGUMENT}"
run-trigger-job:
stage: downstream
inherit:
variables:
- SHARED_ARGUMENT
trigger: my-org/triggers_dest
results in the desired output:
$ echo "${THIS_PROJECT_NAME}"
trigger-dest
$ echo "${SHARED_ARGUMENT}"
hello world!
I have the following
stages:
- stage1
- stage2
variables:
MY_ENV_VAR: env_$CI_JOB_ID
stage1_build:
stage: stage1
script:
- echo $MY_ENV_VAR
stage2_build:
stage: stage2
script:
- echo $MY_ENV_VAR
I get different values for $MY_ENV_VAR in the two stages (which means $CI_JOB_ID changes on every stage).
What I want is set $MY_ENV_VAR once with one value of $CI_JOB_ID and make it a constant, so that the same value of $MY_ENV_VAR is used across all stages.
Use $CI_PIPELINE_ID instaed, which will be constant across all jobs in the pipeline.
variables:
MY_ENV_VAR: env_$CI_PIPELINE_ID
See predefined environment variables for additional reference.
If you really want an environment variable to be created in one job and persist for the rest of the pipeline, you can pass variables between jobs using artifacts:reports:dotenv.
stages:
- stage1
- stage2
set_env:
stage: .pre
script:
echo "MY_ENV_VAR=env_$CI_JOB_ID" > .myenv
artifacts:
reports:
dotenv: .myenv
stage1_build:
stage: stage1
script:
- echo $MY_ENV_VAR
stage2_build:
stage: stage2
script:
- echo $MY_ENV_VAR
I have a gitlab-ci like this:
stages:
- calculation
- execution
calculation-job:
stage: calculation
script: ./calculate_something_and_output_results.sh
tags:
- my-runner
execution-job:
stage: execution
script: ./execute_something_with_calculation_results.sh foo
tags:
- my-runner
The foo argument in execution-job is base on the results of calculation-job. I want to pass the results from one job to another job via variables. How can I do that?
If you're looking to get the results without storing a file anywhere you can use artifacts: reports: dotenv. This is taken entirely from DarwinJS shared-variables-across-jobs repo.
stages:
- calculation
- execution
calculation-job:
stage: calculation
script: - |
# stores new or updates existing env variables, ex. $OUTPUT_VAR1
./calculate_something_and_output_results.sh >> deploy.env
tags:
- my-runner
artifacts:
reports:
#propagates variables into the pipeline level, but never stores the actual file
dotenv: deploy.env
execution-job:
stage: execution
script: - |
echo "OUTPUT_VAR1: $OUTPUT_VAR1"
./execute_something_with_calculation_results.sh foo
tags:
- my-runner
AFAIK it is not possible to pass a variable directly from one job to another job. Instead you have to write them into a file and pass that as artifact to the receiving job. To make parsing of the file easy, I recommend to create it with bash export statements and source it in the receiving job's script:
calculation-job:
stage: calculation
script:
- ./calculate_something_and_output_results.sh
- echo "export RESULT1=$calculation_result1" > results
- echo "export RESULT2=$calculation_result2" >> results
tags:
- my-runner
artifacts:
name: "Calculation results"
path: results
execution-job:
stage: execution
script:
- source ./results
# You can access $RESULT1 and $RESULT2 now
- ./execute_something_with_calculation_results.sh $RESULT1 $RESULT2 foo
tags:
- my-runner
needs: calculation-job
Note the ./ when sourcing results might be necessary in case of a POSIX compliant shell that does not source files in the current directory directly like, for example, a bash started as sh.
As a simpler version of what #bjhend answered (no need for export or source statements), since GitLab 13.1 the docs. recommend using a dotenv artifact.
stages:
- calculation
- execution
calculation-job:
stage: calculation
script:
# Output format must feature one "VARIABLE=value" statement per line (see docs.)
- ./calculate_something_and_output_results.sh >> calculation.env
tags:
- my-runner
artifacts:
reports:
dotenv: calculation.env
execution-job:
stage: execution
script:
# Any variables created by above are now in the environment
- ./execute_something_with_calculation_results.sh
tags:
- my-runner
# The following is technically not needed, but serves as good documentation
needs:
job: calculation-job
artifacts: true
If you have a job after the calculation stage that you don't want to use the variables, you can add the following to it:
needs:
job: calculation-job
artifacts: false
I am trying to pass the variable to parallel/matrix and do not see that getting expanded and the job failing. This is being set in the job from the environment variable. I am trying to echo the variable in script and see it shows the right value, but does not get substituted in parallel/matrix. Am I missing anything?
.common_deploy:
script:
- |
echo "showing the regions from environment"
echo $qa_regions
echo "showing the regions from job variable"
echo $REGIONS
parallel:
matrix:
- REGION: "${REGIONS}"
DeployToQA:
variables:
ENVIRONMENT: qa
REGIONS: $qa_regions
extends:
- .common_deploy
stage: deploy
rules:
- if: '$CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == "master"'
allow_failure: true
Here the variable $qa_regions has the value of "us-west-2,us-east-1", I was expecting to see the 2 jobs for those regions , but i am seeing the job as DeployToQA: [${REGIONS}]
Variable expansion for the parallel keyword is currently not supported. There is an open issue for this request.
You can take a look at the documentation where variables can be used.
I'm setting a variable in a job in the pipeline and use it in the artifacts / dotenv.
It doesn't seem to work to use the variable as a condition in the rules.
How can I set a env variable and use it in another Job's rule?
Thanks a lot for your thoughts!
include:
- local: '/gitlab/cicd/.gitlab-ci_test.yml'
stages:
- build
- test
build_rule:
stage: build
rules:
- if: $CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_MESSAGE =~ /R::/m
script:
- echo "Hello World"
- |
echo "RUNTESTS=1" > gitlabcicd.env
artifacts:
reports:
dotenv: gitlabcicd.env
test_always:
stage: test
script:
- echo "TestEnv" $RUNTESTS
# prints TestEnv 1
test_sometimes:
stage: test
rules:
- if: $RUNTESTS
# $RUNTESTS == "1" doesnt work either
script:
- echo "Runs only if variable was set"
# doesnt run, even if it prints above
As I posted in a comment before, but just so people can find it more easily, the answer is that, unfortunately, right now it is not possible to do that.
There is an issue about it here: https://gitlab.com/gitlab-org/gitlab/-/issues/235812
No satisfactory workaround that I'm aware of.
Is needed to load the .env file on every job
"Set variables":
stage: preparation
script:
- echo version=1.1 > varfile
- echo foo=bar >> varfile
artifacts:
paths:
- varfile
Dothings:
stage: run
script:
- load varfile
- echo ${foo}