I'm trying to ensure that my Gitlab CI only runs on a particular host, regardless of the Enable/Disable CI status in the GUI. I have this at the top of my .gitlab-ci.yml file, but CI now runs nowhere. I've confirmed that the host value is as expected by disabling my header and printing it from one of the script jobs.
workflow:
rules:
- if: '$CI_SERVER_HOST != "run-here.example.com"'
when: never
In other words, I'm sure that the value of CI_SERVER_HOST is run-here.example.com, and my pipelines still don't run anywhere. Removing the above header causes them to run everywhere.
You will also have to add an always to run it in other cases.
workflow:
rules:
- if: '$CI_SERVER_HOST != "run-here.example.com"'
when: never
- when: always
Could you check with this workflow.
Related
Consider the following pipeline:
image: alpine
stages:
- test
this-does-not-run:
stage: test
rules:
- if: $CI_PROJECT_DIR == "/builds/test"
script:
- echo "It works!"
this-runs:
stage: test
rules:
- if: $CI_PROJECT_DIR != "/builds/test"
script:
- echo "It doesn't work! CI_PROJECT_DIR- $CI_PROJECT_DIR"
And assume my repo is called test. The output of this CI will always be:
It doesn't work! CI_PROJECT_DIR- /builds/test
Obviously $CI_PROJECT_DIR == /builds/test, so I would expect the output to be It works!. Am I missing something, or is $CI_PROJECT_DIR not respected in rules?
The problem here is that variables in rules: must be expanded (evaluated by) the GitLab server at pipeline creation time, but $CI_PROJECT_DIR is determined by the runner's builds_dir setting, which is only known by the runner at runtime when the job actually runs:
References to unavailable variables are left intact. In this case, the runner attempts to expand the variable value at runtime. For example, a variable like CI_BUILDS_DIR is known by the runner only at runtime.
That is to say, like CI_BUILDS_DIR, you can't use CI_PROJECT_DIR or other runtime-only variables in rules:if:. Instead, you should use a variable that is known to the gitlab server at pipeline creation time, such as CI_PROJECT_NAME CI_PROJECT_ID, CI_PROJECT_URL, CI_PROJECT_PATH_SLUG or similar.
Try to wrap the expression in quotes. Afaik this is an expression and you would need to wrap it.
Eg:
rules:
- if: '$CI_PROJECT_DIR == "/builds/test"'
If you only have variables, then from experience you don't need to wrap it.
Eg:
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
I have some problems with understanding how and why "rules" in GitLab CI work.
I have written a minimal code showing the issue in a GitLab project: https://gitlab.com/daniel.grabowski/gitlab-ci-rules-problems It contains two directories ("files1/" and "files2/") with some files in them and a .gitlab-ci.yml file.
My configuration
Here's the CI configuration:
stages:
- build
workflow:
rules:
- if: $CI_PIPELINE_SOURCE == "push"
.job_tpl:
image: alpine:latest
stage: build
variables:
TARGET_BRANCH: $CI_DEFAULT_BRANCH
rules:
- if: $CI_COMMIT_BRANCH == $TARGET_BRANCH
changes:
- $FILES_DIR/**/*
variables:
JOB_ENV: "prod"
- if: $CI_COMMIT_BRANCH != $TARGET_BRANCH
changes:
- $FILES_DIR/**/*
when: manual
allow_failure: true
variables:
JOB_ENV: "dev"
script:
- echo "CI_COMMIT_BRANCH=$CI_COMMIT_BRANCH"
- echo "TARGET_BRANCH=$TARGET_BRANCH"
- echo "JOB_ENV=$JOB_ENV"
files1 job:
extends: .job_tpl
variables:
FILES_DIR: files1
files2 job:
extends: .job_tpl
variables:
FILES_DIR: files2
As you can see in the above code I'm using workflow to run only "branch pipelines" and have two "twin" jobs configured to watch for changes in one of the project's directories each. The TARGET_BRANCH variable is of course unnecessary in the demo project but i need something like this in the real one and it shows one of my problems. Additionally the jobs behave differently depending on the branch for which they are run.
My expectations
What I want to achieve is:
Each of the jobs should be added to a pipeline only when I push changes to files1/ or files2/ directory respectively.
When I push changes to a branch different then "main" a manual job responsible for the changed directory shoud be added to a pipeline.
When I merge changes to the "main" branch a job responsible for the changed directory shoud be added to a pipeline and it should be automatically started.
Test scenario
I'm creating a new branch from "main", make some change in the file1/test.txt and push the branch to GitLab.
what I expect: a pipeline created with only "files1 job" runnable manually
what I get: a pipeline with both jobs (both manual). Actually I've found explanation of such behaviour here: https://docs.gitlab.com/ee/ci/jobs/job_control.html#jobs-or-pipelines-run-unexpectedly-when-using-changes - "The changes rule always evaluates to true when pushing a new branch or a new tag to GitLab."
On the same branch I make another change in the file1/test.txt and make push.
what I expect: a pipeline created with only "files1 job" runnable manually
what I get: exactly what I expect since the branch isn't a "new" one
I create a Merge Request from my branch to main and make the merge.
what I expect: a pipeline created with only "files1 job" which starts automatically
what I get: a pipeline created with only "files1 job" but a manual one
My questions/problems
Can you suggest me any way to bypass the issue with "changes" evaluating always to "true" on new branches? Actually it behaves exactly as I want it if I don't use "rules" but let's assume I need "rules".
Why the jobs run as "manual" on the main branch in spite of the "if" condition in which both CI_COMMIT_BRANCH and TARGET_BRANCH variables are (or should be) set to "main". To debug it I'm printing those vars in job's "script" and when I run it on "main" pipeline I'm getting:
$ echo "CI_COMMIT_BRANCH=$CI_COMMIT_BRANCH"
CI_COMMIT_BRANCH=main
$ echo "TARGET_BRANCH=$TARGET_BRANCH"
TARGET_BRANCH=main
$ echo "JOB_ENV=$JOB_ENV"
JOB_ENV=dev
so theoretically CI should enter into the "automatic" job path.
Generally I find the CI "rules" quite inconvenient and confusing but as I understand it GitLab prefers them to "only/except" solution so I'm trying to refactor my CI/CD to use them which will fail if I don't find solution for the above difficulties :(
I am trying to include an external GitLab CI YAML file into my project local .gitlab-ci.yml file. The external YAML, which is in my other GitLab project ci/cd > templates contains some workflow rules:
# ci-cd.yml
workflow:
rules:
- if: '$TRACK != null' # TRACK is the environment type (staging/production)
when: always
- if: '$CI_PIPELINE_SOURCE =~ /^trigger|pipeline|web|api$/'
when: always
Below is my project local .gitlab-ci.yml:
include:
- '/.gitlab-ci-test.yml'
- project: 'ci-cd/templates'
file: 'ci-cd.yml'
...
The problem is - none of the jobs I have defined inside locally included .gitlab-ci-test.yml get triggered when I push the changes to GitLab, even when the job has when: always set. Seems like the workflow rules in external ci-cd.yml are not letting the jobs run.
I've also tried to locally add a workflow rule to .gitlab-ci.yml that evaluates to true, because the GitLab workflow keyword docs say
When no rules evaluate to true, the pipeline does not run.
That means if any one of the rules evaluates to true, pipeline should have run, which did not happen when I added a local workflow rule.
EDIT - the external file which has workflow rules is used by many projects so this can't be modified to have "push" in $CI_PIPELINE_SOURCE. My intention is to let it be as a global rule and try to 'override' locally in my project.
I hope I was clear in issue description. Would appreciate any help!
You are missing push event for $CI_PIPELINE_SOURCE in you workflow rule.
workflow:
rules:
- if: '$TRACK != null' # TRACK is the environment type (staging/production)
when: always
- if: '$CI_PIPELINE_SOURCE =~ /^trigger|pipeline|push|web|api$/'
when: always
EDIT: if you are not able to change the workflow rule in the included file, the only option I see is to duplicate the workflow in your gitlab-ci.yml and add the missed push there.
workflow rules take precedence before all other rules and it is not possible to merge two workflow blocks. If a worfklow block is used in an included yml file and the gitlab-ci.yml itself, the workflow from the gitlab-ci.yml is used. You can check this in CI/CD -> Editor -> View merged YAML in gitlab.
include:
- '/.gitlab-ci-test.yml'
- project: 'ci-cd/templates'
file: 'ci-cd.yml'
workflow:
rules:
- if: '$TRACK != null' # TRACK is the environment type (staging/production)
when: always
- if: '$CI_PIPELINE_SOURCE =~ /^trigger|pipeline|push|web|api$/'
when: always
...
As you see below I have two stages.
1- Build
2- Deploy
And I have 3 kind of branches
1- master
2- test
3- Other branches which created by developers. (Development)
I want to run build for all branches but I have a condition for deploy
1- If branch is test run deploy.
2- If branch is other branches don't run deploy.
3- if branch is master run deploy only manual.
But this three condition doesn't works properly with my gitlab-ci.yml
stages:
- build
- deploy
default:
image: node:latest
before_script:
- |
if [ "$CI_BUILD_REF_NAME" == "master" ]; then
ENV="prod"
elif [ "$CI_BUILD_REF_NAME" == "test" ]; then
ENV="test"
else
ENV="dev"
fi
- npm install
build:
stage: build
script:
- cp ./src/env-${ENV}.js ./src/env.js
- npm run build-compressed
deploy:
stage: deploy
rules: //this section doesn't work properly
- if: '$ENV == "test"'
when: always
- if: '$ENV == "prod"'
when: manual
- when: never
script:
- cp ./src/env-${ENV}.js ./src/env.js
- npm run build-compressed
- npm run publish:${ENV}
I recommend doing such a differentiation via Rules. As you can easily override variables with rules, see GitLab CI Doc.
Below you can find a short example
rules
- if: $CI_COMMIT_BRANCH =~ /test/
variables: # Override ENV defined
ENV: "test" # at the job level.
- if: $CI_COMMIT_BRANCH =~ /master/
variables: # Override ENV defined
ENV: "prod" # at the job level.
- variables: # not 100% sure about this one, as when can stand without a if clause, i assume variables can too :D
ENV: "dev"
Furthermore i would also do the rule check based on the branch within the deploy stage. - as it is easier to verify the branch, than to ensure at a later stage, that the Env variable is still set properly.
Also important to notice is the order of evaluation. Just because there is a before_script in the default section, does not mean it is evaluated just once for the whole. The before_script is evaluated once per Job.
Additionally the rules are evaluated, before the JOB runs, as it needs to determine if the job should be executed or not. This means you are not able to reference the variable from the before_script.
If you want to pass on the variable from the first job to the second job, you need to persist it in a dotenv-artifact - this way, the variables are available before the second stage start. (I am still not sure if you can use them for evaluation in rules, that is something you need to check)
I hope my additional clarifications, based on your comments, help you to find the best way.
I have an A project and an E2E project. I want to deploy A project trigger E2E pipeline run test but I just want the trigger test stage. we don't need trigger E2E to build deploy ...etc
e2e_tests:
stage: test
trigger:
project: project/E2E
branch: master
strategy: depend
stage: test
I have tried to use the stage in config. but got error unknown keys: stage
have any suggestions?
In your E2E project, the one that receives the trigger, you can tell a job to only run when the pipeline source is a trigger using the rules syntax:
build-from-trigger:
stage: build
when: never
rules:
- if: "$CI_COMMIT_REF_NAME == 'master' && $CI_PIPELINE_SOURCE == 'trigger'
when: always
script:
- ./build.sh # this is just an example, you'd run whatever you normally would here
The first when statement, when: never sets the default for the job. By default, this job will never run. Then using the rule syntax, we set a condition that will allow the job to run. If the CI_COMMIT_REF_NAME variable (the branch or tag name) is master AND the CI_PIPELINE_SOURCE variable (whatever kicked off this pipeline) is a trigger, then we run this job.
You can read about the when keyword here: https://docs.gitlab.com/ee/ci/yaml/#when, and you can read the rules documentation here: https://docs.gitlab.com/ee/ci/yaml/#rules