I need to exclude branches with the names feature/ussues- from some steps. There is a regular expression, but when I set except,rule or only, either the job always was included, or was not included at all.
Initially I tried this using rules::
rules:
- if: $CI_COMMIT_BRANCH =~ '^feature\.*\/.*
when: never
I expected this to match on branches like feature/issues-70, feature/issues-771, etc.
The regex rule you used is slightly off. First, you need surrounding / for the regex pattern. Second, the \. in the pattern will mean to match a literal . character, which is not what you want, based on the branch names you expect to match. Lastly, you need a matching rule for the job to be created at all (the default case when your rule doesn't match).
This should match all the example branch names you provided:
rules:
- if: $CI_COMMIT_TAG
when: never # same as except: - tags
- if: $CI_COMMIT_BRANCH == "test"
when: never # same as except: - test
- if: $CI_COMMIT_BRANCH =~ /^feature\/.*/
when: never
- when: on_success # the default when the first rule doesn't match
Related
I am currently trying to build a pipeline in gitlab ci. It contains a job (here called speed) that should only be run on a specific day (e.g. the 6th of every month). The configuration yaml looks like this:
stages:
- speed
- watcher
variables:
# setup cache in root folder so gitlab cache can pick it up
PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
TODAYS_DATE: "$(date +%Y-%m-%d)"
TODAYS_DAY: "$(date +%d)"
TODAYS_MONTH: "$(date +%m)"
include:
- local: "ci/ci_job_speed.yml"
- local: "ci/ci_job_watcher.yml"
speed:
extends: .speed
stage: speed
rules:
- if: '$TODAYS_DAY == "06"'
when: always
watcher:
when: always
extends: .watcher
stage: watcher
rules:
- if: '($CI_COMMIT_AUTHOR !~ /.*Gitlab Runner.*/ && $CI_COMMIT_BRANCH == "test") || $CI_PIPELINE_SOURCE != "push"'
cache:
paths:
- .cache/pip
- .cache/poetry
- .venv
key: "globalcache"
My problem is, that the speed job seems to be ignored, and only the watcher job is started, even though I expect speed to be run (since today is the 6th). The ci linter in GitLab validates my yaml as correct. Does anyboy have an idea what I am doing wrong?
I have tried different combinations as '$TODAYS_DAY == '06'', $TODAYS_DAY == '06' etc, none of them worked yet (the job is simply ignored).
Thanks in advance for any advice :)
Looking at your yaml i would suggest, that TODAYS_DAY: "$(date +%d)" is not executed as intended.
If you assigned this Variable in a bash script, the command date +%d would be executed and the result would be stored in the variable, but i think in your case $(date +%d) just gets stored as literal string.
I could be wrong about this variable, but this is my best guess.
You could consider using GitLabs scheduled Pipelines Feature: https://docs.gitlab.com/ee/ci/pipelines/schedules.html#add-a-pipeline-schedule
#joreign you are correct, the variable TODAYS_DATE was not assigned the day of the current date, but a string value "$(date +%d)". After trying a bit, I could not find a way to enforce the desired behaviour.
Instead, I used a workaround as described in https://gitlab.com/gitlab-org/gitlab/-/issues/20769#note_215109166 and set a random variable in a Pipeline scheduler and gave the ci job a corresponding run condition.
speed:
extends: .speed
stage: speed
only:
refs:
- schedules
variables:
- $REFSPEED_SCHEDULE
I see this kind of syntax for rules:if and rules:when in GitLab CI/CD pipeline a lot:
job:
script: echo "This job does NOT create double pipelines!"
rules:
- if: $CI_PIPELINE_SOURCE == "push"
when: never
- when: always
Is that also equivalent to this?
job:
script: echo "This job does NOT create double pipelines!"
rules:
- if: $CI_PIPELINE_SOURCE != "push"
- when: always
rules mean that job is evaluated in order until the first match.
first rules
rules:
- if: $CI_PIPELINE_SOURCE == "push"
when: never
- when: always
are equivalent second rules
rules:
- if: $CI_PIPELINE_SOURCE != "push"
but, I recommend using the first rules, because it is easy to know and easy to read.
I have a test for their two rules. first, I set the IS_JOB_ON variable to off, that pipeline only test job 3 create, and second set IS_JOB_ON to on, three jobs are created.
first .gitlab-ci.yml and Pipeline
second .gitlab-ci.yml and Pipeline
sample:
variables:
IS_JOB_ON: 'off'
test job 1:
script:
- echo "run job 1"
rules:
- if: $IS_JOB_ON == "off"
when: never
- when: always
test job 2:
script:
- echo "run job 2"
rules:
- if: $IS_JOB_ON != "off"
test job 3:
script:
- echo "run job 3 always run"
I am trying to pass the variable to parallel/matrix and do not see that getting expanded and the job failing. This is being set in the job from the environment variable. I am trying to echo the variable in script and see it shows the right value, but does not get substituted in parallel/matrix. Am I missing anything?
.common_deploy:
script:
- |
echo "showing the regions from environment"
echo $qa_regions
echo "showing the regions from job variable"
echo $REGIONS
parallel:
matrix:
- REGION: "${REGIONS}"
DeployToQA:
variables:
ENVIRONMENT: qa
REGIONS: $qa_regions
extends:
- .common_deploy
stage: deploy
rules:
- if: '$CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == "master"'
allow_failure: true
Here the variable $qa_regions has the value of "us-west-2,us-east-1", I was expecting to see the 2 jobs for those regions , but i am seeing the job as DeployToQA: [${REGIONS}]
Variable expansion for the parallel keyword is currently not supported. There is an open issue for this request.
You can take a look at the documentation where variables can be used.
I am trying to convert a .gitlab-ci.yml job from the deprecated only/except pattern to rules, specifically I want to run a job only if it's for a tag but not if that tag starts with config-.
Currently we use this, which at least works1.
build_image:
stage: build_image
only:
- tags
except:
- /^config-.*$/
I have tried the rules based approach as follows, but that runs for every tag, including config:
build_image:
stage: build_image
rules:
# only tags, except config
- if: $CI_COMMIT_TAG
when: on_success
- if: '$CI_COMMIT_TAG =~ /^config-.*$/'
when: never
- when: never
Additionally I tried to use negative lookahead regex - without success, since gitlab's ci linter screams jobs:build_image:rules:rule if invalid expression syntax.
build_image:
stage: build_image
rules:
# only tags, except config (using negative lookahead)
- if: '$CI_COMMIT_TAG =~ /^(?!config-).*$/'
when: on_success
- when: never
So how can I trigger a job only for tags excluding a pattern with the new rules based approach?
1: apart from not being able to be combined with other rules later on
You just have to combine the unlike operator !~ with the regular tag check $CI_COMMIT_TAG:
build_image:
stage: build_image
rules:
# only tags and only those not starting with config-
- if: '$CI_COMMIT_TAG && ($CI_COMMIT_TAG !~ /^config-.*$/ )'
when: on_success
- when: never
This rules worked for me (with Gitlab 13.12) :
rules:
- if: '$CI_COMMIT_TAG && $CI_COMMIT_TAG !~ /^config-.*$/'
I would like to create rule for:
changes in folder foo && branch = master && tag pushed
My current rules not working:
rules:
- if: $CI_COMMIT_TAG && '$CI_COMMIT_BRANCH == "master"'
- changes:
- foo/**/*
Looks like gitlab take like a OR.
What is wrong?
As stated in the gitlab documentation:
To conjoin if, changes, and exists clauses with an AND, use them in the same rule.
So it should be:
rules:
- if: $CI_COMMIT_TAG && '$CI_COMMIT_BRANCH == "master"'
changes:
- foo/**/*