I'm trying to define a regex expression as a global variable in gitlab-ci.yml.
the variable is:
variables:
-SPEEDY_REGEX: '(chore|docs|feat|fix|refactor|style|test)\:( |\t)+\[SPEEDY\].*'
now I want to use it in if statements for getting specific builds to run:
build:
rules:
- if : '$CI_COMMIT_MESSAGE =~ $SPEEDY_REGEX'
when: never
- when: on_success
build:speedy:
rules:
- if : '$CI_COMMIT_MESSAGE =~ $SPEEDY_REGEX'
when: on_success
- when: never
This didn't want to work for a reason I ignore.
Unfortunately, because of the way regex is parsed in rules: you won't be able to use variables as part of your regex.
Instead, for a DRY approach, you can consider other methods for re-using YAML sections, such as anchors, extends or !reference tags.
.speedy_rules:
rules:
- if : '$CI_COMMIT_MESSAGE =~ /(chore|docs|feat|fix|refactor|style|test)\:( |\t)+\[SPEEDY\].*/'
when: never
- when: on_success
build:
rules: !reference [.speedy_rules, rules]
# ...
example:
rules:
- if: '$SOMETHING == "another rule"'
- !refernece [.speedy_rules, rules]
example2:
extends: .speedy_rules
# ...
Related
I want to be able to set a list of branches on the GitLab CICD variables on the UI and have the pipeline check that list to see if it should run a specific job.
I am trying to turn this rule:
rules:
- if: $CI_COMMIT_BRANCH == "poc5"
- if: $CI_COMMIT_BRANCH == "poc5-s1"
- if: $CI_COMMIT_BRANCH == "poc5-s2"
- if: $CI_COMMIT_BRANCH == "poc5-s3"
Into this:
rules:
- if: $CI_COMMIT_BRANCH in $DEFAULT_BRANCHES
Where $DEFAULT_BRANCHES is a variable defined in the projects CICD variables as follows:
DEFAULT_BRANCHES:
poc5
poc5-s1
poc5-s2
poc5-s3
or
poc5 poc5-s1 poc5-s2 poc5-s3
The best I could come up with was the following, which results in an invalid syntax for GitLab CI rules:
rules:
- if: echo $DEFAULT_BRANCHES | tr ' ' '\n' | grep -q -E "^${CI_COMMIT_BRANCH}$"
I am essentially trying to not have the list of branches hard coded into the pipeline yml file.
Is there a way to achieve what I am trying to do?
A solution is:
# .gitlab-ci.yml
if: "$CI_COMMIT_BRANCH == 'poc5' || $CI_COMMIT_BRANCH == 'poc5-s1' || $CI_COMMIT_BRANCH == 'poc5-s2' || $CI_COMMIT_BRANCH == 'poc5-s3'"
when: on_success
But a shorter version with regex is:
# .gitlab-ci
if: $CI_COMMIT_BRANCH =~ /^poc5(-s[1-3])?$/
when: on_success
I am currently trying to structure my .gitlab-ci.yml file like following:
There is one pipeline called "a" which should run only from the develop branch
There is one pipeline called "b" which should run only from the develop branch but only when triggered from the web
In order to avoid putting these rules in the jobs directly (in reality there are about 30 jobs) I want to reuse those rules globally. I could not find a better way other than using workflow:rules.
workflow:
rules:
- if $CI_COMMIT_BRANCH == "develop"
variables:
RUN_A: "true"
- if $CI_COMMIT_BRANCH == "develop" && $CI_PIPELINE_SOURCE == "web"
variables:
RUN_B: "true"
job-a:
rules:
- if: $RUN_A == "true"
script:
- exit 0
job-b:
rules:
- if: $RUN_B == "true"
script:
- exit 0
The problem is, that if there is a web trigger on develop, it will not set "RUN_B" to true. It will just have "RUN_A" with true. I added another job to print out the variables just to make sure:
test:
image: alpine
script:
- echo $RUN_A
- echo $RUN_B
This will only print true for RUN_A but nothing for RUN_B. I could not find anything in Gitlabs documentation that states it will only use the first matching rule. Anyhow, is there a better way to handle this? What am I missing?
Because GitLab ci Rules evaluated in order until the first match.
in your first rule if $CI_COMMIT_BRANCH == "develop" is match, the pipeline is create, in this times variable RUN_A: "true" is set, but variable RUN_B is undefined.
In your case, you can modify your .gitlab-ci.yml. variable setting and rules order.
workflow:
rules:
- if: $CI_COMMIT_BRANCH == "develop" && $CI_PIPELINE_SOURCE == "web"
variables:
RUN_B: "true"
RUN_A: "true"
- if: $CI_COMMIT_BRANCH == "develop"
variables:
RUN_A: "true"
default:
image: alpine
job-a:
rules:
- if: $RUN_A == "true"
script:
- exit 0
job-b:
rules:
- if: $RUN_B == "true"
script:
- exit 0
test:
script:
- echo $RUN_A
- echo $RUN_B
I currently have the below:
jobA:
only:
refs:
- merge_requests
- main
except:
refs:
- pipelines
The jobA above is triggered from an upstream pipeline using the trigger: keyword
How to represent the logic above with rules?
Is the above logically equivalent to the below?
rules:
- if: '$CI_PIPELINE_SOURCE == "pipelines"'
when: never
- if: ('$CI_PIPELINE_SOURCE == "merge_request_event"' || $CI_COMMIT_BRANCH == "mainline"')
when: on_success
I'd like to make the stage uninterruptible that runs the uninterruptible_job invisible (for master and release/v branches). Is there any function in gitlab ci to make that work ?
stages:
- uninterruptible
- pre_check
- build
- assign_test
- host_test
- deploy
workflow:
rules:
- if: '$CI_COMMIT_REF_NAME != "master" && $CI_COMMIT_BRANCH !~ /^release\/v/ && $CI_COMMIT_TAG !~ /^v\d+\.\d+(\.\d+)?($|-)/ && $CI_PIPELINE_SOURCE == "push"'
when: never
- if: $CI_OPEN_MERGE_REQUESTS != null
variables:
PIPELINE_COMMIT_SHA: $CI_MERGE_REQUEST_SOURCE_BRANCH_SHA
- if: $CI_OPEN_MERGE_REQUESTS == null
variables:
PIPELINE_COMMIT_SHA: $CI_COMMIT_SHA
- when: always
before_script:
- echo "Before script is started here"
default:
interruptible: true
retry:
max: 2
when:
- always
uninterruptible_job:
stage: uninterruptible
interruptible: false
resource_group: production
script:
- echo "$CI_COMMIT_BRANCH is uninterruptible"
tags:
- assign_test
rules:
- if: '$CI_COMMIT_REF_NAME == "master" || $CI_COMMIT_BRANCH =~ /^release\/v/'
when: always
include:
- '.gitlab/ci/pre_check.yml'
- '.gitlab/ci/build.yml'
- '.gitlab/ci/assign_test.yml'
- '.gitlab/ci/host_test.yml'
- '.gitlab/ci/deploy.yml'
No. There is no feature of GitLab to hide jobs visibly in the UI.
You can delete job traces and artifacts or delete entire pipelines (like after they finish running), but there is no way to just hide stages or jobs in GitLab's pipeline visualization UI.
I have a few jobs in my yaml files, and I was finding a way to only run some of the jobs when a particular scheduled is run (i.e. when a certain variable is set).
e.g.
** I have created a new schedule called 'Hourly Schedule', and created a variable called $HOURLY, which is set to TRUE.
HOURLY SCHEDULE
I also have another schedule called 'Daily Schedule'
DAILY SCHEDULE
stages:
- build
- test
- deploy
build-job:
stage: build
script:
- echo "Build."
test-job:
stage: test
script:
- echo "Test."
deploy-job:
stage: deploy
script:
- echo "Deploy."
hourly-deploy-job:
stage: deploy
script:
- echo "Hourly Deploy."
rules:
- if: $HOURLY
My question is:
How do I make 'build-job', 'test-job', and 'deploy-job' to run during 'commit', 'push', or 'Daily Schedule'
And during 'Hourly Schedule', only the 'hourly-deploy-job' should run (and not other jobs).
I know I can do the following:
stages:
- build
- test
- deploy
.hourly_deploy: &hourly_deploy
except:
variables:
- $HOURLY == "TRUE"
build-job:
<<: *hourly_deploy
stage: build
script:
- echo "Build."
test-job:
<<: *hourly_deploy
stage: test
script:
- echo "Test."
deploy-job:
<<: *hourly_deploy
stage: deploy
script:
- echo "Deploy."
hourly-deploy-job:
stage: deploy
script:
- echo "Hourly Deploy."
rules:
- if: $HOURLY
However, I ideally do a case only only on the 'hourly-deploy-job'. This is because my yaml might potentially be bigger at some point, and if I'm not careful, I might forget to add the <<: *hourly_deploy tag on the new job, which means that job will run during Hourly Schedule.
This is not an easy thing to achieve, and I think you should rethink your CICD strategy and use templates.
You mentioned this pipeline might grow and using templates might help you a lot.
Here is my example that works but it's not pretty.
stages:
- build
- test
- deploy
- hourly-deploy-job
build-job:
stage: build
rules:
- if: ($CI_PIPELINE_SOURCE == "push")
when: always
- if: ($CI_PIPELINE_SOURCE == "schedule") && ($DAILY == "true")
when: always
script:
- echo "Build."
test-job:
stage: test
rules:
- if: ($CI_PIPELINE_SOURCE == "push")
when: always
- if: ($CI_PIPELINE_SOURCE == "schedule") && ($DAILY == "true")
when: always
script:
- echo "Test."
deploy-job:
stage: deploy
rules:
- if: ($CI_PIPELINE_SOURCE == "push")
when: always
- if: ($CI_PIPELINE_SOURCE == "schedule") && ($DAILY == "true")
when: always
script:
- echo "Deploy."
hourly-deploy-job:
stage: hourly-deploy-job
rules:
- if: ($CI_PIPELINE_SOURCE == "schedule") && ($HOURLY == "true")
when: always
script:
- echo "Hourly Deploy."
Source:
https://docs.gitlab.com/ee/ci/yaml/#keywords
https://docs.gitlab.com/ee/ci/yaml/workflow.html
https://docs.gitlab.com/ee/ci/jobs/job_control.html
https://docs.gitlab.com/ee/ci/yaml/#rules