I would like to achieve this Gitlab CI/CD behaviour:
Two jobs defined in the pipeline:
job_beta triggered when a tag is created with a semver number containing a "-rc*" suffix. For example: v1.0.0-rc1.
job_production triggered when a tag is created with a semver number, without any suffix. For example: v1.0.0.
Is there a way to parse the $CI_COMMIT_TAG variable in the .gitlab-ci.yml file of the project? Or any other ideas about how to achieve this?
To achieve this kind of behaviour, you can rely on the only keyword to control which job will be created. Simply specify a regular expression that will either match a semver with the rc suffix or one without any suffix.
You could do it the following way if your semantic versions are prefixed with v:
build-beta:
image: your-build-image:latest
stage: build
only:
- /^v[0-9]+\.[0-9]+\.[0-9]+-rc[0-9]+$/
script:
- ./build-beta.sh # Do something...
build-prod:
image: your-build-image:latest
stage: build
only:
- /^v[0-9]+\.[0-9]+\.[0-9]+$/
script:
- ./build-prod.sh # Do something...
You can also achieve something similar with the rules keyword by matching $CI_COMMIT_REF_NAME with the appropriate regex in an if condition.
Related
I'm wondering if using extends: in global variables: is supposed to work.
The documentation here states, that:
Keyword type: Job keyword. You can use it ONLY as part of a job.
(my highlighting).
However, if I use something like
stages:
- test
.common_variables:
__FOO: "foo"
variables:
extends: .common_variables
__BAR: "bar"
test:
stage: test
script:
- env
rules:
- when: always
then I find both __FOO as well as __BAR variables defined on the runner. However, I also find the extends variable (set to .common_variables defined on the runner).
So I wonder if this is the expected behaviour on which one can rely and the presence of extends variable is just a minor side effect or just a bug in the GitLab version that we use and this keyword is supported only inside jobs as the docs state?
P.S. note, that i specifically ask about extends because i plan to off-load the shared variables into a separate file in a separate repo to be able to reuse it in pipelines of multiple repositories
Yes it works like that.
But you can also read in addition how it can be used correctly if you want to run several scripts in one YAML file.
You can read more details here
I have a Pipeline that has a few stages: detect, test, build, deploy
The detect stage detects the type of application and the test and build stages have jobs that are included or excluded based on what is computed in detect. The detect stage writes it's value to a environment variable called BUILD_MODE.
I am using rules like so:
ng-build:
extends:
- '.ng/job/build'
stage: build
rules:
- if: $BUILD_MODE == "ANGULAR"
when: always
npm-build:
extends:
- '.npm/job/build'
stage: build
rules:
- if: $BUILD_MODE == "NPM"
when: always
The problem with this is that the BUILD_MODE variable is evaluated statically when the pipeline is created not after the detect stage runs so the above never works unless I set the variable explicitly in the top level YML file like so:
variables:
BUILD_MODE: "ANGULAR"
What is the best way to solve this problem? The summary of what I want to do is evaluate some condition, either set the stages dynamically or set the variable itself before the stages in the Pipleline are created so they will be created with the rules evaluated correctly.
You could take a look at dynamic child-pipelines. Maybe you could solve your problem by dynamically creating your npm/ng build jobs.
I am using Gitlab and added a .gitlab-ci.yml file to trigger my pipeline. What I don't understand is the purpose of refs when used in combination with the only keyword. Here is an example I use:
only:
- /^newlib.*$/
But in another example I found this:
only:
refs:
- master
The documentation only states:
The refs strategy can take the same values as the simplified only/except configuration.
Can anyone share some light on this? What is the difference between the two?
There is no difference between your two examples. As you quoted the docs yourself:
The refs strategy can take the same values as the simplified only/except configuration.
That said, you only need to use refs if you also want to use the other options: changes, kubernetes and variables.
The docs has some examples using multiple options such as:
test:
script: npm run test
only:
refs:
- master
- schedules
variables:
- $CI_COMMIT_MESSAGE =~ /run-end-to-end-tests/
kubernetes: active
If you remove the refs you will get an invalid yml file
While there are many questions and answers around conditional job execution of GitLab CI pipelines I could not find a solution for my problem: To trigger a particular job from pipeline when there are changes in some files OR a env variable is set to a specific value. Something like this:
job_build:
tags:
- executor-shell
stage:
build
script:
- ./gitlab_ci_helper.sh build
only:
changes:
- /path/to/files/with/changes/*
variables:
- $BUILD_IS_A_MUST == "TRUE"
As per the documentation from GitLab itself, we have the following:
Now the question I have is how to make the condition NOT(any of variables) AND (any of changes)) which is same as (any of variables) OR (any of changes) but applied for the only parameter?
Well, in the end, looks like found myself the answer to the question. It turns out that starting from GitLab version 12.3 a new configuration parameter was introduced called rules:
Looks like this parameter is exactly addressing the problem I was looking for an answer. While I don't have yet GitLab 12.3 version to test it, the above ci job would be changed into something like this:
job_build:
tags:
- executor-shell
stage:
build
script:
- ./gitlab_ci_helper.sh build
rules:
- changes:
- /path/to/files/with/changes/*
when: on_success
- if: $BUILD_IS_A_MUST == "TRUE"
when: on_success
In this case, the default rule will get resolved to when: never which is what I'm after thus no need to specify it.
We have a job (deploy to production) that we generally manually click after checking that build on staging. However, very occasionally we have an issue that we've accidentally deployed and want to get a fix out ASAP. In those case we run the tests locally (much faster) and put [urgent-fix] in our commit message to stop the tests running in CI (skipping straight to Docker image build and staging deploy).
What we'd like to do is if we put [urgent-fix] it automatically triggers the production deploy (usually a when: manual step). Can we achieve this somehow?
Sounds like you can use a combination of the only:variables syntax and $CI_COMMIT_MESSAGE predefined variable.
A rough idea (untested):
.deploy_production: &deploy_production
stage: deploy production
script:
- echo "I'm deploy production here"
tags:
- some special tag
deploy::manual:
<< *deploy_production
when: manual
allow_failure: false
deploy:urgent_fix:
<< *deploy_production
only:
variables:
- $CI_COMMIT_MESSAGE =~/[urgent-fix]/
As of GitLab v12.3 (~September 2019) GitLab comes with "Flexible Rules for CI Build config". The feature is intended to replace the only/except functionality and is fully documented here.
With rules: you can now fully influence the when: behaviour of your job based on various conditions (in contrast to only/except: which forced you to create separate jobs for situations like the one described in the OP; see accepted answer).
For example you can do:
deploy:
rules:
- if: '$CI_COMMIT_TITLE =~ /urgent-fix/'
when: on_success
- when: manual # default fallback
script:
- sh deploy.sh
One thing to highlight is that in the above example I used $CI_COMMIT_TITLE instead of $CI_COMMIT_MESSAGE (see gitlab docs) to avoid the string "urgent-fix" reoccuring in a commit message automatically assembled in the course of a git merge, which would then accidentally retrigger the job.
Disclaimer: Please be aware that the new rules: feature stands in conflict with only/except: and thus requires you to remove only/except: occurences. Also, please note that rules: should only be used in combination with workflow: (read the docs) to avoid unwanted "detached" pipelines being triggered.