How to check a null variable in Gitlab pipeline when it's declared with a content of another null variable? Like the variable VAR_NULL below when NO_VAR is null:
variables:
VAR_EMPTY: ""
VAR_NULL: "${NO_VAR}"
Check the pipeline result out where only VAR_EMPTY == "" and NO_VAR == null evaluate to true all others are false.
Pipeline result (a screenshot for convenience, the full result: https://gitlab.com/labaz/test-gitlab-pipeline-null-var/-/pipelines/493036820):
Full pipeline script (https://gitlab.com/labaz/test-gitlab-pipeline-null-var/-/blob/main/.gitlab-ci.yml):
variables:
VAR_EMPTY: ""
VAR_NULL: "${NO_VAR}"
jobTest-Var_Empty-IsNull: # This job runs in the build stage, which runs first.
rules:
- if: '$VAR_EMPTY == null'
script:
- 'echo "VAR_EMPTY IS null"'
jobTest-Var_Empty-IsEmpty: # This job runs in the build stage, which runs first.
rules:
- if: '$VAR_EMPTY == ""'
script:
- 'echo "VAR_EMPTY IS \"\""'
jobTest-Var_Null-IsNull: # This job runs in the build stage, which runs first.
rules:
- if: '$VAR_NULL == null'
script:
- 'echo "VAR_NULL IS null"'
jobTest-Var_Null-IsEmpty: # This job runs in the build stage, which runs first.
rules:
- if: '$VAR_NULL == ""'
script:
- 'echo "VAR_NULL IS Empty"'
jobTest-No_Var-IsNull: # This job runs in the build stage, which runs first.
rules:
- if: '$NO_VAR == null'
script:
- 'echo "NO_VAR IS null"'
jobTest_No_Var-IsEmpty: # This job runs in the build stage, which runs first.
rules:
- if: '$NO_VAR == ""'
script:
- 'echo "NO_VAR IS Empty"'
The problem you're encountering is that VAR_NULL: "${NO_VAR}" is not a null variable. It's actually the same thing as VAR_EMPTY: "" -- you are declaring the variable (therefore it is not null) with an empty value.
The only way to test if the variable was created with another empty variable is to test the original variable itself. That is, test NO_VAR, not VAR_NULL.
An alternative strategy would be to use rules:variables: in order to conditionally declare VAR_NULL
workflow:
rules:
- if: '$NO_VAR' # or '$NO_VAR != null' depending on what you want
variables:
VAR_NULL: "$NO_VAR"
- when: always
jobTest-Var_Null-IsNull:
rules:
- if: '$VAR_NULL == null'
script:
- echo "VAR_NULL is null"
As soon as any "if" is matched the job runs.
there's no overriding it with a later "never"
also 'if when never' - who thought that was a good idea?
Anyhow to make the if less matchy ... you can use parentheses.
rules:
- if: ($CI_COMMIT_BRANCH && $SOME_VAR == null)
However this doesn't work:
- if: ('$CI_COMMIT_BRANCH == develop' && $SOME_VAR == null)
not sure why considering this is valid...
- if: '$CI_COMMIT_BRANCH == develop'
essentially I want to skip some jobs when 'some_var' is set (which happens when I run a manual pipeline)
So putting it all together ...
And before your eyes glaze over "workflow" and its "rules" are a separate beast from "job_name:" "rules:"
ostensibly workflow needs "always or never" and the rules can control when jobs run.. but only if you put a "never" .. unless i'm missing something... )
workflow:
rules:
- if: '$SOMEVAR == null'
variables:
BLOCK_JOB: "False"
- when: always
#then in my job we can do something like:
rules:
- if: ($CI_COMMIT_BRANCH == "develop" && $BLOCK_JOB)
yes all of that because this doesn't work
- if: ('$CI_COMMIT_BRANCH == develop' && $SOME_VAR == null)
so now I can have a job that runs on develop unless its run manually (which is when I set SOMEVAR)
and then its skipped.
Workflow also seems to just randomly refuse to do somethings...
for example this Doesn't work ( trying to set a var based on project name) never evals as true so CI_PATH_TYPE remains unset... but imo looking at it with my wetware... it should...
rules:
- if: $CI_PROJECT_NAME == "some-project-name"
variables:
CI_PATH_TYPE: project
But this does? because.. null is magic..?
rules:
- if: '$somevar == null'
variables:
BLOCK_JOB: "False"
also any time something is quoted in various ways in these examples gitlab ci made me do it exactly like that, any other variations and it shits the bed.
so once again the latest tool is just more black magic, hot take: gitlab-ci is bad.
Related
I want to be able to set a list of branches on the GitLab CICD variables on the UI and have the pipeline check that list to see if it should run a specific job.
I am trying to turn this rule:
rules:
- if: $CI_COMMIT_BRANCH == "poc5"
- if: $CI_COMMIT_BRANCH == "poc5-s1"
- if: $CI_COMMIT_BRANCH == "poc5-s2"
- if: $CI_COMMIT_BRANCH == "poc5-s3"
Into this:
rules:
- if: $CI_COMMIT_BRANCH in $DEFAULT_BRANCHES
Where $DEFAULT_BRANCHES is a variable defined in the projects CICD variables as follows:
DEFAULT_BRANCHES:
poc5
poc5-s1
poc5-s2
poc5-s3
or
poc5 poc5-s1 poc5-s2 poc5-s3
The best I could come up with was the following, which results in an invalid syntax for GitLab CI rules:
rules:
- if: echo $DEFAULT_BRANCHES | tr ' ' '\n' | grep -q -E "^${CI_COMMIT_BRANCH}$"
I am essentially trying to not have the list of branches hard coded into the pipeline yml file.
Is there a way to achieve what I am trying to do?
A solution is:
# .gitlab-ci.yml
if: "$CI_COMMIT_BRANCH == 'poc5' || $CI_COMMIT_BRANCH == 'poc5-s1' || $CI_COMMIT_BRANCH == 'poc5-s2' || $CI_COMMIT_BRANCH == 'poc5-s3'"
when: on_success
But a shorter version with regex is:
# .gitlab-ci
if: $CI_COMMIT_BRANCH =~ /^poc5(-s[1-3])?$/
when: on_success
I am currently trying to structure my .gitlab-ci.yml file like following:
There is one pipeline called "a" which should run only from the develop branch
There is one pipeline called "b" which should run only from the develop branch but only when triggered from the web
In order to avoid putting these rules in the jobs directly (in reality there are about 30 jobs) I want to reuse those rules globally. I could not find a better way other than using workflow:rules.
workflow:
rules:
- if $CI_COMMIT_BRANCH == "develop"
variables:
RUN_A: "true"
- if $CI_COMMIT_BRANCH == "develop" && $CI_PIPELINE_SOURCE == "web"
variables:
RUN_B: "true"
job-a:
rules:
- if: $RUN_A == "true"
script:
- exit 0
job-b:
rules:
- if: $RUN_B == "true"
script:
- exit 0
The problem is, that if there is a web trigger on develop, it will not set "RUN_B" to true. It will just have "RUN_A" with true. I added another job to print out the variables just to make sure:
test:
image: alpine
script:
- echo $RUN_A
- echo $RUN_B
This will only print true for RUN_A but nothing for RUN_B. I could not find anything in Gitlabs documentation that states it will only use the first matching rule. Anyhow, is there a better way to handle this? What am I missing?
Because GitLab ci Rules evaluated in order until the first match.
in your first rule if $CI_COMMIT_BRANCH == "develop" is match, the pipeline is create, in this times variable RUN_A: "true" is set, but variable RUN_B is undefined.
In your case, you can modify your .gitlab-ci.yml. variable setting and rules order.
workflow:
rules:
- if: $CI_COMMIT_BRANCH == "develop" && $CI_PIPELINE_SOURCE == "web"
variables:
RUN_B: "true"
RUN_A: "true"
- if: $CI_COMMIT_BRANCH == "develop"
variables:
RUN_A: "true"
default:
image: alpine
job-a:
rules:
- if: $RUN_A == "true"
script:
- exit 0
job-b:
rules:
- if: $RUN_B == "true"
script:
- exit 0
test:
script:
- echo $RUN_A
- echo $RUN_B
I have two jobs.
I kept conditions in such a way to not continue if the previous job fails.
But it seems, it is working a differently, please find the details below.
First one build-rpm, as below.
build-rpm:
stage: build_preconfig
image: ubuntu:latest
rules:
#- if: '$PACKAGE_URL && $PACKAGE_URL != ""'
# when: never
- if: "$ACTION && $BUILDRPMREQUIRED" #should run for default when
- when: never #should skip as default condition
And the next job.
.ifawsdeploy:
# just checking for ACTION variable, not value, as this is for deploy stage and can be new/existing cluster
# Removed $PACKAGE_URL as deploy should work in existing cluster like refresh even without passing package url
rules:
- if: "$PREVIOUS_JOB_ID"
when: never
- if: '$ACTION && $REGION && $ROLE_ARN && $CLOUD_PLATFORM == "aws" && $ROLE_ARN != "" && $REGION != ""'
#when we want to deploy a new admin server and cluster
.ifawsfulldeploy:
rules:
- if: "$BUILDRPMREQUIRED == null && $PACKAGE_URL == null"
when: never
- if: '$ADMIN_SERVER_IP && $ADMIN_SERVER_IP != ""'
when: never
- if: '$ACTION && $REGION && $ROLE_ARN && $ACTION == "new" && $CLOUD_PLATFORM == "aws" && $ROLE_ARN != "" && $REGION != ""'
when: always
create-admin-server:
<<: *yum-cache
stage: build
allow_failure: false
image:
name: amazon/aws-cli:latest
entrypoint: [""]
extends:
- .ifawsfulldeploy
This is running even if the buildrpm failed
And the next is not running even the before job is successful.
deployment:
stage: deploy
allow_failure: true
extends:
- .ifawsdeploy
So, I removed always part from the .ifawsdeploy then it is skipping the other part.
But, I wasn't able to run it later that particular job to resume pipeline after fixes.
commit-message-validator:
stage: validate-commit-message
script:
- echo "$CI_COMMIT_MESSAGE"
- echo "$CI_COMMIT_BRANCH"
- echo "check the Prefix of the commit message should have one of 'fix' || 'feat' || 'major' || 'minor' in case sensitive"
- exit 1
rules:
- if: $CI_COMMIT_MESSAGE =~ /fix:/
when: never
- if: $CI_COMMIT_MESSAGE =~ /feat:/
when: never
- if: $CI_COMMIT_MESSAGE =~ /major:/
when: never
- if: $CI_COMMIT_MESSAGE =~ /minor:/
when: never
- if: "$CI_COMMIT_MESSAGE =~ /^chore\(release\):.*/"
when: never
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
when: never
- when: always
enter image description here
The last step in your script block is exit 1. Gitlab CI will fail any job that has an exit code > 0.
The commit-message-validator job doesn't currently do any checks, it simply calls echo 3 times. If you want it to check for fix, feat, major, minor you will need to do write some logic in the script block to do so.
I want to run a gitlab job in every case EXCEPT the variable RUN is defined and contains the value run3.
So again:
variable RUN is undefined: run the job
variable RUN is defined and does not contain the value run3: run the job
variable RUN is defined and does contain the value run3: DO NOT run the job
I tried the following rules:
rules:
- if: $RUN != null && $RUN == "run3"
when: never
rules:
- if: $RUN != "" && $RUN == "run3"
when: never
rules:
- if: $RUN == "run3"
when: never
Without the variable RUN being defined, no job was run. I expected the job to have been run.
How to fix it? Where is the problem in the logic?
After some time-consuming try-and-error trials I found the solution: As you cannot test if a variable exists, you have to create the rule the other way around. It means, you run the job if the value does not match:
The correct rule is then:
- if: $RUN != "run3"
when: always
- when: never
The first part only evaluates to true if the variable exists AND if the variable does not evaluate to run3. In that case the job is run.
The second part then tells to NOT run the job in any other case.
You're close
rules:
- if: $RUN == "run3"
when: never
This is correct, but you also need a default - when:
To emulate the default behavior of no rules when the first rule does not match, set the default as on_success
rules:
- if: $RUN == "run3"
when: never # don't run the job in this case
- when: on_success # run per usual in all other cases
Alex, maybe you can try:
if: $RUN != undefined && $RUN == "run3"
when: never