I have two jobs.
I kept conditions in such a way to not continue if the previous job fails.
But it seems, it is working a differently, please find the details below.
First one build-rpm, as below.
build-rpm:
stage: build_preconfig
image: ubuntu:latest
rules:
#- if: '$PACKAGE_URL && $PACKAGE_URL != ""'
# when: never
- if: "$ACTION && $BUILDRPMREQUIRED" #should run for default when
- when: never #should skip as default condition
And the next job.
.ifawsdeploy:
# just checking for ACTION variable, not value, as this is for deploy stage and can be new/existing cluster
# Removed $PACKAGE_URL as deploy should work in existing cluster like refresh even without passing package url
rules:
- if: "$PREVIOUS_JOB_ID"
when: never
- if: '$ACTION && $REGION && $ROLE_ARN && $CLOUD_PLATFORM == "aws" && $ROLE_ARN != "" && $REGION != ""'
#when we want to deploy a new admin server and cluster
.ifawsfulldeploy:
rules:
- if: "$BUILDRPMREQUIRED == null && $PACKAGE_URL == null"
when: never
- if: '$ADMIN_SERVER_IP && $ADMIN_SERVER_IP != ""'
when: never
- if: '$ACTION && $REGION && $ROLE_ARN && $ACTION == "new" && $CLOUD_PLATFORM == "aws" && $ROLE_ARN != "" && $REGION != ""'
when: always
create-admin-server:
<<: *yum-cache
stage: build
allow_failure: false
image:
name: amazon/aws-cli:latest
entrypoint: [""]
extends:
- .ifawsfulldeploy
This is running even if the buildrpm failed
And the next is not running even the before job is successful.
deployment:
stage: deploy
allow_failure: true
extends:
- .ifawsdeploy
So, I removed always part from the .ifawsdeploy then it is skipping the other part.
But, I wasn't able to run it later that particular job to resume pipeline after fixes.
Related
I am currently trying to structure my .gitlab-ci.yml file like following:
There is one pipeline called "a" which should run only from the develop branch
There is one pipeline called "b" which should run only from the develop branch but only when triggered from the web
In order to avoid putting these rules in the jobs directly (in reality there are about 30 jobs) I want to reuse those rules globally. I could not find a better way other than using workflow:rules.
workflow:
rules:
- if $CI_COMMIT_BRANCH == "develop"
variables:
RUN_A: "true"
- if $CI_COMMIT_BRANCH == "develop" && $CI_PIPELINE_SOURCE == "web"
variables:
RUN_B: "true"
job-a:
rules:
- if: $RUN_A == "true"
script:
- exit 0
job-b:
rules:
- if: $RUN_B == "true"
script:
- exit 0
The problem is, that if there is a web trigger on develop, it will not set "RUN_B" to true. It will just have "RUN_A" with true. I added another job to print out the variables just to make sure:
test:
image: alpine
script:
- echo $RUN_A
- echo $RUN_B
This will only print true for RUN_A but nothing for RUN_B. I could not find anything in Gitlabs documentation that states it will only use the first matching rule. Anyhow, is there a better way to handle this? What am I missing?
Because GitLab ci Rules evaluated in order until the first match.
in your first rule if $CI_COMMIT_BRANCH == "develop" is match, the pipeline is create, in this times variable RUN_A: "true" is set, but variable RUN_B is undefined.
In your case, you can modify your .gitlab-ci.yml. variable setting and rules order.
workflow:
rules:
- if: $CI_COMMIT_BRANCH == "develop" && $CI_PIPELINE_SOURCE == "web"
variables:
RUN_B: "true"
RUN_A: "true"
- if: $CI_COMMIT_BRANCH == "develop"
variables:
RUN_A: "true"
default:
image: alpine
job-a:
rules:
- if: $RUN_A == "true"
script:
- exit 0
job-b:
rules:
- if: $RUN_B == "true"
script:
- exit 0
test:
script:
- echo $RUN_A
- echo $RUN_B
I'd like to make the stage uninterruptible that runs the uninterruptible_job invisible (for master and release/v branches). Is there any function in gitlab ci to make that work ?
stages:
- uninterruptible
- pre_check
- build
- assign_test
- host_test
- deploy
workflow:
rules:
- if: '$CI_COMMIT_REF_NAME != "master" && $CI_COMMIT_BRANCH !~ /^release\/v/ && $CI_COMMIT_TAG !~ /^v\d+\.\d+(\.\d+)?($|-)/ && $CI_PIPELINE_SOURCE == "push"'
when: never
- if: $CI_OPEN_MERGE_REQUESTS != null
variables:
PIPELINE_COMMIT_SHA: $CI_MERGE_REQUEST_SOURCE_BRANCH_SHA
- if: $CI_OPEN_MERGE_REQUESTS == null
variables:
PIPELINE_COMMIT_SHA: $CI_COMMIT_SHA
- when: always
before_script:
- echo "Before script is started here"
default:
interruptible: true
retry:
max: 2
when:
- always
uninterruptible_job:
stage: uninterruptible
interruptible: false
resource_group: production
script:
- echo "$CI_COMMIT_BRANCH is uninterruptible"
tags:
- assign_test
rules:
- if: '$CI_COMMIT_REF_NAME == "master" || $CI_COMMIT_BRANCH =~ /^release\/v/'
when: always
include:
- '.gitlab/ci/pre_check.yml'
- '.gitlab/ci/build.yml'
- '.gitlab/ci/assign_test.yml'
- '.gitlab/ci/host_test.yml'
- '.gitlab/ci/deploy.yml'
No. There is no feature of GitLab to hide jobs visibly in the UI.
You can delete job traces and artifacts or delete entire pipelines (like after they finish running), but there is no way to just hide stages or jobs in GitLab's pipeline visualization UI.
How to check a null variable in Gitlab pipeline when it's declared with a content of another null variable? Like the variable VAR_NULL below when NO_VAR is null:
variables:
VAR_EMPTY: ""
VAR_NULL: "${NO_VAR}"
Check the pipeline result out where only VAR_EMPTY == "" and NO_VAR == null evaluate to true all others are false.
Pipeline result (a screenshot for convenience, the full result: https://gitlab.com/labaz/test-gitlab-pipeline-null-var/-/pipelines/493036820):
Full pipeline script (https://gitlab.com/labaz/test-gitlab-pipeline-null-var/-/blob/main/.gitlab-ci.yml):
variables:
VAR_EMPTY: ""
VAR_NULL: "${NO_VAR}"
jobTest-Var_Empty-IsNull: # This job runs in the build stage, which runs first.
rules:
- if: '$VAR_EMPTY == null'
script:
- 'echo "VAR_EMPTY IS null"'
jobTest-Var_Empty-IsEmpty: # This job runs in the build stage, which runs first.
rules:
- if: '$VAR_EMPTY == ""'
script:
- 'echo "VAR_EMPTY IS \"\""'
jobTest-Var_Null-IsNull: # This job runs in the build stage, which runs first.
rules:
- if: '$VAR_NULL == null'
script:
- 'echo "VAR_NULL IS null"'
jobTest-Var_Null-IsEmpty: # This job runs in the build stage, which runs first.
rules:
- if: '$VAR_NULL == ""'
script:
- 'echo "VAR_NULL IS Empty"'
jobTest-No_Var-IsNull: # This job runs in the build stage, which runs first.
rules:
- if: '$NO_VAR == null'
script:
- 'echo "NO_VAR IS null"'
jobTest_No_Var-IsEmpty: # This job runs in the build stage, which runs first.
rules:
- if: '$NO_VAR == ""'
script:
- 'echo "NO_VAR IS Empty"'
The problem you're encountering is that VAR_NULL: "${NO_VAR}" is not a null variable. It's actually the same thing as VAR_EMPTY: "" -- you are declaring the variable (therefore it is not null) with an empty value.
The only way to test if the variable was created with another empty variable is to test the original variable itself. That is, test NO_VAR, not VAR_NULL.
An alternative strategy would be to use rules:variables: in order to conditionally declare VAR_NULL
workflow:
rules:
- if: '$NO_VAR' # or '$NO_VAR != null' depending on what you want
variables:
VAR_NULL: "$NO_VAR"
- when: always
jobTest-Var_Null-IsNull:
rules:
- if: '$VAR_NULL == null'
script:
- echo "VAR_NULL is null"
As soon as any "if" is matched the job runs.
there's no overriding it with a later "never"
also 'if when never' - who thought that was a good idea?
Anyhow to make the if less matchy ... you can use parentheses.
rules:
- if: ($CI_COMMIT_BRANCH && $SOME_VAR == null)
However this doesn't work:
- if: ('$CI_COMMIT_BRANCH == develop' && $SOME_VAR == null)
not sure why considering this is valid...
- if: '$CI_COMMIT_BRANCH == develop'
essentially I want to skip some jobs when 'some_var' is set (which happens when I run a manual pipeline)
So putting it all together ...
And before your eyes glaze over "workflow" and its "rules" are a separate beast from "job_name:" "rules:"
ostensibly workflow needs "always or never" and the rules can control when jobs run.. but only if you put a "never" .. unless i'm missing something... )
workflow:
rules:
- if: '$SOMEVAR == null'
variables:
BLOCK_JOB: "False"
- when: always
#then in my job we can do something like:
rules:
- if: ($CI_COMMIT_BRANCH == "develop" && $BLOCK_JOB)
yes all of that because this doesn't work
- if: ('$CI_COMMIT_BRANCH == develop' && $SOME_VAR == null)
so now I can have a job that runs on develop unless its run manually (which is when I set SOMEVAR)
and then its skipped.
Workflow also seems to just randomly refuse to do somethings...
for example this Doesn't work ( trying to set a var based on project name) never evals as true so CI_PATH_TYPE remains unset... but imo looking at it with my wetware... it should...
rules:
- if: $CI_PROJECT_NAME == "some-project-name"
variables:
CI_PATH_TYPE: project
But this does? because.. null is magic..?
rules:
- if: '$somevar == null'
variables:
BLOCK_JOB: "False"
also any time something is quoted in various ways in these examples gitlab ci made me do it exactly like that, any other variations and it shits the bed.
so once again the latest tool is just more black magic, hot take: gitlab-ci is bad.
I want to run a gitlab job in every case EXCEPT the variable RUN is defined and contains the value run3.
So again:
variable RUN is undefined: run the job
variable RUN is defined and does not contain the value run3: run the job
variable RUN is defined and does contain the value run3: DO NOT run the job
I tried the following rules:
rules:
- if: $RUN != null && $RUN == "run3"
when: never
rules:
- if: $RUN != "" && $RUN == "run3"
when: never
rules:
- if: $RUN == "run3"
when: never
Without the variable RUN being defined, no job was run. I expected the job to have been run.
How to fix it? Where is the problem in the logic?
After some time-consuming try-and-error trials I found the solution: As you cannot test if a variable exists, you have to create the rule the other way around. It means, you run the job if the value does not match:
The correct rule is then:
- if: $RUN != "run3"
when: always
- when: never
The first part only evaluates to true if the variable exists AND if the variable does not evaluate to run3. In that case the job is run.
The second part then tells to NOT run the job in any other case.
You're close
rules:
- if: $RUN == "run3"
when: never
This is correct, but you also need a default - when:
To emulate the default behavior of no rules when the first rule does not match, set the default as on_success
rules:
- if: $RUN == "run3"
when: never # don't run the job in this case
- when: on_success # run per usual in all other cases
Alex, maybe you can try:
if: $RUN != undefined && $RUN == "run3"
when: never
I have a few jobs in my yaml files, and I was finding a way to only run some of the jobs when a particular scheduled is run (i.e. when a certain variable is set).
e.g.
** I have created a new schedule called 'Hourly Schedule', and created a variable called $HOURLY, which is set to TRUE.
HOURLY SCHEDULE
I also have another schedule called 'Daily Schedule'
DAILY SCHEDULE
stages:
- build
- test
- deploy
build-job:
stage: build
script:
- echo "Build."
test-job:
stage: test
script:
- echo "Test."
deploy-job:
stage: deploy
script:
- echo "Deploy."
hourly-deploy-job:
stage: deploy
script:
- echo "Hourly Deploy."
rules:
- if: $HOURLY
My question is:
How do I make 'build-job', 'test-job', and 'deploy-job' to run during 'commit', 'push', or 'Daily Schedule'
And during 'Hourly Schedule', only the 'hourly-deploy-job' should run (and not other jobs).
I know I can do the following:
stages:
- build
- test
- deploy
.hourly_deploy: &hourly_deploy
except:
variables:
- $HOURLY == "TRUE"
build-job:
<<: *hourly_deploy
stage: build
script:
- echo "Build."
test-job:
<<: *hourly_deploy
stage: test
script:
- echo "Test."
deploy-job:
<<: *hourly_deploy
stage: deploy
script:
- echo "Deploy."
hourly-deploy-job:
stage: deploy
script:
- echo "Hourly Deploy."
rules:
- if: $HOURLY
However, I ideally do a case only only on the 'hourly-deploy-job'. This is because my yaml might potentially be bigger at some point, and if I'm not careful, I might forget to add the <<: *hourly_deploy tag on the new job, which means that job will run during Hourly Schedule.
This is not an easy thing to achieve, and I think you should rethink your CICD strategy and use templates.
You mentioned this pipeline might grow and using templates might help you a lot.
Here is my example that works but it's not pretty.
stages:
- build
- test
- deploy
- hourly-deploy-job
build-job:
stage: build
rules:
- if: ($CI_PIPELINE_SOURCE == "push")
when: always
- if: ($CI_PIPELINE_SOURCE == "schedule") && ($DAILY == "true")
when: always
script:
- echo "Build."
test-job:
stage: test
rules:
- if: ($CI_PIPELINE_SOURCE == "push")
when: always
- if: ($CI_PIPELINE_SOURCE == "schedule") && ($DAILY == "true")
when: always
script:
- echo "Test."
deploy-job:
stage: deploy
rules:
- if: ($CI_PIPELINE_SOURCE == "push")
when: always
- if: ($CI_PIPELINE_SOURCE == "schedule") && ($DAILY == "true")
when: always
script:
- echo "Deploy."
hourly-deploy-job:
stage: hourly-deploy-job
rules:
- if: ($CI_PIPELINE_SOURCE == "schedule") && ($HOURLY == "true")
when: always
script:
- echo "Hourly Deploy."
Source:
https://docs.gitlab.com/ee/ci/yaml/#keywords
https://docs.gitlab.com/ee/ci/yaml/workflow.html
https://docs.gitlab.com/ee/ci/jobs/job_control.html
https://docs.gitlab.com/ee/ci/yaml/#rules