I am a bit confused between Gitlab CI pipeline workflow:rules and job:rules
workflow:
rules:
- if: '$CI_PIPELINE_SOURCE == "push"'
- if: '$CI_PIPELINE_SOURCE != "schedule"'
and
test:
stage: test
image: image
script:
- echo "Hello world!"
rules:
- if: $CI_PIPELINE_SOURCE == "schedule"
What happens if both of them were used in the same GitLab-ci YAML file.
With worfklow you configure when a pipeline is created while with rules you configure when a job is created.
So in your example pipelines are created for pushes but cannot be scheduled while your test job will only run when scheduled.
But as workflow rules take precedence over job rules, no pipeline will be created in your example as your rules for workflow and job are mutually exclusive.
Related
In case the terminology is not standard, here is how I am using the below terms:
branch pipeline: A pipeline that is run when pushing to a branch.
MR pipeline: A pipeline that is run on a merge request, or pushes to a merge request branch.
I want to write a pipeline with two jobs, job_A and job_B. job_A should run on all pipelines. job_B should run only on merge request pipelines. One solution is to combine the workaround proposed in issue 194129, adding a workflow rule of - if: $CI with a merge_request_event rule, i.e.:
image: alpine
workflow:
rules:
- if: $CI
stages:
- stage_A
- stage_B
job_A:
stage: stage_A
script:
- echo "Hello from Job A"
job_B:
stage: stage_B
rules:
- if: $CI_PIPELINE_SOURCE == 'merge_request_event'
script:
- echo "Hello from Job B"
Now my pipeline runs in full on the MR -- which is what I wanted. However, two pipelines are being run now, the branch pipeline and the MR pipeline.
I want both job_A and job_B to run on MR pipelines even though job_A doesn't have the merge_request_event rule. But, I only want one pipeline to run when an MR is open -- the MR pipeline. How can I achieve this?
The correct answer is found in the gitlab docs.
workflow:
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
- if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS
when: never
- if: $CI_COMMIT_BRANCH
Try using only:variables. I had the same issue when mixing only/except with rules in one pipeline.
job_B:
stage: stage_B
only:
variables:
- $CI_PIPELINE_SOURCE == "merge_request_event"
script:
- echo "Hello from Job B"
While developing gitlab ci-cd pipeline i want to run pipeline manually for a particular branch for example branch name is "develop-tool". what i want is anything done on this branch should not trigger pipeline automatically. it should be manually. what i have tried is:
workflow:
rules:
- when: manual # Error: workflow:rules:rule when unknown value: manual
then i tried to do this:
- if: '$CI_PIPELINE_SOURCE == "web" && $CI_COMMIT_REF_NAME !~ /^.*-$develop-tool/'
- when: always
its taking job to some infinite loop and it keeps running until and unless you cancel it and run the job again. does anyone have any other way to achieve this goal.
Have the configuration skip pipelines when the source is not the web for that branch. So the only way it can be triggered on that branch is through the web UI.
workflow:
rules:
- if: '$CI_COMMIT_BRANCH == "develop-tool" && $CI_PIPELINE_SOURCE != "web"'
when: never
- when: always
I'm new to this Gitlab CI/CD features, and I encountered the following issues.
I have these 2 jobs in my gitlab-ci.yml, the automation test and my deployment job.
automation_test_scheduled:
stage: test
script:
- yarn test:cypress
only:
- schedules
deploy_to_staging:
stage: deploy-staging
environment: staging
only:
- staging
I want to run my automation test automatically on a daily basis and I have created a new pipeline schedule against my staging branch.
however, when the scheduler is triggered, it also runs my deployment job, which is not needed because I only want my automation test to run in the scheduled pipeline.
Does this happen because my deploy_to_staging job has only: - staging rules? if so, how can I set my scheduled pipeline to only run the automation test without triggering another job?
If you wanted to do this with only/except, it would probably be sufficient to add
except:
- schedules
to your deployment job.
Though as
Though notably, the rules based system is preferred at this point.
This also allows for more expressive and detailed decisions when it comes to running jobs.
The simplest way to set the rules for the two jobs would be:
automation_test_scheduled:
stage: test
script:
- yarn test:cypress
rules:
- if: $CI_PIPELINE_SOURCE == "schedule"
deploy_to_staging:
stage: deploy-staging
environment: staging
rules:
- if: $CI_PIPELINE_SOURCE == "schedule"
when: never
- if: $CI_COMMIT_REF_SLUG == "staging"
And that might be all you need.
Though when it comes to rules, a particularly convenient way of handling them is defining some common rules for the configuration, and reusing these through yaml anchors. The following are some reusable definitions for your case:
.definitions:
rules:
- &if-scheduled
if: $CI_PIPELINE_SOURCE == "schedule"
- ¬-scheduled
if: $CI_PIPELINE_SOURCE == "schedule"
when: never
- &if-staging
if: $CI_COMMIT_REF_SLUG == "staging"
And after that you could use them in your jobs like this:
automation_test_scheduled:
stage: test
script:
- yarn test:cypress
rules:
- *if-scheduled
deploy_to_staging:
stage: deploy-staging
environment: staging
rules:
- *not-scheduled
- *if-staging
This way of handling the rules makes it a bit easier to have a overview, and to reuse rules, which in large configurations definitely makes sense
You should use rules instead of only as the latter is not in active development any more.
With that in mind you can change to the following rules clause using the predefined variables CI_COMMIT_REF_SLUG and CI_PIPELINE_SOURCE. The automation_test_scheduled is only run on the branch staging if triggered by a schedule and the deploy_to_staging job is run on any change on the staging branch.
automation_test_scheduled:
stage: test
script:
- yarn test:cypress
rules:
- if: '$CI_COMMIT_REF_SLUG == "staging" && $CI_PIPELINE_SOURCE == "schedule"'
deploy_to_staging:
stage: deploy-staging
environment: staging
rules:
- if: '$CI_COMMIT_REF_SLUG == "staging"'
I want to see sonar results in the MR(merge request) command section when I create a MR.
My main expectations:
if there is an existing MR for the source branch, trigger detached pipeline (do not trigger feature pipeline. I need only that one for reviewing sonar results in MR commands)
if there isn't an existing MR for the source branch, just trigger the normal feature(source) branch pipeline
I tried to do it with the below example stage. But when I pushed the commit to the source pipeline, while MR is exist for source branch. I still getting double pipeline. Detach and source pipelines are running and I don't want to see both in same time, plus except not working with rules configuration. How can I integrate except section with rules part.
This is my gitlab-ci stage:
deploy:
stage: deployment
when: manual
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event" && $CUSTOM_VARIABLE == "true" || $CUSTOM_VARIABLE == "true"'
script:
- ....
- ....
except:
- tags
- main
I also tried below rules, if one of them fit my condition don't run the other one. But it still trigger both pipelines.
deploy:
stage: deployment
when: manual
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event" && $CUSTOM_VARIABLE == "true"'
when: on_success
- if: '$CI_PIPELINE_SOURCE == "push" && $CUSTOM_VARIABLE == "true"'
when: on_success
script:
- ....
- ....
except:
- tags
- main
Covered in workflow:rules templates, In this case, you can use the CI_OPEN_MERGE_REQUESTS variable to determine whether to run the pipeline for merge request or just the feature branch.
If you use both [pipelines for merge requests and branch pipelines], duplicate pipelines might run at the same time. To prevent duplicate pipelines, use the CI_OPEN_MERGE_REQUESTS variable.
Using workflow:rules you can do this for the entire pipeline, but the same principle can also be applied to individual jobs.
workflow:
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS'
when: never
- if: '$CI_COMMIT_BRANCH'
This means your pipeline will run:
for merge requests
for branch pipelines UNLESS there is an open merge requests
I have a gitlab-ci.yml file like the following:
stages:
- test
- job1
- job2
test:
stage: test
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
script:
...
myjob1:
stage: job1
script:
...
myjob2:
stage: job2
script:
...
According to the documentation HERE (or at least how I understood it), the first stage/job is only run when I create a merge request.
This is true, but the next stage (job1) is run in parallel when the first job (test) has been started. As far as I understand the stages which are defined in the order test -> job1 -> job2) always run in sequence.
So what am I doing wrong? Why do job test and job1 run in parallel, and not in sequence as expected?
After a lot of trial-end-errors and reading and rereading parts of the really unclear and confusing documentation I might have found a solution.
First, the stage you only want to run on a merge request (or which you do not want to run if you schedule a trigger or start the pipeline manually), you need to change that stage as follows:
test:
rules:
- if: $CI_PIPELINE_SOURCE == "web" || $CI_PIPELINE_SOURCE == "schedule"
when: never
- when: on_success
stage: test
script:
- 'echo "Running Test"'
- 'echo $CI_PIPELINE_SOURCE'
Here, you define a rule that checks if the variable CI_PIPELINE_SOURCE is either web or schedule. If the variable is web this means a manual pipeline trigger (i.e. you pressed manually on Run pipeline, which is not explained in the documentation), or if the pipeline is triggered by a schedule (not tested, but I assume that is what schedule means).
So if the pipeline is triggered by a scheduled event or manually, never tells gitlab to not execute that stage. The when: on_success is like an else statement, which tells gitlab to run that stage in any other case.
However, that is not the complete story. Because when you use git to make changes to the code and push it to gitlab via git push, you have two triggers in gitlab! A trigger merge_request_event and a trigger push. That means, the pipeline is started twice!
To avoid the pipeline started twice you need to use the workflow key, which helps to define if the pipeline (=workflow) is run or not. (The term workflow seems to mean pipeline). Here is the code to be put into the gitlab-ci.yml file:
workflow:
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
when: never
- when: always
This construct suppresses the pipeline to be run when the trigger is merge_request_event. In that case, the additional pipeline is not run. In all other cases (when the trigger is e.g. push), the pipeline is run.
So here is the complete gitlab-ci.yaml code:
workflow:
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
when: never
- when: always
stages:
- test
- stage1
- stage2
test:
rules:
- if: $CI_PIPELINE_SOURCE == "web" || $CI_PIPELINE_SOURCE == "schedule"
when: never
- when: on_success
stage: test
script:
- 'echo "Running Test"'
my_stage1:
stage: stage1
script:
- 'echo "Running stage 1"'
my_stage2:
stage: stage2
script:
- 'echo "Running stage 2"'
If you make a git push then one pipeline is run with the stages test, my_stage1 and my_stage2, and when you start the pipeline manually or if it is triggered by a schedule, one pipeline is started with the stages my_stage1 and my_stage2.
As to why this is so complicated and confusing, I have not the slightest idea.