Difference between different GitLab CI Merge Request rules - gitlab

As several other users, I'm facing issues with duplicate pipelines in GitLab CI/CD. While there is some documentation on how to prevent this scattered around in the GitLab docs,
my impression is that the indivdual docs pages and sections are rather inconsistent.
My question is, what are the differences between the following rules? Or, more specifically, are there cases in which these rules are evaluated differently?
Switch between branch pipelines and merge request pipelines suggest this to identify merge request pipelines:
if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
Additionally, Switch between branch pipelines and merge request pipelines also uses this rule in combination with $CI_COMMIT_BRANCH &&:
if: '$CI_OPEN_MERGE_REQUESTS'
Moreover, the MergeRequest-Pipelines.gitlab-ci.yml uses a third rule:
if: $CI_MERGE_REQUEST_IID
Any explanation or hints to docs pages I might have overlooked is highly appreciated.

In order to avoid duplicate pipeline creation and the requirement that you want to switch between Branch-Pipelines and Merge-Request-Pipelines I recommend using these workflow rules
workflow:
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS'
when: never
- if: '$CI_COMMIT_BRANCH' || '$CI_COMMIT_TAG'
There is another SO-question that asks how to prevent duplicate pipelines here
The Explanation(s):
In the following section I will try to explain your different rules and how GitLab CI will evaluate them during pipeline creation.
The merge_request_event-rule
Using this rule:
if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
will create a pipeline each time a Merge-Request is created/updated, but there will also be a pipeline created for the branch if you do not have another prevention mechanism (rule).
As the variable naming also points out, this is about the source of the pipeline trigger, other sources could be schedule, push, trigger etc.
The CI_OPEN_MERGE_REQUESTS variable:
Using a rule like:
if: '$CI_OPEN_MERGE_REQUESTS'
GitLab will create new pipelines if there is an open Merge-Request for this branch. Pipelines because there will be a Merge-Request pipeline (denoted with the detached flag) and a branch pipeline for the branch you pushed changes.
if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS'
This rule above will create a pipeline for your branch when, and only when there is an open MR on that branch.
if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS'
when: never
When using the above combination no pipeline will be created if there are open Merge-Requests on that branch, which might also be undesirable since the CI should run tests for branches and/or Merge-Requests.
But how to be able to have pipelines for MRs and Branches, but prevent duplications in pipeline creation?
- if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS'
when: never
- if: '$CI_COMMIT_BRANCH' || '$CI_COMMIT_TAG'
With this rule set above GitLab will create pipelines for branches and Merge-Requests (the detached ones), as well as pipelines for git-tags, but it will prevent GitLab from duplicating pipelines.
The last rule evaluates to true either when there is a commit to a branch or there is a git-tag.
Further links
Official docs on switching between MR- and Branch-Pipelines
Docs on how to avoid duplicate pipelines with rules examples

Related

Non-duplicate branch and merge request pipelines in GitLab CI with a conditionally automated build stage

We're running a self-hosted GitLab CE instance. I'm trying to construct a very particular ruleset that works in our project environment. I'm aware of the basic building blocks: branch pipelines and MR pipelines, workflow:rules for pipelines, rules for jobs, predefined $CI_* variables, etc. I'm having trouble bringing them all together in a way that would also avoid duplicate pipelines, a common problem that may require very specific rule definitions to overcome.
We have a main branch from which feature branches are created, and merged back via merge requests. Occasionally some trivial fixes are pushed directly to main.
We've defined a pipeline with three jobs, currently all set to when: manual until we figure out this scheme. The eventual goal is this:
pre-build tests (stage: .pre)
Always runs as the first job, when the pipeline runs.
Has allow_failure: false – if the tests don't pass, no further jobs should run.
build (stage: build)
Conditions detailed below.
deploy to production (stage: deploy)
Should only be run manually in all situations.
Has dependencies: [build]
The conditions we'd like to have for the creation of the entire pipeline:
A merge request for a feature branch is merged to main: run the pipeline for main.
Commits are pushed directly to main: run the pipeline for main.
(Side note: we want to prevent duplicate pipelines from the above two conditions.)
Commits are pushed to a new or existing feature branch: run the pipeline for the feature branch. Does not matter if it has an open MR or not.
All other cases (tags, new MR creation from/to any branch, etc.): DON'T run the pipeline.
If the pipeline runs, a couple of extra conditions for the build job specifically:
The pipeline is running for the main branch (via a push or an MR, doesn't matter): always build, assuming the tests passed.
The pipeline is running for a feature branch: must build manually.
Achieving this setup is probably not THAT many rules in total, but I'd like to not have to do hours of trial and error if this is something that an expert can readily help with.
I eventually figured it out. Here's how the above conditions are met, just in case they're of use to someone else down the line:
workflow:
rules:
# Explicitly enable merge request pipelines to be created (not done by default)
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
# Prevent duplicate pipelines from push events to branches that are the source
# branch of one or more open merge requests. If $CI_OPEN_MERGE_REQUESTS is
# non-empty on a branch pipeline, it means that the the above rule has already
# caused (or will cause) a merge request pipeline to run. These rules are
# separately evaluated for the merge request event resulting from the same push.
- if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS
when: never
# Run a normal branch pipeline on a branch that isn't the source of an open MR
- if: $CI_COMMIT_BRANCH
pre-build tests:
stage: .pre
when: always
allow_failure: false
# etc.
build:
stage: build
rules:
# Automatically done assuming tests pass, but only on the default branch.
# Also runs when a feature branch MR is merged to the default branch; the
# pipeline source for that is again `push`, not `merge_request_event`.
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
when: on_success
- when: manual
# etc.
deploy to production:
stage: deploy
when: manual
dependencies: [build]
# etc.
The one part of the question that I couldn't do is this one:
new MR creation from/to any branch: DON'T run the pipeline
...while also satisfying this one (much more important):
Commits are pushed to a new or existing feature branch: run the pipeline for the feature branch.
A merge request pipeline is run when a merge request is created for an existing branch. When that branch was first pushed, a branch pipeline was run for a certain $CI_COMMIT_SHA. When the MR creation event pipeline runs, it builds for that same SHA. (However, this doesn't constitute a "duplicate pipeline" as described, because the pipelines are created by two different user actions—unlike a single push to an MR source branch without the $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS trick.)
There's no official way of preventing this; a related GitLab issue can be found here, open as of this writing. Thankfully, the practical impact of this is negligible in our case.

enable merge request pipeline only on master

I am lots of branches in my repo, now I want to add MR pipeline for the master branch, and enable the Merge checks and Pipelines must succeed optional. unfortunately the repo have so many branches that I don't want to create .gitlab-ci.yml for them one by one. but as I have enabled the options for merge check, MR to the branched are stuck because they are not passed pipeline(no pipeline is configured for the branch).
So anyone can help me out?
Can you also paste an snipped of your .gitlab-ci.yml file?
Hoping you must have tried this to your stages?
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event" && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'master'
This basically will run only for a merge request to the master branch. And this chunk should be inside the source branch .gitlab-ci.yml

How to run a pipeline manually without getting "No stages / jobs for this pipeline"? [duplicate]

My GitLab pipelines execute automatically on every push, I want to manually run pipeline and not on every push.
Pipeline docs: https://docs.gitlab.com/ee/ci/yaml/#workflowrules
I tried this in
.gitlab-ci.yml
workflow:
rules:
- when: manual # Error: workflow:rules:rule when unknown value: manual
as mentioned in the documentation, I think you should specify a condition that tells Gitlab to not run the pipeline specifically on push events like so:
workflow:
rules:
- if: '$CI_PIPELINE_SOURCE == "push"'
when: never # Prevent pipeline run for push event
- when: always # Run pipeline for all other cases
Well, this was all from the official documentation but I hope that this may help you :)
We can define your jobs to be only executed on Gitlab.
The web option is used for pipelines created by using Run pipeline button in the GitLab UI, from the project's CI/CD > Pipelines section.
only:
- web
Here is the solution I cam up with:
workflow:
rules:
- if: '$CI_PIPELINE_SOURCE == "web"'
when: always
- when: never
This specifies that it will only run if you click the "Run Pipeline" button in the web UI. In all other cases it will not be triggered.
the when section should not be within rules
workflow:
when: manual

How to write a rule in Gitlab CI for running a job after merge to master?

I need to run a certain job on master branch after each merge to it. Pipelines that were scheduled or run manually should not contain this job. Is there a way to do that in GitLab CI?
As a side note, I want to say that I don't want to use "merge_request_event" because it triggers an additional pipeline after every commit to a merge request and it's something that doesn't suit me.
But it will also run while triggering a pipeline on master manually and I don't need that
You can add this in you .gitlab-ci.yml file.
job_name:
script:
- your_job_part_1
- your_job_part_2
only:
- master
It will run you job_name each time master is updated, hence, each time a merge request is accepted and branch is merged into master.
You can take a look at GitLab CI/CD for more options.
You can use rules keyword:
job_name:
rules:
- if: $CI_COMMIT_BRANCH == 'master' && $CI_PIPELINE_SOURCE == 'merge_request_event'
You can also replace master string by $CI_DEFAULT_BRANCH variable if you master/main branch is the default.
See: https://docs.gitlab.com/ee/ci/yaml/index.html#rules

GitLab run pipeline only manually and not automatically

My GitLab pipelines execute automatically on every push, I want to manually run pipeline and not on every push.
Pipeline docs: https://docs.gitlab.com/ee/ci/yaml/#workflowrules
I tried this in
.gitlab-ci.yml
workflow:
rules:
- when: manual # Error: workflow:rules:rule when unknown value: manual
as mentioned in the documentation, I think you should specify a condition that tells Gitlab to not run the pipeline specifically on push events like so:
workflow:
rules:
- if: '$CI_PIPELINE_SOURCE == "push"'
when: never # Prevent pipeline run for push event
- when: always # Run pipeline for all other cases
Well, this was all from the official documentation but I hope that this may help you :)
We can define your jobs to be only executed on Gitlab.
The web option is used for pipelines created by using Run pipeline button in the GitLab UI, from the project's CI/CD > Pipelines section.
only:
- web
Here is the solution I cam up with:
workflow:
rules:
- if: '$CI_PIPELINE_SOURCE == "web"'
when: always
- when: never
This specifies that it will only run if you click the "Run Pipeline" button in the web UI. In all other cases it will not be triggered.
the when section should not be within rules
workflow:
when: manual

Resources