Why is gitlab run duplicate pipelines on commit? - gitlab

Question

I have the following content of the .gitlab-ci.yml job:
stages:
- stage1
- stage2
job1:
stage: stage1
script:
- echo "Running default stage1, pipeline_source=$CI_PIPELINE_SOURCE"
job2:
stage: stage2
rules:
- if: $CI_PIPELINE_SOURCE == "push"
- when: always
script:
- echo "Running STAGE2! pipeline_source=$CI_PIPELINE_SOURCE"
when I commit this change to a merge-request branch, it seems two pipelines are being started.
Is this a known issue in gitlab? Or do I understand something wrong here?

Answer 1

GitLab creates pipelines both for your branch and for the merge request. This is an "expected"[1] feature of GitLab as a consequence of using rules:. (oddly enough, when using only/except, merge request pipelines will only happen when using only: - merge_requests).
If you simply want to disable the 'pipelines for merge requests' and only run branch pipelines, you can include the default branch pipelines template, which provides a workflow: that prevents pipelines for merge requests.
include:
- template: 'Workflows/Branch-Pipelines.gitlab-ci.yml'
Additionally, you can see this answer for a workflow that will prevent duplicates between the pipelines for merge requests and branch pipelines only when a merge request is open.
[1]: I've always found this to be a quirk of GitLab and, as an administrator of GitLab for hundreds of users, I've gotten this question many many times. So, you're not alone in being surprised by this 'expected feature'

Answer 2

You didn't do anything wrong. This is actually intended, though it's a weird side-effect of the fact that Merge Requests have their own pipeline contexts. So when you commit to a branch that's associated with a merge request, two pipelines start:
A branch-based pipeline, with no context of the merge request
A merge request pipeline, with all the merge request variables populated (this is called a "detached" pipeline)
You can control this behavior by using a workflow keyword in your pipeline. We use the following workflow settings on our repositories:
workflow:
rules:
- if: $CI_MERGE_REQUEST_IID
- if: $CI_COMMIT_TAG
- if: $CI_PIPELINE_SOURCE == "schedule"
- if: $CI_COMMIT_REF_PROTECTED == "true"
The above rules will prevent the branch pipelines from running unless the branch is a protected branch (I.e., you're merging into the main branch), a tagged commit (I.e., you're releasing code), or the pipeline has been scheduled. This means that when you commit to a MR, the branch-based pipeline (#1 from the above numbers) doesn't run, and you are left with one pipeline running.