I am lots of branches in my repo, now I want to add MR pipeline for the master branch, and enable the Merge checks and Pipelines must succeed optional. unfortunately the repo have so many branches that I don't want to create .gitlab-ci.yml for them one by one. but as I have enabled the options for merge check, MR to the branched are stuck because they are not passed pipeline(no pipeline is configured for the branch).
So anyone can help me out?
Can you also paste an snipped of your .gitlab-ci.yml file?
Hoping you must have tried this to your stages?
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event" && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'master'
This basically will run only for a merge request to the master branch. And this chunk should be inside the source branch .gitlab-ci.yml
Related
I have created a simple readme file in the main repository. Now for example if someone creates a feature branch out of the main branch and edits the readme file. Those commits will be then committed in the feature branch.
Now I want to create a pipeline which should do the following:-
Whenever there is a push in the repo it should run the pipeline automatically.
The pipeline should check that every NEW content to the README.md has to have at least a one time mentioning of words like "ABQ" OR "abq" OR "albuquerque" OR "Albuquerque".
So basically The push based pipeline shall check if the added text in the feature branch does at least have one occurrence of the above mentioned words.
Here is the yml file i have that runs whenever there is a push. Can someone guide me here for achieving the above goal?
workflow:
rules:
- if: $CI_PIPELINE_SOURCE == "push"
when: always
We're running a self-hosted GitLab CE instance. I'm trying to construct a very particular ruleset that works in our project environment. I'm aware of the basic building blocks: branch pipelines and MR pipelines, workflow:rules for pipelines, rules for jobs, predefined $CI_* variables, etc. I'm having trouble bringing them all together in a way that would also avoid duplicate pipelines, a common problem that may require very specific rule definitions to overcome.
We have a main branch from which feature branches are created, and merged back via merge requests. Occasionally some trivial fixes are pushed directly to main.
We've defined a pipeline with three jobs, currently all set to when: manual until we figure out this scheme. The eventual goal is this:
pre-build tests (stage: .pre)
Always runs as the first job, when the pipeline runs.
Has allow_failure: false – if the tests don't pass, no further jobs should run.
build (stage: build)
Conditions detailed below.
deploy to production (stage: deploy)
Should only be run manually in all situations.
Has dependencies: [build]
The conditions we'd like to have for the creation of the entire pipeline:
A merge request for a feature branch is merged to main: run the pipeline for main.
Commits are pushed directly to main: run the pipeline for main.
(Side note: we want to prevent duplicate pipelines from the above two conditions.)
Commits are pushed to a new or existing feature branch: run the pipeline for the feature branch. Does not matter if it has an open MR or not.
All other cases (tags, new MR creation from/to any branch, etc.): DON'T run the pipeline.
If the pipeline runs, a couple of extra conditions for the build job specifically:
The pipeline is running for the main branch (via a push or an MR, doesn't matter): always build, assuming the tests passed.
The pipeline is running for a feature branch: must build manually.
Achieving this setup is probably not THAT many rules in total, but I'd like to not have to do hours of trial and error if this is something that an expert can readily help with.
I eventually figured it out. Here's how the above conditions are met, just in case they're of use to someone else down the line:
workflow:
rules:
# Explicitly enable merge request pipelines to be created (not done by default)
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
# Prevent duplicate pipelines from push events to branches that are the source
# branch of one or more open merge requests. If $CI_OPEN_MERGE_REQUESTS is
# non-empty on a branch pipeline, it means that the the above rule has already
# caused (or will cause) a merge request pipeline to run. These rules are
# separately evaluated for the merge request event resulting from the same push.
- if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS
when: never
# Run a normal branch pipeline on a branch that isn't the source of an open MR
- if: $CI_COMMIT_BRANCH
pre-build tests:
stage: .pre
when: always
allow_failure: false
# etc.
build:
stage: build
rules:
# Automatically done assuming tests pass, but only on the default branch.
# Also runs when a feature branch MR is merged to the default branch; the
# pipeline source for that is again `push`, not `merge_request_event`.
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
when: on_success
- when: manual
# etc.
deploy to production:
stage: deploy
when: manual
dependencies: [build]
# etc.
The one part of the question that I couldn't do is this one:
new MR creation from/to any branch: DON'T run the pipeline
...while also satisfying this one (much more important):
Commits are pushed to a new or existing feature branch: run the pipeline for the feature branch.
A merge request pipeline is run when a merge request is created for an existing branch. When that branch was first pushed, a branch pipeline was run for a certain $CI_COMMIT_SHA. When the MR creation event pipeline runs, it builds for that same SHA. (However, this doesn't constitute a "duplicate pipeline" as described, because the pipelines are created by two different user actions—unlike a single push to an MR source branch without the $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS trick.)
There's no official way of preventing this; a related GitLab issue can be found here, open as of this writing. Thankfully, the practical impact of this is negligible in our case.
I want a certain job to trigger when a tag is pushed to main branch
I have the following gitlab ci config
job1:
rules:
- if: ($CI_COMMIT_TAG =~ /(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)/ && $CI_COMMIT_BRANCH == "main")
I usually do the following in git
git checkout main
git merge develop
git tag "1.0.0"
git push --atomic origin main "1.0.0"
However it is not triggering, what am I doing wrong?
There are different types of pipelines that can be triggered in GitLab. What pre-defined variables are available will differ depending on the type of pipeline. As per the documentation:
CI_COMMIT_TAG is "available only in pipelines for tags."
CI_COMMIT_BRANCH is "available in branch pipelines."
Since your pipeline is not going to be a tag pipeline and branch pipeline at the same time, your condition will never evaluate to true.
I suggest only using the first part where you've specified the formatting for the tag.
As several other users, I'm facing issues with duplicate pipelines in GitLab CI/CD. While there is some documentation on how to prevent this scattered around in the GitLab docs,
my impression is that the indivdual docs pages and sections are rather inconsistent.
My question is, what are the differences between the following rules? Or, more specifically, are there cases in which these rules are evaluated differently?
Switch between branch pipelines and merge request pipelines suggest this to identify merge request pipelines:
if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
Additionally, Switch between branch pipelines and merge request pipelines also uses this rule in combination with $CI_COMMIT_BRANCH &&:
if: '$CI_OPEN_MERGE_REQUESTS'
Moreover, the MergeRequest-Pipelines.gitlab-ci.yml uses a third rule:
if: $CI_MERGE_REQUEST_IID
Any explanation or hints to docs pages I might have overlooked is highly appreciated.
In order to avoid duplicate pipeline creation and the requirement that you want to switch between Branch-Pipelines and Merge-Request-Pipelines I recommend using these workflow rules
workflow:
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS'
when: never
- if: '$CI_COMMIT_BRANCH' || '$CI_COMMIT_TAG'
There is another SO-question that asks how to prevent duplicate pipelines here
The Explanation(s):
In the following section I will try to explain your different rules and how GitLab CI will evaluate them during pipeline creation.
The merge_request_event-rule
Using this rule:
if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
will create a pipeline each time a Merge-Request is created/updated, but there will also be a pipeline created for the branch if you do not have another prevention mechanism (rule).
As the variable naming also points out, this is about the source of the pipeline trigger, other sources could be schedule, push, trigger etc.
The CI_OPEN_MERGE_REQUESTS variable:
Using a rule like:
if: '$CI_OPEN_MERGE_REQUESTS'
GitLab will create new pipelines if there is an open Merge-Request for this branch. Pipelines because there will be a Merge-Request pipeline (denoted with the detached flag) and a branch pipeline for the branch you pushed changes.
if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS'
This rule above will create a pipeline for your branch when, and only when there is an open MR on that branch.
if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS'
when: never
When using the above combination no pipeline will be created if there are open Merge-Requests on that branch, which might also be undesirable since the CI should run tests for branches and/or Merge-Requests.
But how to be able to have pipelines for MRs and Branches, but prevent duplications in pipeline creation?
- if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS'
when: never
- if: '$CI_COMMIT_BRANCH' || '$CI_COMMIT_TAG'
With this rule set above GitLab will create pipelines for branches and Merge-Requests (the detached ones), as well as pipelines for git-tags, but it will prevent GitLab from duplicating pipelines.
The last rule evaluates to true either when there is a commit to a branch or there is a git-tag.
Further links
Official docs on switching between MR- and Branch-Pipelines
Docs on how to avoid duplicate pipelines with rules examples
I need to run a certain job on master branch after each merge to it. Pipelines that were scheduled or run manually should not contain this job. Is there a way to do that in GitLab CI?
As a side note, I want to say that I don't want to use "merge_request_event" because it triggers an additional pipeline after every commit to a merge request and it's something that doesn't suit me.
But it will also run while triggering a pipeline on master manually and I don't need that
You can add this in you .gitlab-ci.yml file.
job_name:
script:
- your_job_part_1
- your_job_part_2
only:
- master
It will run you job_name each time master is updated, hence, each time a merge request is accepted and branch is merged into master.
You can take a look at GitLab CI/CD for more options.
You can use rules keyword:
job_name:
rules:
- if: $CI_COMMIT_BRANCH == 'master' && $CI_PIPELINE_SOURCE == 'merge_request_event'
You can also replace master string by $CI_DEFAULT_BRANCH variable if you master/main branch is the default.
See: https://docs.gitlab.com/ee/ci/yaml/index.html#rules