Run all jobs on a gitlab ci MR pipeline, even if some don't have a merge_request_event rule, but do not run both MR and branch pipelines - gitlab

In case the terminology is not standard, here is how I am using the below terms:
branch pipeline: A pipeline that is run when pushing to a branch.
MR pipeline: A pipeline that is run on a merge request, or pushes to a merge request branch.
I want to write a pipeline with two jobs, job_A and job_B. job_A should run on all pipelines. job_B should run only on merge request pipelines. One solution is to combine the workaround proposed in issue 194129, adding a workflow rule of - if: $CI with a merge_request_event rule, i.e.:
image: alpine
workflow:
rules:
- if: $CI
stages:
- stage_A
- stage_B
job_A:
stage: stage_A
script:
- echo "Hello from Job A"
job_B:
stage: stage_B
rules:
- if: $CI_PIPELINE_SOURCE == 'merge_request_event'
script:
- echo "Hello from Job B"
Now my pipeline runs in full on the MR -- which is what I wanted. However, two pipelines are being run now, the branch pipeline and the MR pipeline.
I want both job_A and job_B to run on MR pipelines even though job_A doesn't have the merge_request_event rule. But, I only want one pipeline to run when an MR is open -- the MR pipeline. How can I achieve this?

The correct answer is found in the gitlab docs.
workflow:
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
- if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS
when: never
- if: $CI_COMMIT_BRANCH

Try using only:variables. I had the same issue when mixing only/except with rules in one pipeline.
job_B:
stage: stage_B
only:
variables:
- $CI_PIPELINE_SOURCE == "merge_request_event"
script:
- echo "Hello from Job B"

Related

Gitlab-ci: if MR exist just trigger merge_request detach pipeline, if not trigger source branch pipeline. Those 2 pipelines shouldn't run in same time

I want to see sonar results in the MR(merge request) command section when I create a MR.
My main expectations:
if there is an existing MR for the source branch, trigger detached pipeline (do not trigger feature pipeline. I need only that one for reviewing sonar results in MR commands)
if there isn't an existing MR for the source branch, just trigger the normal feature(source) branch pipeline
I tried to do it with the below example stage. But when I pushed the commit to the source pipeline, while MR is exist for source branch. I still getting double pipeline. Detach and source pipelines are running and I don't want to see both in same time, plus except not working with rules configuration. How can I integrate except section with rules part.
This is my gitlab-ci stage:
deploy:
stage: deployment
when: manual
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event" && $CUSTOM_VARIABLE == "true" || $CUSTOM_VARIABLE == "true"'
script:
- ....
- ....
except:
- tags
- main
I also tried below rules, if one of them fit my condition don't run the other one. But it still trigger both pipelines.
deploy:
stage: deployment
when: manual
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event" && $CUSTOM_VARIABLE == "true"'
when: on_success
- if: '$CI_PIPELINE_SOURCE == "push" && $CUSTOM_VARIABLE == "true"'
when: on_success
script:
- ....
- ....
except:
- tags
- main
Covered in workflow:rules templates, In this case, you can use the CI_OPEN_MERGE_REQUESTS variable to determine whether to run the pipeline for merge request or just the feature branch.
If you use both [pipelines for merge requests and branch pipelines], duplicate pipelines might run at the same time. To prevent duplicate pipelines, use the CI_OPEN_MERGE_REQUESTS variable.
Using workflow:rules you can do this for the entire pipeline, but the same principle can also be applied to individual jobs.
workflow:
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS'
when: never
- if: '$CI_COMMIT_BRANCH'
This means your pipeline will run:
for merge requests
for branch pipelines UNLESS there is an open merge requests

How to make gitlab run jobs in sequential order?

I have a gitlab-ci.yml file like the following:
stages:
- test
- job1
- job2
test:
stage: test
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
script:
...
myjob1:
stage: job1
script:
...
myjob2:
stage: job2
script:
...
According to the documentation HERE (or at least how I understood it), the first stage/job is only run when I create a merge request.
This is true, but the next stage (job1) is run in parallel when the first job (test) has been started. As far as I understand the stages which are defined in the order test -> job1 -> job2) always run in sequence.
So what am I doing wrong? Why do job test and job1 run in parallel, and not in sequence as expected?
After a lot of trial-end-errors and reading and rereading parts of the really unclear and confusing documentation I might have found a solution.
First, the stage you only want to run on a merge request (or which you do not want to run if you schedule a trigger or start the pipeline manually), you need to change that stage as follows:
test:
rules:
- if: $CI_PIPELINE_SOURCE == "web" || $CI_PIPELINE_SOURCE == "schedule"
when: never
- when: on_success
stage: test
script:
- 'echo "Running Test"'
- 'echo $CI_PIPELINE_SOURCE'
Here, you define a rule that checks if the variable CI_PIPELINE_SOURCE is either web or schedule. If the variable is web this means a manual pipeline trigger (i.e. you pressed manually on Run pipeline, which is not explained in the documentation), or if the pipeline is triggered by a schedule (not tested, but I assume that is what schedule means).
So if the pipeline is triggered by a scheduled event or manually, never tells gitlab to not execute that stage. The when: on_success is like an else statement, which tells gitlab to run that stage in any other case.
However, that is not the complete story. Because when you use git to make changes to the code and push it to gitlab via git push, you have two triggers in gitlab! A trigger merge_request_event and a trigger push. That means, the pipeline is started twice!
To avoid the pipeline started twice you need to use the workflow key, which helps to define if the pipeline (=workflow) is run or not. (The term workflow seems to mean pipeline). Here is the code to be put into the gitlab-ci.yml file:
workflow:
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
when: never
- when: always
This construct suppresses the pipeline to be run when the trigger is merge_request_event. In that case, the additional pipeline is not run. In all other cases (when the trigger is e.g. push), the pipeline is run.
So here is the complete gitlab-ci.yaml code:
workflow:
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
when: never
- when: always
stages:
- test
- stage1
- stage2
test:
rules:
- if: $CI_PIPELINE_SOURCE == "web" || $CI_PIPELINE_SOURCE == "schedule"
when: never
- when: on_success
stage: test
script:
- 'echo "Running Test"'
my_stage1:
stage: stage1
script:
- 'echo "Running stage 1"'
my_stage2:
stage: stage2
script:
- 'echo "Running stage 2"'
If you make a git push then one pipeline is run with the stages test, my_stage1 and my_stage2, and when you start the pipeline manually or if it is triggered by a schedule, one pipeline is started with the stages my_stage1 and my_stage2.
As to why this is so complicated and confusing, I have not the slightest idea.

What is the purpose of "workflow:rules" in Gitlab-ci pipelines?

I am a bit confused between Gitlab CI pipeline workflow:rules and job:rules
workflow:
rules:
- if: '$CI_PIPELINE_SOURCE == "push"'
- if: '$CI_PIPELINE_SOURCE != "schedule"'
and
test:
stage: test
image: image
script:
- echo "Hello world!"
rules:
- if: $CI_PIPELINE_SOURCE == "schedule"
What happens if both of them were used in the same GitLab-ci YAML file.
With worfklow you configure when a pipeline is created while with rules you configure when a job is created.
So in your example pipelines are created for pushes but cannot be scheduled while your test job will only run when scheduled.
But as workflow rules take precedence over job rules, no pipeline will be created in your example as your rules for workflow and job are mutually exclusive.

Gitlab run a pipeline job when a merge request is merged

I have a gitlab pipeline where there are two stages, one is build and the other one is deploy. The build stage is run when a commit is made. I want a way to run the deploy job when the merge request is merged to master. I tried several things but no luck. Can anyone help?
stages:
- build
- deploy
dotnet:
script: "echo This builds!"
stage: build
production:
script: "echo This deploys!"
stage: deploy
only:
refs:
- master
Try using the gitlab-ci.yml "rules" feature to check for the merge request event.
Your current gitlab-ci.yml will run your "dotnet" job every commit, merge request, schedule, and manually triggered pipeline.
https://docs.gitlab.com/ee/ci/yaml/#workflowrules
dotnet:
script: "echo This builds!"
stage: build
rules:
- if: '$CI_COMMIT_REF_NAME != "master" && $CI_PIPELINE_SOURCE == "push" || $CI_PIPELINE_SOURCE == "merge_request_event"'
production:
script: "echo This deploys!"
stage: deploy
rules:
- if: '$CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_REF_NAME == "master"'
If you want your job run on only after merging the merge request, then you can trigger a job based on the commit message, as shown below.
rules:
- if: '$CI_COMMIT_MESSAGE =~ /See merge request/'
Basically, all the merge request comes with a "See merge request" commit message so you can depend on that message to trigger your job.
You can run the pipeline after merge by using Gitlab ci predefined variable $CI_MERGE_REQUEST_APPROVED this will return true after merge and available from gitlab v14.1.
If you want to run the pipeline when merge request is created to main branch you can use $CI_MERGE_REQUEST_TARGET_BRANCH_NAME with $CI_MERGE_REQUEST_APPROVED.
you can add the rule like this in your job.
rules:
- if: $CI_MERGE_REQUEST_APPROVED && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "main"
CI_MERGE_REQUEST_APPROVED variable seems to be designed for check approve status, not for triggering pipeline. (Issue, https://gitlab.com/gitlab-org/gitlab/-/issues/375908)
When you use merge request predefined variables, gitlab creates pipeline at merge request created, not at merge request approved.
Instead, use CI_COMMIT_BRANCH predefined variables with regular expression and restrict commit at master branch using branch access rights.
stages:
- build
- deploy
dotnet:
stage: build
rules:
- if: $CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == "master"
script: "echo This builds!"
production:
stage: deploy
rules:
- if: $CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == "master"
script: "echo This deploys!"
Edit answer for incorrect example. (edited at 2023. 01. 26)
Assume that we have rc and master branch.
We can configure master branch as protected, and use only for merge request.
In this case, we can define gitlab-ci file like below.
stages:
- build
- deploy
dotnet:
stage: build
rules:
- if: $CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == "rc"
script: "echo This builds!"
production:
stage: deploy
rules:
- if: $CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == "master"
script: "echo This deploys!"
Thanks for notification #Kappacake

Accept merge request without running manual stages

I have a pipeline with 3 stages: build, deploy-test and deploy-prod. I want stages to have following behavior:
always run build
run deploy-test automatically when on master or manually when on other branches
run deploy-prod manually, only available on master branch
My pipeline configuration seems to achieve that but I have a problem when trying to merge branches into master. I don't want to execute deploy-test stage on every branch before doing merge. Right now I am required to do that as the merge button is disabled with a message Pipeline blocked. The pipeline for this merge request requires a manual action to proceed. The setting Pipelines must succeed in project is disabled.
I tried adding additional rule to prevent deploy-test stage from running in merge requests but it didn't change anything:
rules:
- if: '$CI_MERGE_REQUEST_ID'
when: never
- if: '$CI_COMMIT_BRANCH == "master"'
when: on_success
- when: manual
Full pipeline configuration:
stages:
- build
- deploy-test
- deploy-prod
build:
stage: build
script:
- echo "build"
deploy-test:
stage: deploy-test
script:
- echo "deploy-test"
rules:
- if: '$CI_COMMIT_BRANCH == "master"'
when: on_success
- when: manual
deploy-prod:
stage: deploy-prod
script:
- echo "deploy-prod"
only:
- master
The only way I got it to work was to set ☑️ Skipped pipelines are considered successful in Setttings > General > Merge requests > Merge Checks
and marking the manual step as "allow_failure"
upload:
stage: 'upload'
rules:
# Only allow uploads for a pipeline source whitelisted here.
# See: https://docs.gitlab.com/ee/ci/jobs/job_control.html#common-if-clauses-for-rules
- if: $CI_COMMIT_BRANCH
when: 'manual'
allow_failure: true
After this clicking the Merge when Pipeline succeeds button …
… will merge the MR without any manual interaction:
I've opened a merge request from branch "mybranch" into "master" with the following .gitlab-ci.yml:
image: alpine
stages:
- build
- deploy-test
- deploy-prod
build:
stage: build
script:
- echo "build"
# run deploy-test automatically when on master or manually when on other branches
# Don't run on merge requests
deploy-test:
stage: deploy-test
script:
- echo "deploy-test"
rules:
- if: $CI_MERGE_REQUEST_ID
when: never
- if: '$CI_COMMIT_BRANCH == "master"'
when: on_success
- when: manual
# run deploy-prod manually, only available on master branch
deploy-prod:
stage: deploy-prod
script:
- echo "deploy-prod"
rules:
- if: '$CI_COMMIT_BRANCH == "master"'
when: manual
Notes:
only is deprecated, so I replaced it with if
I added Alpine image to make the jobs run faster (slimmer container); it doesn't affect the logic
When I pushed changes to branch "mybranch", GitLab did the following:
showed a blue "Merge when pipeline succeeds" button on my MR
ran "build" stage
skipped "deploy-prod" stage (only available on "master" branch)
gave me a manual "play" button to run the job on "mybranch"
at this point, the pipeline status is "blocked" and the MR is showing "Pipeline blocked. The pipeline for this merge request requires a manual action to proceed"
now I manually start the "deploy-test" stage by selecting the Play icon in the Pipelines screen
pipeline status indicator changes to "running" and then to "passed"
my merge request shows the pipeline passed and gives me the green "Merge" button
There are a number of variables that are available to the pipeline on runtime - Predefined variables reference
Some are available specifically for pipelines associated with merge requests - Predefined variables for merge request pipelines
You can utilize one or more of these variables to determine if you would want to run the deploy-test job for that merge request.
For example, you could use mention the phrase "skip_cicd" in your merge request title, access it with CI_MERGE_REQUEST_TITLE variable and create a rule. Your pipeline would look somewhat like this (please do test the rule, I have edited the pipeline off the top of my head and could be wrong) -
stages:
- build
- deploy-test
- deploy-prod
build:
stage: build
script:
- echo "build"
deploy-test:
stage: deploy-test
script:
- echo "deploy-test"
rules:
- if: '$CI_MERGE_REQUEST_TITLE == *"skip_cicd"*'
when: never
- if: '$CI_COMMIT_BRANCH == "master"'
when: on_success
- when: manual
deploy-prod:
stage: deploy-prod
script:
- echo "deploy-prod"
only:
- master

Resources