Gitlab CI CD only run Pipeline Step when certain criteria is met - gitlab

Question

I am trying to configure my CI/CD pipeline so that a certain step is only executed if some conditions are met.
My current rule definition looks like this:
rules:
- if: '($CI_PIPELINE_SOURCE == "push" || $CI_PIPELINE_SOURCE == "web" || $CI_PIPELINE_SOURCE == "merge_request_event") && $CI_COMMIT_BRANCH == "develop" '
exists:
- $MY_FILE
What I want to achieve:
Execute step only if a merge from branch_x was done into develop
Or Execute step when pipeline triggered over the GitLab UI
Or Execute step if a commit is pushed directly to develop
And the file in $MY_FILE is present in the repository
Other than expected, the step gets not executed on
Push
Merge Branch_X into develop
Manually trigger over GitLab UI
The corresponding file exists within the repository.
The step work is if I change back to
only:
- develop
But then I can not set the condition that the file needs to be present.

Answer 1

Part of the issue you're running into is that some Predefined Variables only exist in certain types of pipelines. For example, the CI_COMMIT_TAG variable, which would hold the name of a tag, only exists if the pipeline is a "Tag pipeline" (ran after a tag is created). However, if a tag points to a commit that is the HEAD of a branch that is the Source of a Merge Request, it is still just a Tag pipeline, so none of the Merge Request specific variables will exist.
Due to this, your conditional will get slightly more complex since we'll need some more parenthesis, or more if clauses for our rules section:
rules:
- if: ($CI_PIPELINE_SOURCE == 'push' || $CI_PIPELINE_SOURCE == 'web') && $CI_COMMIT_REF_NAME == 'develop'
exists:
- $MY_FILE
when: always
- if: $CI_PIPELINE_SOURCE == 'merge_request_event' && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'develop'
exists:
- $MY_FILE
when: always
- when: never
Note: you could write this using only one if clause and just use more parenthesis.
Let's look at these changes one by one.
First, the push and web pipeline sources are more or less identical in terms of how the pipeline is ran, and which Predefined Variables are available.
Second, the variable $CI_COMMIT_BRANCH is only available in certain pipelines. Most noticeably, it only exists when there is a Branch. If the pipeline runs for a tag, the variable will not exist, it's not just empty. If it's a merge request event, the variable will not exist. $CI_COMMIT_REF_NAME is more reliable, even though it can have more values (Commit SHAs, Branch names, or Tag names).
Third, the merge_request_event source is a totally different animal compared to the other sources we're dealing with here. The Predefined Variables available are totally different since there are now two branches (the source and the target). Also, a merge_request_event pipeline can only ever run if you have rules in your pipeline definition for merge_request_event's.
I ran some tests with this rules scenario, with this pipeline definition:
stages:
- run
Run Job:
stage: run
image: alpine:latest
script:
- echo $CI_PIPELINE_SOURCE
- echo $CI_COMMIT_REF_NAME
- echo $CI_COMMIT_BRANCH
- echo $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME
- echo $CI_MERGE_REQUEST_TARGET_BRANCH_NAME
Second Run Job:
stage: run
image: alpine:latest
rules:
- if: ($CI_PIPELINE_SOURCE == 'push' || $CI_PIPELINE_SOURCE == 'web') && $CI_COMMIT_REF_NAME == 'develop'
exists:
- 'a_file'
when: always
- if: $CI_PIPELINE_SOURCE == 'merge_request_event' && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'new_branch'
exists:
- 'a_file'
when: always
- when: never
script:
- echo $CI_PIPELINE_SOURCE
- echo $CI_COMMIT_REF_NAME
- echo $CI_COMMIT_BRANCH
- echo $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME
- echo $CI_MERGE_REQUEST_TARGET_BRANCH_NAME
Here's the job output for a normal Push event to a branch other than develop:
Executing "step_script" stage of the job script
00:02
$ echo $CI_PIPELINE_SOURCE
push
$ echo $CI_COMMIT_REF_NAME
other_branch
$ echo $CI_COMMIT_BRANCH
other_branch
$ echo $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME
$ echo $CI_MERGE_REQUEST_TARGET_BRANCH_NAME
Note: only the first job runs since the branch isn't develop. If the branch is develop and a_file exists, both jobs run, and the output is identical.
Here's the output of a job with the 'web' source, to a branch other than develop:
Executing "step_script" stage of the job script
00:00
$ echo $CI_PIPELINE_SOURCE
web
$ echo $CI_COMMIT_REF_NAME
other_branch
$ echo $CI_COMMIT_BRANCH
other_branch
$ echo $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME
$ echo $CI_MERGE_REQUEST_TARGET_BRANCH_NAME
Again, in this case there is only one job in the pipeline. If instead we hit Run Pipeline for the develop branch and a_file exists, we run both jobs and the output is identical:
Executing "step_script" stage of the job script
00:00
$ echo $CI_PIPELINE_SOURCE
web
$ echo $CI_COMMIT_REF_NAME
develop
$ echo $CI_COMMIT_BRANCH
develop
$ echo $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME
$ echo $CI_MERGE_REQUEST_TARGET_BRANCH_NAME
Lastly, if we push to a branch that's the source of a merge request targeting develop with a_file in the repository, we get the merge_request_event, but only get the second job due to the way Merge Request Pipelines work:
Executing "step_script" stage of the job script
00:01
$ echo $CI_PIPELINE_SOURCE
merge_request_event
$ echo $CI_COMMIT_REF_NAME
some_other_branch
$ echo $CI_COMMIT_BRANCH
$ echo $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME
some_other_branch
$ echo $CI_MERGE_REQUEST_TARGET_BRANCH_NAME
new_branch
Here's the documentation for Merge Request Pipelines for more information.