I'd like to add a workflow rule to allow the pipeline to run when it is a "web pipeline" (created by using Run pipeline button in the GitLab UI) and that the selected branch is main. So my first thought was to write:
if: $CI_PIPELINE_SOURCE == "web" && $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
However, when reading the Gitlab doc, I am given to understand that $CI_COMMIT_BRANCH is only available in "branch pipelines", see here:
CI_COMMIT_BRANCH: The commit branch name. Available in branch pipelines, including pipelines for the default branch
The doc seems to define "branch pipelines" as pipelines that run when a push occurs in a branch, see here and there:
You can configure your pipeline to run every time you commit changes to a branch. This type of pipeline is called a branch pipeline.
Branch pipelines that run for Git push events to a branch, like new commits or tags.
Example: if: $CI_COMMIT_BRANCH == "main": If changes are pushed to main.
(Note however that "branch pipelines" are not a type of pipeline in $CI_PIPELINE_SOURCE (see here: push, web, schedule, api, external, chat, webide, merge_request_event, external_pull_request_event, parent_pipeline, trigger, or pipeline).)
Therefore, after carefully reading the doc, I can only conclude that "web" pipelines are not branch pipelines, since no commit or push was responsible for starting the pipeline, and therefore CI_COMMIT_BRANCH will not available. So I started looking for another predefined variable that tells me the current branch name (unsuccessfully).
However, to my surprise, I can see by experimenting that CI_COMMIT_BRANCH is in fact available in web pipelines! Isn't this incredibly confusing? So are web pipelines "branch pipelines" after all? Why? Which other types of $CI_PIPELINE_SOURCE are regarded as "branch pipelines"?
You’re comparing terms which are not mutually exclusive.
See the predefined variables reference.
The following are mutually exclusive variables:
CI_COMMIT_BRANCH - The commit branch name. Available in branch pipelines, including pipelines for the default branch. Not available in merge request pipelines or tag pipelines.
CI_COMMIT_TAG - The commit tag name. Available only in pipelines for tags.
All variables specific to merge request pipelines
This is totally separate from the action which triggered the build:
CI_PIPELINE_SOURCE
How the pipeline was triggered. Can be push, web, schedule, api,
external, chat, webide, merge_request_event,
external_pull_request_event, parent_pipeline, trigger, or pipeline.
For a description of each value, see Common if clauses for rules,
which uses this variable to control when jobs run.
If you trigger a build on main from the web gui, it doesn’t stop being on main. It just wasn’t triggered by a push event. But GitLab is still running the pipeline with your repo at a specific commit on a specific branch. If you manually trigger a tag build from the web or if you schedule a tag build, the source would reflect that (web or schedule) and CI_COMMIT_BRANCH would be unset, and CI_COMMIT_TAG would be set.
Related
I want to send a slack notification as soon as trigger the pipeline build manually.
Steps I am following:
Go to your project’s CI/CD > Pipelines and select Run pipeline.
Choose the branch you want to run the pipeline for.
Input the variable and its value in the UI.
Click on the Run pipeline button
From the documentation I see:
How the pipeline was triggered. Can be push, web, schedule, api, external, chat, webide, merge_request_event, external_pull_request_event, parent_pipeline, trigger, or pipeline. For a description of each value, see Common if clauses for rules, which uses this variable to control when jobs run.
In this init stage I want to execute a script only when I run the pipeline using the above steps
init:
stage: init
environment: $DEPLOY_ENVIRONMENT
script:
- bundle exec fastlane slack_build_start_summary
rules:
- if: $CI_PIPELINE_SOURCE == "trigger" // also tried with "pipeline"
when: always
But I can't see this stage in the Gitlab pipeline stages.
I want to know which CI_PIPELINE_SOURCE ("trigger" or "pipeline" or something else?) I need to check when I trigger the pipeline manually from Gitlab using the above steps.
I got the answer here from the documentation
web -> For pipelines created by using Run pipeline button in the GitLab UI, from the project’s CI/CD > Pipelines section.
So I need to use:
- if: $CI_PIPELINE_SOURCE == "web"
I have created a simple readme file in the main repository. Now for example if someone creates a feature branch out of the main branch and edits the readme file. Those commits will be then committed in the feature branch.
Now I want to create a pipeline which should do the following:-
Whenever there is a push in the repo it should run the pipeline automatically.
The pipeline should check that every NEW content to the README.md has to have at least a one time mentioning of words like "ABQ" OR "abq" OR "albuquerque" OR "Albuquerque".
So basically The push based pipeline shall check if the added text in the feature branch does at least have one occurrence of the above mentioned words.
Here is the yml file i have that runs whenever there is a push. Can someone guide me here for achieving the above goal?
workflow:
rules:
- if: $CI_PIPELINE_SOURCE == "push"
when: always
We're running a self-hosted GitLab CE instance. I'm trying to construct a very particular ruleset that works in our project environment. I'm aware of the basic building blocks: branch pipelines and MR pipelines, workflow:rules for pipelines, rules for jobs, predefined $CI_* variables, etc. I'm having trouble bringing them all together in a way that would also avoid duplicate pipelines, a common problem that may require very specific rule definitions to overcome.
We have a main branch from which feature branches are created, and merged back via merge requests. Occasionally some trivial fixes are pushed directly to main.
We've defined a pipeline with three jobs, currently all set to when: manual until we figure out this scheme. The eventual goal is this:
pre-build tests (stage: .pre)
Always runs as the first job, when the pipeline runs.
Has allow_failure: false – if the tests don't pass, no further jobs should run.
build (stage: build)
Conditions detailed below.
deploy to production (stage: deploy)
Should only be run manually in all situations.
Has dependencies: [build]
The conditions we'd like to have for the creation of the entire pipeline:
A merge request for a feature branch is merged to main: run the pipeline for main.
Commits are pushed directly to main: run the pipeline for main.
(Side note: we want to prevent duplicate pipelines from the above two conditions.)
Commits are pushed to a new or existing feature branch: run the pipeline for the feature branch. Does not matter if it has an open MR or not.
All other cases (tags, new MR creation from/to any branch, etc.): DON'T run the pipeline.
If the pipeline runs, a couple of extra conditions for the build job specifically:
The pipeline is running for the main branch (via a push or an MR, doesn't matter): always build, assuming the tests passed.
The pipeline is running for a feature branch: must build manually.
Achieving this setup is probably not THAT many rules in total, but I'd like to not have to do hours of trial and error if this is something that an expert can readily help with.
I eventually figured it out. Here's how the above conditions are met, just in case they're of use to someone else down the line:
workflow:
rules:
# Explicitly enable merge request pipelines to be created (not done by default)
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
# Prevent duplicate pipelines from push events to branches that are the source
# branch of one or more open merge requests. If $CI_OPEN_MERGE_REQUESTS is
# non-empty on a branch pipeline, it means that the the above rule has already
# caused (or will cause) a merge request pipeline to run. These rules are
# separately evaluated for the merge request event resulting from the same push.
- if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS
when: never
# Run a normal branch pipeline on a branch that isn't the source of an open MR
- if: $CI_COMMIT_BRANCH
pre-build tests:
stage: .pre
when: always
allow_failure: false
# etc.
build:
stage: build
rules:
# Automatically done assuming tests pass, but only on the default branch.
# Also runs when a feature branch MR is merged to the default branch; the
# pipeline source for that is again `push`, not `merge_request_event`.
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
when: on_success
- when: manual
# etc.
deploy to production:
stage: deploy
when: manual
dependencies: [build]
# etc.
The one part of the question that I couldn't do is this one:
new MR creation from/to any branch: DON'T run the pipeline
...while also satisfying this one (much more important):
Commits are pushed to a new or existing feature branch: run the pipeline for the feature branch.
A merge request pipeline is run when a merge request is created for an existing branch. When that branch was first pushed, a branch pipeline was run for a certain $CI_COMMIT_SHA. When the MR creation event pipeline runs, it builds for that same SHA. (However, this doesn't constitute a "duplicate pipeline" as described, because the pipelines are created by two different user actions—unlike a single push to an MR source branch without the $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS trick.)
There's no official way of preventing this; a related GitLab issue can be found here, open as of this writing. Thankfully, the practical impact of this is negligible in our case.
I want a certain job to trigger when a tag is pushed to main branch
I have the following gitlab ci config
job1:
rules:
- if: ($CI_COMMIT_TAG =~ /(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)/ && $CI_COMMIT_BRANCH == "main")
I usually do the following in git
git checkout main
git merge develop
git tag "1.0.0"
git push --atomic origin main "1.0.0"
However it is not triggering, what am I doing wrong?
There are different types of pipelines that can be triggered in GitLab. What pre-defined variables are available will differ depending on the type of pipeline. As per the documentation:
CI_COMMIT_TAG is "available only in pipelines for tags."
CI_COMMIT_BRANCH is "available in branch pipelines."
Since your pipeline is not going to be a tag pipeline and branch pipeline at the same time, your condition will never evaluate to true.
I suggest only using the first part where you've specified the formatting for the tag.
I have created a pipeline in my repository which is used to validate code by executing unit tests for code that is being pushed to features/* branches. The same pipeline is used as Build validation pipeline set as Branch Policy on the develop branch to validate incoming PRs. This is the trigger of the pipeline.
# pipeline.yml
trigger:
batch: false
branches:
include:
- features/*
However we have come across the following condition: Given an open PR from refs/heads/features/azure-pipelines -> refs/heads/develop we push a commit on the features/azure-pipelines branch.
This causes the pipeline to trigger twice. To my understanding one of the runs is due to the trigger of the pipeline (The one marked as Individual CI on the screenshot) and the second run is due to branch policy trying to validate code being pushed onto the open PR to develop. (The PR Automated)
Is there any way to disable one of the executions since it's essentially a duplicate? I was maybe looking for a way to retrieve open PRs and abort execution of a pipeline for Individual CI if there is an open PR for a branch but I am not sure that's the best way around that and I am looking for options.
You can set
trigger: none
This way only the branch policy will trigger the pipeline.
Is there any way to disable one of the executions since it's essentially a duplicate?
As we know, we could not disable the Build validation pipeline set as Branch Policy on the develop branch to validate incoming PRs unless we cancel the Build validation.
For your situation, you could try to include [skip ci] in the commit message or description of the HEAD commit to make the Azure Pipelines skip running CI when you plan to merge the features branch to the develop branch.
You could check the document Skipping CI for individual commits for some more details.
Here it depends if they does the same. You can have conditional checks in the pipeline which does a different things for PR and CI runs. However, I'm pretty sure that this is not possible, because one is defined on the YAML and the second on the Azure DevOps portal. So even if you disnle PR trigger here in YAML, a branch policy still runs a PR. And you can specify antyhing in YAML to block branch policy.