When pushing a commit two pipeline jobs are triggered. But the same thing has not occurred when starting the pipeline manually.
Where I should check?
What is the meaning of the arrow from the left or from the right indicating branch activities?
One thing I have to say is that there is a merge request pending, does it cause this issue?
The thing with your solution is that it only avoids pipeline execution when you have a merge request event, but there still will be duplicate pipelines, e.g. merge-request pipelines (the detached ones) and branch pipelines (others), also when pushing a tag your setup will create a separate pipeline I think.
Following the docs you can avoid duplicate pipelines and switch between Branch- and MR-Pipelines when using the following rules-set for workflow (I added the || $CI_COMMIT_TAG) since when pushing a tag also a pipeline should be created (but maybe only a few jobs will be added into this pipeline)
workflow:
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS'
when: never
- if: '$CI_COMMIT_BRANCH' || '$CI_COMMIT_TAG'
this pipeline is a merge-request pipeline, you can see this because it's detached and because of the Merge-Request Symbol and the number of the MR on the left hand side of the commit id
The following screenshot shows a 'normal' branch-pipeline, which is denoted by the branch-name and the GitLab branch symbol on the left of your commit id
Pending merge request cause second pipeline jobs triggered. After adding the following into gitlab-ci.yml file it is resolved
workflow:
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
when: never
- when: always
Related
I have pipeline that makes some changes and commits another change which triggers another pipeline and I dont want that automatic update to trigger pipeline.
I had an idea that I will specify commit message and that ignore it, but for some reason I cannot get it working.
Can you help me with that?
variables:
COMMIT_MESSAGE: "MyCommitMessage"
workflow:
rules:
- if: $CI_COMMIT_MESSAGE != $COMMIT_MESSAGE
...
You have to add the never keyword and use a regex like this :
variables:
COMMIT_MESSAGE: "MyCommitMessage"
workflow:
rules:
- if: $CI_COMMIT_MESSAGE =~ /^.*COMMIT_MESSAGE/
when: never
- when: always
The if will be evaluate to true and the pipeline will never run.
I want to trigger a single job manually in GitLab CI/CD. What steps I need to follow?
I have declared environment variable as well but while triggering a pipeline manually, selecting the branch it trigger all other jobs as well.
A Gitlab pipeline is a stream of jobs, by default you can not just execute a single job in isolation.
To work around it, you can use the rules keyword https://docs.gitlab.com/ee/ci/jobs/job_control.html#specify-when-jobs-run-with-rules
Example
job1:
...
rules:
- if: '$execute_job_1 == "true"'
when: always
- when: never
job2:
...
rules:
- if: '$execute_job_2 == "true"'
when: always
- when: never
So if want to execute only job1, you pass the appropriate variable when you run the pipeline
I found the following proposal and tested it out (see code sample), but could not make it work.
We run on Gitlab 14.3.4, how can I determine if this is available for this version? If this feature is not working, how can I deploy to different environments if I have different runners one for my prod one for dev environment? So far, I have one pipeline for each environment using its dedicated tags - as dynamic tags are not available so far.
Any help would be appreciated - thanks!
workflow:
rules:
- if: '$CI_PIPELINE_SOURCE == "web"'
- if: '$CI_PIPELINE_SOURCE == "parent_pipeline"'
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- if: "$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS"
when: never
- if: '$CI_COMMIT_BRANCH =~ /^feature.*$/'
variables:
TARGET: dev
- if: "$CI_COMMIT_BRANCH"
I think you may be confusing what is and isn't available (and I think that answer you linked is too). You can absolutely use variables to populate what runner your job should run with, I use it today in workflows on the SaaS offering.
Using variables to determine the runner tag can still be confusing though, because whether or not the variables works properly is heavily dependent on where you're defining it. If your variable is within the root scope of the CI/CD pipeline (I.e., either within a top-level variable block or within a workflow block) it will work properly. If you're attempting to define the rules within the scope of a job (I.e., within a job:rules:if:variables block), it will not work properly. Since your above example is within the workflow block, it will properly select your tag and apply it to downstream jobs.
You can see more information here: https://gitlab.com/gitlab-org/gitlab/-/issues/35742#note_704836498
Here is an example proving out the dynamic tag:
workflow:
rules:
- variables:
RUNNER: shared-macos-amd64
test:
image: alpine:latest
tags:
- $RUNNER
script:
- echo $CI_RUNNER_DESCRIPTION
This properly picks up on the macos runner (and prints an error since I'm not in their beta):
Updating on the answer of #Patrick I made it work and want to describe my solution. My problem was caused by the variables used—as I tested using a PR I never had CI_COMMIT_BRANCH set, but using CI_MERGE_REQUEST_SOURCE_BRANCH_NAME it worked.
workflow:
rules:
# No build duplicates when committing to a branch with open pull request (PR)
- if: "$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS"
when: never
# Deploy to development when PR from feature-branch
- if: "$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME =~ /^feature.*$/"
variables:
TARGET: dev
# Deploy to test when committing (after merging) to develop-branch
- if: '$CI_COMMIT_BRANCH == "dev"'
variables:
TARGET: test
# Deploy to staging when committing (after merging) to release-branch
- if: '$CI_COMMIT_BRANCH =~ /^release\/.*$/'
variables:
TARGET: stage
# Deploy to production when committing (after merging) to master-branch
- if: '$CI_COMMIT_BRANCH == "main"'
variables:
TARGET: prod
- if: '$CI_PIPELINE_SOURCE == "web"'
- if: '$CI_PIPELINE_SOURCE == "parent_pipeline"'
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- if: "$CI_COMMIT_BRANCH"
The solution lays in the used predefined variables.
CI_COMMIT_REF_NAME - The branch or tag name for which project is built.
CI_COMMIT_BRANCH - The commit branch name. Available in branch pipelines, including pipelines for the default branch. Not available in merge request pipelines or tag pipelines.
CI_COMMIT_REF_NAME - Unclear
CI_MERGE_REQUEST_SOURCE_BRANCH_NAME - The source branch name of the merge request.
CI_MERGE_REQUEST_TARGET_BRANCH_NAME - The target branch name of the merge request.
CI_OPEN_MERGE_REQUESTS - A comma-separated list of up to four merge requests that use the current branch and project as the merge request source. Only available in branch and merge request pipelines if the branch has an associated merge request. For example, gitlab-org/gitlab!333,gitlab-org/gitlab-foss!11.
In the gitlab documentation you find a list of predefined variables HERE, where the variable CI_PIPELINE_SOURCE is explained to have the possible values "push, web, schedule, api, external, chat, webide, merge_request_event, external_pull_request_event, parent_pipeline, trigger, or pipeline."
However, it is not explained, what they mean.
push: When you push something to a branch?
web: When you trigger a pipeline from the web GUI?
schedule: When a pipeline is triggered by a schedule
api: When the pipeline is triggered by an API request
external: ???
chat: ???
webide: ???
merge_request_event: Seems to be triggered when a merge request is created. Does not trigger when a change is actually merged
external_pull_request_event: ???
parent_pipeline: ???
trigger: ???
pipeline: another pipeline?
If someone knows where the documentation for that is hiding, I appreciate if you can let me know where to find it.
In addition, how can I figure out when some changes are actually merged into a branch? How can I trigger a pipeline in that event?
Regarding your first set of questions, i have to point you forward to the gitlab CI Documentation and the rules:if section. They have their a good explanation of the states and also some addtion https://docs.gitlab.com/ee/ci/jobs/job_control.html#common-if-clauses-for-rules - i am just screenshoting this, so people can relate to it in the future if the link gets outdated:
Regarding your additional question:
A merge is a push. We do not check on some branches for CI_PIPELINE_SOURCE but for the branch name and do checks simply against that like:
rules:
- if: '$CI_COMMIT_BRANCH == "master"'
- if: '$CI_COMMIT_BRANCH == "develop"'
- if: '$CI_COMMIT_BRANCH =~ /^release.*$/i'
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
This works eg perfectly in our case for gitflow. But you can vary your rules and easily define them to your own needs - the rules documentation gives a lot of good examples see: https://docs.gitlab.com/ee/ci/jobs/job_control.html#common-if-clauses-for-rules
I would like to run one or multiple jobs within a scheduled context and therefore used the specific rule to declare it like this.
Problem is that the pipeline will neither be triggered by my schedule configuration nor when i manually trigger it via the scheduling pipeline UI. I just don't see any triggered scheduled pipeline at all.
Gitlab Version: 12.9.2
gitlab-ci.yml (partially):
workflow:
rules:
- if: $CI_COMMIT_TAG
- if: $CI_COMMIT_BRANCH
non-scheduled-job:
...
rules:
- if: '$CI_PIPELINE_SOURCE != "schedule"'
scheduled-job:
...
rules:
- if: '$CI_PIPELINE_SOURCE == "schedule"'
when: always
- if: '$CI_PIPELINE_SOURCE != "schedule"'
when: never
I know that the second rule for the scheduling-job is not needed but even without this the pipeline is not properly running.
Unfortunately, as for Workflows logic, Gitlab is very twiggy.
If you use Workflow section and Rules conditions for your jobs, you have to obviously declare 'Scheduled' type of pipeline in the Workflow section to enable it in your configuration:
workflow:
rules:
- if: '$CI_PIPELINE_SOURCE == "schedule"'
...
Or you may enable all types of pipelines in your configuration just adding at the end of workflow:rules
workflow:
rules:
...
- when: always
In other way, use may use except/only notation for scheduled jobs and it will work, but only if your workflow conditions met:
only:
- schedules
This approach may appear merging conflicts if you use include option.
Because merging lets you extend and override dictionary mappings, but you cannot add or modify items to an included array. Thus you have to obviously declare all previously added workflow:rules array items in the last included YAML or use anchors.
See issue for details.
You can use only.
E.g.
scheduled-job:
only:
- schedules
https://docs.gitlab.com/ee/ci/pipelines/schedules.html
But, when reusing the snippet in other jobs,
do not use rules togetther with only / except.