I have a parent pipeline as follows:
workflow:
rules:
- if: $CI_PIPELINE_SOURCE == "web"
- if: $CI_MERGE_REQUEST_IID
- if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS
when: never
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
- if: $CI_COMMIT_BRANCH == "master"
stages:
- triggers
deidentify:
stage: triggers
trigger:
include: microservice_a/.gitlab-ci.a.yml
strategy: depend
only:
changes:
- microservice_a/*
ebridge_export:
stage: triggers
trigger:
include: microservice_b/.gitlab-ci.b.yml
strategy: depend
only:
changes:
- microservice_b/*
If I push any changes under microservice_a/ e.g. microservice_a/.gitlab-ci.a.yml it NEVER starts any pipeline, but if I change
only:
changes:
- microservice_a/*
To
rules:
- changes:
- microservice_a/*
On both, it will always run both regardless of whether the change was actually under microservice_a or microservice_b.
Now, the child pipelines both have a small test job that looks like this:
stages:
- test
test:
stage: test
rules:
- when: always
If I remove the rules section on both (while using rules in the parent pipeline), both will fail because it couldn't find any jobs to add to the pipeline
What am I doing wrong?
Related
I'm writing a GitLab CI workflow in order to be re-used accross by project, using Docker for build and deployment.
On each project, I would like to only specify the needed changes to trigger this workflow under a merge request like so:
# .gitlab-ci.yml
include:
- { project: internal, ref: main, file: workflow.gitlab-ci.yml }
.workflow:changes:
- Dockerfile
- docker-compose.yml
- 'package*.json'
I was able to achieve it with the following:
# workflow.gitlab-ci.yml
build:
stage: build
script: docker build
rules:
- if: $CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_TAG =~ /^v[0-9]+\.[0-9]+\.[0-9]+$/
- if: $CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
changes: !reference [.workflow:changes]
test:
stage: test
script: docker test
rules:
- if: $CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
changes: !reference [.workflow:changes]
However, I also want to store my repeated rules conditions onto configuration I can reference to them like described on the official documentation:
# workflow.gitlab-ci.yml
.rules:production:
rules:
- if: $CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_TAG =~ /^v[0-9]+\.[0-9]+\.[0-9]+$/
.rules:staging:
rules:
- if: $CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
.rules:review:
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
changes: !reference [.workflow:changes]
build:
stage: build
script: docker build
rules:
- !reference [.rules:production, rules]
- !reference [.rules:staging, rules]
- !reference [.rules:review, rules]
test:
stage: test
script: docker test
rules:
- !reference [.rules:staging, rules]
- !reference [.rules:review, rules]
The .rules:review section has the exact same configuration as when it was placed under the build and test section of the first example. However, it give me the following invalid configuration error:
Unable to create pipeline
jobs:build:rules:rule:changes should be an array or a hash
Is that a GitLab CI limitation of the !reference keyword? Is there any workaround?
Note: The other .rules:production and .rules:staging configuration block are working properly.
I'm trying to use changes block inside rules block in the downstream pipeline:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
changes:
- "projects/testing/**"
When doing it like this the job is not created. If i remove the changes part and just go with:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
It works just fine. I dont know if i'm missing something here.
Edit. I managed to get it working in the MR pipelines with this code:
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
changes:
- "projects/demo-app/**"
- "*.tf"
when: on_success
- if: $CI_MERGE_REQUEST_ID
changes:
- "projects/demo-app/**"
- "*.tf"
when: on_success
- when: never
But when change is merged it doesn't trigger the pipeline anymore.
Hello I'm trying to figure out how to run SAST jobs only when merging branch into master because they last 5 minutes and they are being run in every push for any branch.
This means that every time someone makes a push to their MR branch the security stage is executed with all SAST jobs.
What I want to achieve is that SAST jobs are executed when the branch is going to be merged to master.
gitlab-ci.yml:
include:
- template: Jobs/SAST.gitlab-ci.yml
stages:
- security
- tests
my_tests:
stage: tests
script:
- echo Running tests ...
sast:
stage: security
What I tried so far is using:
sast:
stage: security
only:
- master
But it fails because the included template Jobs/SAST.gitlab-ci.yml already uses rules and rules with only/except can't be used together.
jobs:sast config key may not be used with rules: only
In the sourcecode Jobs/SAST.gitlab-ci.yml does not use except but rules which also are incompatible with only.
But you could also switch to rules syntax:
sast:
stage: security
rules:
- if: "$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH"
when: always
That should do the trick
You can use this configuration:
include:
- template: Jobs/SAST.gitlab-ci.yml
phpcs-security-audit-sast:
rules:
- if: $CI_PIPELINE_SOURCE == 'merge_request_event'
exists:
- '**/*.php'
semgrep-sast:
rules:
- if: $CI_PIPELINE_SOURCE == 'merge_request_event'
exists:
- '**/*.go'
- '**/*.html'
- '**/*.js'
- '**/*.jsx'
- '**/*.ts'
- '**/*.tsx'
gosec-sast:
rules:
- if: $CI_PIPELINE_SOURCE == 'merge_request_event'
exists:
- '**/*.go'
nodejs-scan-sast:
rules:
- if: $CI_PIPELINE_SOURCE == 'merge_request_event'
exists:
- '**/package.json'
bandit-sast:
rules:
- if: $CI_PIPELINE_SOURCE == 'merge_request_event'
exists:
- '**/*.py'
flawfinder-sast:
rules:
- if: $CI_PIPELINE_SOURCE == 'merge_request_event'
exists:
- '**/*.c'
- '**/*.cpp'
eslint-sast:
rules:
- if: $CI_PIPELINE_SOURCE == 'merge_request_event'
exists:
- '**/*.html'
- '**/*.js'
- '**/*.jsx'
- '**/*.ts'
- '**/*.tsx'
spotbugs-sast:
rules:
- if: $CI_PIPELINE_SOURCE == 'merge_request_event'
exists:
- '**/*.groovy'
- '**/*.java'
- '**/*.scala'
- '**/*.kt'
workflow:
rules:
- if : '$CI_COMMIT_BRANCH == "Sprint-Release-Branch"'
when: never
- if : '$CI_PIPELINE_SOURCE == "merge_request_event" && $CI_PIPELINE_SOURCE == "push"'
when: never
- when: always
stages:
- Stage1
- Stage2
- Stage3
Task1:
stage: Stage1
script:
- echo "Stage1"
rules:
- if: '$CI_COMMIT_TAG =~ /^\d+\.\d+\.DEV\d+/'
tags:
- Runner
Task2:
stage: Stage1
script:
- echo "Checking code standard as per "Coding Standards""
rules:
- if: '$CI_COMMIT_TAG =~ /^\d+\.\d+\.DEV\d+/'
allow_failure: true
tags:
- Runner
Task3:
stage: Stage2
script:
- echo "Stage2"
when: manual
tags:
- Runner
Task4:
stage: Stage3
script:
- echo "Stage3"
when: manual
tags:
- Runner
Above is my Gitlab CI file, where i am trying
pipeline should not add jobs when there is merge & push requests happened on "Sprint-Release-Branch"
but whenever any merge request done "feature branches" onto "Sprint-Release-Branch" jobs which are defined as "when: manual" get added in pipeline.
So in my situation, Dev team is creating different feature branch for different user-stories, and then merging those features branches onto Sprint-Release-Branch having above yml file. So multiple jobs are getting added in pipeline continuously for every merge_request which are defined with "manual" trigger
How can i make optimized my yml so that jobs having manual trigger should not get added in pipeline.
which are defined as "when: manual" get added in pipeline.
You have to repeat the whole logic when you overwrite it.
Task3:
...
rules:
- if : '$CI_COMMIT_BRANCH == "Sprint-Release-Branch"'
when: never
- if : '$CI_PIPELINE_SOURCE == "merge_request_event" && $CI_PIPELINE_SOURCE == "push"'
when: never
- when: manual
Also, it's better to use when: on_success, not always.
Do something like the following with yaml anchors:
.myrules: &myrules
if: $CI_COMMIT_BRANCH == "Sprint-Release-Branch" || ($CI_PIPELINE_SOURCE == "merge_request_event" && $CI_PIPELINE_SOURCE == "push")
when: never
workflow:
rules:
- *myrules
- when: on_success
Task3:
...
rules:
- *myrules
- when: manual
I'm using rule method in my stage. But with rule method, I don't want to trigger CI run for tags.
I need to find a way to avoid tags like except: - tags. Unfortunately except not working with rules. In shortly how can I avoid it?
I use this snippet in my stage but it did not work, it still triggers the pipeline after the creation of the tag.
- if: '$CI_COMMIT_TAG'
when: never
This is my main deployment stage:
deploy:
stage: deployment
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event" && $sample == "one"'
when: manual
- if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS'
when: never
- if: '$CI_COMMIT_TAG'
when: never
- if: '$CI_COMMIT_BRANCH && $sample == "one"'
when: manual
script:
- ...
- ...
I resolved the problem. Just I added - when: never at the end of the rules. As a separate rule, it will block all different trigger cases.
deploy:
stage: deployment
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event" && $sync == "one"'
when: manual
- if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS'
when: never
- if: '$CI_COMMIT_TAG'
when: never
- if: '$CI_COMMIT_BRANCH && $sync == "false"'
when: manual
- when: never
script:
- ...
- ...