I have a monorepo where each package should be built as a docker image.
I created a trigger job for each package that triggers a child pipeline.
In the MR, my changes rule is being ignored and all child pipelines are triggered.
.gitlab-ci.yml
---
workflow:
rules:
- if: $CI_MERGE_REQUEST_ID || $CI_COMMIT_BRANCH
trigger-package-a:
stage: build
trigger:
include: .gitlab/ci/packages/package-gitlab-ci.yml
strategy: depend
rules:
- changes:
- "packages/package-a/**/*"
variables:
PACKAGE: package-a
trigger-package-b:
stage: build
trigger:
include: .gitlab/ci/packages/package-gitlab-ci.yml
strategy: depend
rules:
- changes:
- "packages/package-b/**/*"
variables:
PACKAGE: package-b
done_job:
stage: deploy
script:
- "echo DONE"
- "cat config.json"
stages:
- build
- deploy
package-gitlab-ci.yml
workflow:
rules:
- if: $CI_MERGE_REQUEST_ID
- changes:
- "packages/${PACKAGE}/**/*"
stages:
- bootstrap
- validate
cache:
key: "${PACKAGE}_${CI_COMMIT_REF_SLUG}"
paths:
- packages/${PACKAGE}/node_modules/
policy: pull
install-package:
stage: bootstrap
script:
- echo ${PACKAGE}}
- echo '{"package":${PACKAGE}}' > config.json
- "cd packages/${PACKAGE}/"
- yarn install --frozen-lockfile
artifacts:
paths:
- config.json
cache:
key: "${PACKAGE}_${CI_COMMIT_REF_SLUG}"
paths:
- packages/${PACKAGE}/node_modules/
policy: pull-push
lint-package:
script:
- yarn lint
stage: validate
needs: [install-package]
before_script:
- "cd packages/${PACKAGE}/"
test-package:
stage: validate
needs: [lint-package]
before_script:
- "echo working on ${PACKAGE}"
- "cd packages/${PACKAGE}/"
rules:
- if: $CI_MERGE_REQUEST_ID
script:
- yarn test
It looks like your downstream pipeline is defining a workflow with 2 independent rules: if and changes. This may cause the jobs to be included if the first condition in the if is met, i.e. if it is a MR pipeline. Try removing the dash in front of changes, as in the example here, to treat this as a single rule:
workflow:
rules:
- if: $CI_MERGE_REQUEST_ID
changes:
- "packages/${PACKAGE}/**/*"
EDIT: This recent issue states rules:changes does not work as expected with trigger. So you may actually need to remove the changes from the upstream pipeline and solve this in the downstream pipeline.
Side note, not directly related to your issue: the GitLab Docs provide a workflow template to run branch or MR pipelines without creating duplicates. You can use this in your upstream pipeline if it helps:
workflow:
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS'
when: never
- if: '$CI_COMMIT_BRANCH'
Related
My gitlab ci pipeline always blocks the terraform deploy, requiring manual action to start it. Is it possible to make it automatic instead?
From terraform gitlab yaml example
stages:
- validate
- test
- build
- deploy
- cleanup
sast:
stage: test
include:
- template: Terraform/Base.gitlab-ci.yml # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Terraform/Base.gitlab-ci.yml
fmt:
extends: .terraform:fmt
needs: []
validate:
extends: .terraform:validate
needs: []
build:
extends: .terraform:build
deploy:
extends: .terraform:deploy
dependencies:
- build
environment:
name: $TF_STATE_NAME
action: start
when: on_success
destroy:
extends: .terraform:destroy
environment:
name: $TF_STATE_NAME
action: stop
when: manual
Based on the documentation, when: on_success should automatically run the deploy command when the build stage succeeds. However, it still requires manual actions. Removing the when command is the same, it always requires a manual action to start the deploy.
Given I'm using gitlab's terraform template, is this hard coded to require manual actions to enable a deploy?
It's been a little while since I've worked on GitLab, but the template you reference has it as a rule:
.terraform:deploy: &terraform_deploy
stage: deploy
script:
- cd "${TF_ROOT}"
- gitlab-terraform apply
resource_group: ${TF_STATE_NAME}
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
when: manual
Which is different from just the when keyword that you're using.
What if you tried overriding with with your own rule?
deploy:
extends: .terraform:deploy
dependencies:
- build
environment:
name: $TF_STATE_NAME
action: start
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
when: on_success
Or better yet, just create/manage your own template from a repo of your own. Then you can modify the rules in there and delete the when: manual piece.
I am trying to setup CI in gitlab so
the second job (pushdev) will be available for running manually only after the devjob has run successfully.
the third job pushtostage will only run iff file has changed.
the way the jobs are setup, second and third jobs alway run. What is missing in the pipeline spec
devjob:
image: node:16
stage: publishdev
script:
- echo "running validation checks"
- npm run validate
rules:
- changes:
- ./src/myfile.txt
- when: manual
# - this jobs needs to run after "devjob" has run successfully
# and myfile.txt has changed
# - "needs" artifacts from the "lint" job
pushdev:
image: node:16
stage: publishdev
needs: [ "devjob", "lint"]
script:
- echo "Pushing changes after validation to dev"
- npm run pushdev
rules:
- changes:
- ./src/myfile.txt
when: on_success
- when: manual
pushtostage:
image: node:16
stage: pushstage
script:
- echo "Pushing changes to stage"
rules:
- changes:
- ./src/myfile.txt
- when: manual
I change your sample to look like this:
stages:
- publishdev
- pushstage
default:
image: ubuntu:20.04
lint:
stage: publishdev
script:
- echo "lint job"
devjob:
stage: publishdev
script:
- echo "running validation checks"
rules:
- changes:
- README.md
when: manual
allow_failure: false
pushdev:
stage: publishdev
needs: [ "devjob", "lint"]
script:
- echo "Pushing changes after validation to dev"
rules:
- changes:
- README.md
when: manual
allow_failure: false
pushtostage:
stage: pushstage
script:
- echo "Pushing changes to stage"
rules:
- changes:
- README.md
when: manual
allow_failure: false
I add allow_failure: false, because allow_failure when manual job default is true.
I merge your rules. because GitLab rules one - is one rule:
Rules are evaluated when the pipeline is created, and evaluated in order until the first match.
your .gitlab-ci.yml first job devjob is manual, so it is always a success, and your second job pushdev first rule changes and when: on_success always match, so it always run.
I change your .gitlab-ci.yml, first job devjob merge your rules when file change and set it is manual job and not allow_failure. and so on.
the sample code in Files · try-rules-stackoverflow-72594854-manual · GitLab PlayGround / Workshop / Tryci · GitLab
I want my pipelines to run:
backend_tests job for any push with changes in backend/ folder,
frontend_tests job for any push with changes in frontend/ folder,
old_code_tests job for push with changes in backend/ folder, or when MR is created and there are changes in backend folder in the source branch
My problem is that with the following .gitlab-ci.yml the old_code_tests job will run for any push to the branch with open MR, if there are already changes in the backend folder - even if push did not introduced such changes. I have no idea how to avoid this. The job is created correctly when MR is being created.
In other words: if there already is MR and branch contains backend changes and I push frontend only changes, the old_code_test job should NOT run - and it unexpectedly runs with given configuration.
I do not want to run old_code_tests job if there is no MR for given branch - and it works.
stages:
- frontend_tests
- backend_tests
- old_code
old_code_test:
extends: .test_old_code_template
stage: old_code
needs: []
script:
- echo "Test old code"
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- if: '$CI_PIPELINE_SOURCE == "push"'
when: never
- changes:
- backend/**/*
backend_tests:
stage: backend_tests
needs: []
extends: .backend_template
script:
- echo "Test backend"
rules:
- if: '$CI_PIPELINE_SOURCE == "push" && $CI_OPEN_MERGE_REQUESTS'
when: never
- changes:
- backend/**/*
frontend_tests:
stage: frontend_tests
needs: []
extends: .frontend_template
script:
- echo "Frontend test"
rules:
- if: '$CI_PIPELINE_SOURCE == "push" && $CI_OPEN_MERGE_REQUESTS'
when: never
- changes:
- frontend/**/*
Try to use only keyword in your old_code stage:
only:
changes:
- backend/**/*
I have a problem with gitlab ci child pipelines.
Need to trigger ci pipeline automatically after each commit in repo that have more than one app. Need to configure to detect which folder/files were modified in order to know which app pipeline to trigger
Example of structure
Main/
---- applicationsA/
-------- appA1/
-------- appA2/
-------- appA3/
---- applicationsB/
-------- appB1/
-------- appB2/
-------- appB3/
Main ".gitlab-ci.yml" is:
workflow:
rules:
- if: ‘$CI_PIPELINE_SOURE == “web”’
variables:
APPNAME: $APPNAME
stages:
- child-pipelines
appA1:
stage: child-pipelines
trigger:
include:
- local: applicationA/appA1/gitlab-ci.yml
strategy: depend
rules:
- if: $APPNAME == “appA1” && $CI_PIPELINE_SOURE == “web”
appA2:
stage: child-pipelines
trigger:
include:
- local: applicationA/appA2/gitlab-ci.yml
strategy: depend
rules:
- if: $APPNAME == “appA1” && $CI_PIPELINE_SOURE == “web”
...
appA1 ".gitlab-ci.yml" is:
stages:
- build
- test
build-appA1:
stage: build
script:
- echo "Execute appA1 build!"
publish-appA1:
stage: build
script:
- echo "Execute appA1 publish!"
appA2 ".gitlab-ci.yml" is:
stages:
- build
- test
build-appA2:
stage: build
script:
- echo "Execute appA1 build!"
publish-appA2:
stage: build
script:
- echo "Execute appA1 publish!"
The purpose of this configuration is that , for example, when i change a file inside app**, the pipeline detects the changes and build the app**.
You can use rules:changes with a glob pattern and only run a certain job if anything changes in the specific app folder:
appA1:
stage: child-pipelines
trigger:
include:
- local: applicationA/appA1/gitlab-ci.yml
strategy: depend
rules:
- if: '$APPNAME == "appA1" && $CI_PIPELINE_SOURE == "web"'
changes:
- Main/applicationsA/appA1/**/*
We would like to have a .gitlab-ci.yml which supports the default CI pipeline and the SAST pipeline only scheduled once a day.
lint, build, test-unit (on merge request)
test-sast (scheduled once a day)
What seems logic but didn't work is this configuration:
include:
- template: Security/SAST.gitlab-ci.yml
- template: Workflows/MergeRequest-Pipelines.gitlab-ci.yml
image: node:lts-alpine
stages:
- lint
- build
- test
lint:
stage: lint
script:
- npm i
- npm run lint
build:
stage: build
script:
- npm i
- npm run build
test-unit:
stage: test
script:
- npm i
- npm run test:unit
test-sast:
stage: test
script: [ "true" ]
rules:
- if: $CI_PIPELINE_SOURCE == "schedule"
when: always
- when: never
Then did some tests using the environment variable SAST_DISABLED which didn't work as well.
May be someone has a similiar setup and may help out with a working sample?
Your workflow:rules do not have an explicit allow for $CI_PIPELINE_SOURCE == "schedule"
This is what I use for merge request pipelines:
workflow:
rules:
# Do not start pipeline for WIP/Draft commits
- if: $CI_COMMIT_TITLE =~ /^(WIP|Draft)/i
when: never
# MergeRequest-Pipelines workflow
# For merge requests create a pipeline.
- if: $CI_MERGE_REQUEST_IID || $CI_PIPELINE_SOURCE == "merge_request_event"
# For tags, create a pipeline.
- if: $CI_COMMIT_TAG
# For default branch create a pipeline (this includes on schedules, pushes, merges, etc.).
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
# For other pipeline triggers
- if: $CI_PIPELINE_SOURCE =~ /^trigger|pipeline|web|api$/