I am using below to trigger pipeline if any changes are made in the folders under root directory.
trigger_serviceA:
stage: triggers
rules:
- if: '$CI_COMMIT_BRANCH == "dev"'
changes:
- serviceA/*
when: always
trigger:
include: serviceA/.gitlab-ci.yml
strategy: depend
However, pipeline is not getting triggered if there are any changes in the subfolders under serviceA.
When using only:changes the single wildcard does not covers subdirs - you'll have to use /**/* instead.
Altough the documentation does not say it, i could imagine that it's the same for rules:changes - so maybe you'll want to try that out.
Related
I have a gitlab repo which has two folders(folder-a and folder-b) in it, I want them to get triggered based on the commit on their folder. Currently I have a single gitlab.yml file which has below configuration. It triggers folder-a but not folder-b.
folder-a:
trigger:
include:
- local: folder-a/.gitlab-ci.yml
rules:
- changes: [folder-a/*]
folder-b:
trigger:
include:
- local: folder-b/.gitlab-ci.yml
rules:
- changes: [folder-b/*]
Sorry I had the issue with folder-b .gitlab-ci yml file. this works fine, now able to trigger folder based on commits to the branches
I am working with GitLab 14.6.2-ee.
I am trying to use the YAML reference feature to re-use code snippets in my GitLab configuration files. I've boiled it down to what I think is the simplest use case, but cannot get the file to validate.
.gitlab-ci.yaml:
...
.hidden:
.update-static-site:
rules:
if: $CI_COMMIT_BRANCH =~ /(^develop$|^master$|^staging-.*)/
...
trigger-downstream-pipeline:
stage: .post
trigger:
project: project-x/project-x-manifest
branch: master
variables:
SOURCE_COMMIT_SHA: $CI_COMMIT_SHA
SOURCE_COMMIT_REF_NAME: $CI_COMMIT_REF_NAME
APP_NAME: project-x
needs:
- job: push-project-x-to-harbor
artifacts: false
rules:
- !reference [.hidden, .update-static-site, rules]
I cannot get the !reference line to validate. I keep getting Unresolved tag: !reference.
I found a SO posting (GitLab CI - Reuse rules with !reference tag) that addresses exactly what I'm trying to do, but it does not work.
So, the main problem was that I needed to change my VS Code config to accept the YAML reference tags. Once that was done, no more validation errors.
When I committed the changes and watched the pipeline, there were no issues.
I have a GitLab-CI pipeline in place with my Katalon Studio automation tests that I would like to have the following functionality:
Various nightly schedules that run based on scheduled variables being present.
Changes declaration so pushes to the repo only trigger a pipeline run if certain files have been touched.
I have the scheduled portion running as expected, but I am struggling on pairing that with the 'changes' declaration to only run the pipeline IF someone pushes after changing certain files. Can someone help? I am guessing this is an issue with my YAML formatting.
Here is an example snippet from my current GitLab-CI.yaml
Example Tests:
stage: Example
tags:
- aws-medium-runner
script:
- MY_SCRIPT
rules:
- if: $SCHEDULE_A == "true" # tied to schedule A in scheduler tool
when: always
- if: '$CI_PIPELINE_SOURCE == "push"'
changes: # Only run on pushes if changes have been made to certain directories
- Test\ Cases/Example/*
- Object\ Repository/Example/*
- Test\ Suites/Example/*
- Scripts/Example/*
when: always
dependencies:
- Set Release Version
You are missing 2 spaces on the changes: part.
It should work when you add those. (see the example below)
And in case you want to combine the 2 rules with the same conditions, you can combine them into one rule:
Example Tests:
stage: Example
tags:
- aws-medium-runner
script:
- MY_SCRIPT
rules:
- if: '$CI_PIPELINE_SOURCE == "push" || $SCHEDULE_A == "true"'
changes: # Only run on pushes if changes have been made to certain directories
- Test Cases/Example/*
- Object Repository/Example/*
- Test Suites/Example/*
- Scripts/Example/*
when: always
dependencies:
- Set Release Version
I am trying to include an external GitLab CI YAML file into my project local .gitlab-ci.yml file. The external YAML, which is in my other GitLab project ci/cd > templates contains some workflow rules:
# ci-cd.yml
workflow:
rules:
- if: '$TRACK != null' # TRACK is the environment type (staging/production)
when: always
- if: '$CI_PIPELINE_SOURCE =~ /^trigger|pipeline|web|api$/'
when: always
Below is my project local .gitlab-ci.yml:
include:
- '/.gitlab-ci-test.yml'
- project: 'ci-cd/templates'
file: 'ci-cd.yml'
...
The problem is - none of the jobs I have defined inside locally included .gitlab-ci-test.yml get triggered when I push the changes to GitLab, even when the job has when: always set. Seems like the workflow rules in external ci-cd.yml are not letting the jobs run.
I've also tried to locally add a workflow rule to .gitlab-ci.yml that evaluates to true, because the GitLab workflow keyword docs say
When no rules evaluate to true, the pipeline does not run.
That means if any one of the rules evaluates to true, pipeline should have run, which did not happen when I added a local workflow rule.
EDIT - the external file which has workflow rules is used by many projects so this can't be modified to have "push" in $CI_PIPELINE_SOURCE. My intention is to let it be as a global rule and try to 'override' locally in my project.
I hope I was clear in issue description. Would appreciate any help!
You are missing push event for $CI_PIPELINE_SOURCE in you workflow rule.
workflow:
rules:
- if: '$TRACK != null' # TRACK is the environment type (staging/production)
when: always
- if: '$CI_PIPELINE_SOURCE =~ /^trigger|pipeline|push|web|api$/'
when: always
EDIT: if you are not able to change the workflow rule in the included file, the only option I see is to duplicate the workflow in your gitlab-ci.yml and add the missed push there.
workflow rules take precedence before all other rules and it is not possible to merge two workflow blocks. If a worfklow block is used in an included yml file and the gitlab-ci.yml itself, the workflow from the gitlab-ci.yml is used. You can check this in CI/CD -> Editor -> View merged YAML in gitlab.
include:
- '/.gitlab-ci-test.yml'
- project: 'ci-cd/templates'
file: 'ci-cd.yml'
workflow:
rules:
- if: '$TRACK != null' # TRACK is the environment type (staging/production)
when: always
- if: '$CI_PIPELINE_SOURCE =~ /^trigger|pipeline|push|web|api$/'
when: always
...
I am trying to set up a CI with minimal code duplication using .gitlab-ci.yml.
With that, I am separating the configuration in separate files and reusing parts of it that are common.
I have a separate repository with Gitlab CI settings: gitlab-ci and several projects that use it to form their own CI pipelines.
Contents of gitlab-ci repository
template_jobs.yml:
.sample:
rules:
- if: '$CI_PIPELINE_SOURCE == "push"'
when: on_success
- when: never
jobs_architectureA.yml:
include:
- local: '/template_jobs.yml'
.script_core: &script_core
- echo "Running stage"
test_archA:
extends:
- .sample
stage: test
tags:
- architectureA
script:
- *script_core
jobs_architectureB.yml:
include:
- local: '/template_jobs.yml'
.script_core: &script_core
- echo "Running stage"
test_archB:
extends:
- .sample
stage: test
tags:
- architectureB
script:
- *script_core
Project with code contents:
In the actual project (separate repositories per project, and I have a lot of them), I have the following:
.gitlab-ci.yml:
stages:
- test
include:
- project: 'gitlab-ci'
file: '/jobs_architectureA.yml'
- project: 'gitlab-ci'
file: '/jobs_architectureB.yml'
This configuration works fine and allows to include only some architectures for some modules while sharing rules between the job templates.
However, it's easy to notice one code duplication: both jobs_architectureA.yml and jobs_architectureB.yml contain a common section:
.script_core: &script_core
- echo "Running stage"
It would be ideal to move it into a separate file: template_scripts.yml and include from both jobs_architectureA.yml* and jobs_architectureB.yml. However, that results in the invalid YAML (at least from Gitlab's point of view).
With that, I make a conclusion that I can share the rules as the mechanism of their usage is via extends keyword; however, I am not able to do it with the scripts: as it uses &/* anchoring mechanic on the YAML level.
Ideally, I want something along the lines of:
Contents of the ideal (conceptually) gitlab-ci repository
template_jobs.yml:
.sample:
rules:
- if: '$CI_PIPELINE_SOURCE == "push"'
when: on_success
- when: never
template_scripts.yml:
.script_core: &script_core
- echo "Running stage"
jobs_architectureA.yml:
include:
- local: '/template_jobs.yml'
- local: '/template_scripts.yml'
test_archA:
extends:
- .sample
stage: test
tags:
- architectureA
script:
- *script_core # this becomes invalid, as script_core is in the other file, even though it is included at the top
jobs_architectureB.yml:
include:
- local: '/template_jobs.yml'
- local: '/template_scripts.yml'
test_archB:
extends:
- .sample
stage: test
tags:
- architectureB
script:
- *script_core # this becomes invalid, as script_core is in the other file, even though it is included at the top
Am I doing something wrong?
Am I hitting a limitation of the Gitlab mechanic? Is it the implementation of the include directive in this specific YML type, that limits me?
Do I have options to achieve something close to the desired behaviour?
Note, while this might not look like a big deal, in reality, I have many more pieces to the scripts, and the actual script is much larger. Thus, currently, it is duplicated code all over the place which is very prone to mistakes.
my solution is to not include template_jobs.yml and template_scripts.yml directly in jobs_architectureA.yml but only in the "final" .gitlab-ci.yml
taking you exemple, /template_jobs.yml//template_scripts.yml do not change.
jobs_architectureA.yml loses the include:
test_archA:
extends:
- .sample
stage: test
tags:
- architectureB
script:
- *script_core # this becomes invalid, as script_core is in the other file, even though it is included at the top
and .gitlab-ci.yml becomes:
stages:
- test
include:
- local: '/template_jobs.yml'
- local: '/template_scripts.yml'
- project: 'gitlab-ci'
file: '/jobs_architectureA.yml'
- project: 'gitlab-ci'
file: '/jobs_architectureB.yml'
in reality, I have many more pieces to the scripts, and the actual script is much larger
Adding to Cyril's solution, GitLab 13.12 (May 2021) can help scale those includes:
Support wildcards when including YAML CI/CD configuration files
The includes: keyword for CI/CD pipelines lets you break one long .gitlab-ci.yml file into multiple smaller files to increase readability.
It also makes it easier to reuse configuration in multiple places.
Frequently there are multiple files included into a single pipeline, and they all might be stored in the same place.
In this release, we add support to use the * wildcard with the local includes: keyword. You can now make your includes: sections more dynamic, less verbose, and easier to read, check out how we are dogfooding it in GitLab.
See Documentation and Issue.
The extends: keyword can be used with multiple jobs to extend from (on recent versions of GitLab). You can simply do the following:
include:
- local: '/template_jobs.yml'
.script_core:
- echo "Running stage"
test_archA:
extends:
- .sample
- .script_core
stage: test
tags:
- architectureA
And after that move the .script_core to an import.