I am trying to not to start a pipeline when only specific files are changed.
In case, I change the README.md, README.adoc, .gitignore file. The Pipeline must not run.
I can not make it happen correctly yet.
image: alpine
stages:
- test
workflow:
rules:
- changes:
- "**/*.md"
- "**/*.adoc"
- "**/.gitignore"
when: never
- when: always
test :
script:
- "echo Hello Pipeline"
tags:
- dev
My Test Cases are
Push abc.java and abc.python. -> should run Pipeline : success
Push abc.java file and README.md file -> should run pipeline : But it doesn't. Because the *.md file is in push
Create a Branch -> should run pipeline, but it does't run. Because the *.md Files are in the source branch
Merge a Branch -> should run pipeline, but it does't run. Because the *.md Files are in the source branch.
Create a Tag -> should run pipeline, but it does't run. Because the *.md Files are in the source branch.
has anyone succeed?
I like to make a blacklist of file extension to stop pipeline, instead of whitelisting of file extension to run pipeline. Because, in my projects runs several programming languages and has many different file extensions.
Related
I have some problems with understanding how and why "rules" in GitLab CI work.
I have written a minimal code showing the issue in a GitLab project: https://gitlab.com/daniel.grabowski/gitlab-ci-rules-problems It contains two directories ("files1/" and "files2/") with some files in them and a .gitlab-ci.yml file.
My configuration
Here's the CI configuration:
stages:
- build
workflow:
rules:
- if: $CI_PIPELINE_SOURCE == "push"
.job_tpl:
image: alpine:latest
stage: build
variables:
TARGET_BRANCH: $CI_DEFAULT_BRANCH
rules:
- if: $CI_COMMIT_BRANCH == $TARGET_BRANCH
changes:
- $FILES_DIR/**/*
variables:
JOB_ENV: "prod"
- if: $CI_COMMIT_BRANCH != $TARGET_BRANCH
changes:
- $FILES_DIR/**/*
when: manual
allow_failure: true
variables:
JOB_ENV: "dev"
script:
- echo "CI_COMMIT_BRANCH=$CI_COMMIT_BRANCH"
- echo "TARGET_BRANCH=$TARGET_BRANCH"
- echo "JOB_ENV=$JOB_ENV"
files1 job:
extends: .job_tpl
variables:
FILES_DIR: files1
files2 job:
extends: .job_tpl
variables:
FILES_DIR: files2
As you can see in the above code I'm using workflow to run only "branch pipelines" and have two "twin" jobs configured to watch for changes in one of the project's directories each. The TARGET_BRANCH variable is of course unnecessary in the demo project but i need something like this in the real one and it shows one of my problems. Additionally the jobs behave differently depending on the branch for which they are run.
My expectations
What I want to achieve is:
Each of the jobs should be added to a pipeline only when I push changes to files1/ or files2/ directory respectively.
When I push changes to a branch different then "main" a manual job responsible for the changed directory shoud be added to a pipeline.
When I merge changes to the "main" branch a job responsible for the changed directory shoud be added to a pipeline and it should be automatically started.
Test scenario
I'm creating a new branch from "main", make some change in the file1/test.txt and push the branch to GitLab.
what I expect: a pipeline created with only "files1 job" runnable manually
what I get: a pipeline with both jobs (both manual). Actually I've found explanation of such behaviour here: https://docs.gitlab.com/ee/ci/jobs/job_control.html#jobs-or-pipelines-run-unexpectedly-when-using-changes - "The changes rule always evaluates to true when pushing a new branch or a new tag to GitLab."
On the same branch I make another change in the file1/test.txt and make push.
what I expect: a pipeline created with only "files1 job" runnable manually
what I get: exactly what I expect since the branch isn't a "new" one
I create a Merge Request from my branch to main and make the merge.
what I expect: a pipeline created with only "files1 job" which starts automatically
what I get: a pipeline created with only "files1 job" but a manual one
My questions/problems
Can you suggest me any way to bypass the issue with "changes" evaluating always to "true" on new branches? Actually it behaves exactly as I want it if I don't use "rules" but let's assume I need "rules".
Why the jobs run as "manual" on the main branch in spite of the "if" condition in which both CI_COMMIT_BRANCH and TARGET_BRANCH variables are (or should be) set to "main". To debug it I'm printing those vars in job's "script" and when I run it on "main" pipeline I'm getting:
$ echo "CI_COMMIT_BRANCH=$CI_COMMIT_BRANCH"
CI_COMMIT_BRANCH=main
$ echo "TARGET_BRANCH=$TARGET_BRANCH"
TARGET_BRANCH=main
$ echo "JOB_ENV=$JOB_ENV"
JOB_ENV=dev
so theoretically CI should enter into the "automatic" job path.
Generally I find the CI "rules" quite inconvenient and confusing but as I understand it GitLab prefers them to "only/except" solution so I'm trying to refactor my CI/CD to use them which will fail if I don't find solution for the above difficulties :(
I got two branches in my GitLab repo (uat and production). Two deploy jobs are meant to deploy a branch to a specific environment. There are two gitlab-ci.yml files, one in each branch (with the config for that branch) and production is my default branch.
The jobs should run only if files in dir/ changed and not for scheduled pipelines.
Problem: The deploy job for UAT is just working as expected: it runs if I push directly to the branch or if I accept a merge request. However, although there is no difference except the branch, the deploy job for production is not triggered on any event.
Question: Do you know if I misunderstood something and what would fix this?
Thanks!
gitlab-ci.yml in production
deploy_to_production:
only:
refs:
- production
changes:
- dir/*
except:
- schedules
script:
# upload to prod
gitlab-ci.yml in uat
deploy_to_uat:
only:
refs:
- uat
changes:
- dir/*
except:
- schedules
script:
# upload to uat
Do you have those empty lines before script: in your file?
This will define script under default, because it is not tied to a job.
default:
script:
# upload to uat
The reason that it is only running uat is that on the second reference the first one gets overwritten. You can check this on gitlab on your project page under CI/CD/Editor.
And here you can view the final yaml after merging:
Here is my .gitlab-ci.yml file:
script1:
only:
refs:
- merge_requests
- master
changes:
- script1/**/*
script: echo 'script1 done'
script2:
only:
refs:
- merge_requests
- master
changes:
- script2/**/*
script: echo 'script2 done'
I want script1 to run whenever there is a change in script1 directory; likewise script2.
I tested these with a change in script1, a change in script2, change in both the directories, and no change in either of these directories.
Former 3 cases are passing as expected but 4th case, the one with no change in either directory, is failing.
In the overview, Gitlab gives the message
Could not retrieve the pipeline status. For troubleshooting steps, read thedocumentation.
In the Pipelines tab, I have an option to Run pipeline. Clicking on that gives the error
An error occurred while trying to run a new pipeline for this Merge Request.
If there is no job, I want the pipeline to succeed.
Gitlab pipelines do not have any independent validity outside of jobs. A pipeline, by definition, consists of one or more jobs. In your example 4 above no jobs are created. The simplest hack you can add to your pipeline is a job which always runs:
dummyjob:
script: exit 0
Is the list value build:image just a name like build_image? Or does it have special usage in either the yaml file or the .gitlab-ci.yml file? If there isn't a special usage, what is the value of using name1:name2 instead of name1_name2?
The :image doesn't seem to be put into a variable. When I run this through the gitlab pipeline, the output is
Skipping Git submodules setup
Restoring cache
Downloading artifacts
Running before_script and script
$ echo image is $image
image
is
.gitlab-ci.yml
stages:
- build:image
- tag:image
- deploy
build:
stage: build:image
script:
- echo image is $image
I don't see anything like this in the GitLab CI/CD Pipeline Configuration Reference
Where did you see this .gitlab-ci.yml file?
I ran the .gitlab-ci.yml you provided and it seems to work fine, apparently GitLab CI doesn't treat the colon in any special way -- and I wouldn't expect it to, as there is no mention of it in the documentation.
I have a mono-repo with several projects (not my design choice).
Each project has a .gitlab-ci.yml setup to run a pipeline when a "version" file is changed. This is nice because a user can check-in to stage or master (for a hot-fix) and a build is created and deployed to a test environment.
The problem is when a user does a merge from master to stage and commits back to stage (to pull in any hot-fixes). This causes ALL the pipelines to run; even for projects that do not have actual content changes.
How do I allow the pipeline to run from master and/or stage but ONLY when the contents of the "version" file change? Like when a user changes the version number.
Here is an example of the .gitlab-ci.yml (I have 5 of these, 1 for each project in the mono-repo)
#
# BUILD-AND-TEST - initial build
#
my-project-build-and-test:
stage: build-and-test
script:
- cd $MY_PROJECT_DIR
- dotnet restore
- dotnet build
only:
changes:
- "MyProject/.gitlab-ci.VERSION.yml"
# no needs: here because this is the first step
#
# PUBLISH
#
my-project-publish:
stage: publish
script:
- cd $MY_PROJECT_DIR
- dotnet publish --output $MY_PROJECT_OUTPUT_PATH --configuration Release
only:
changes:
- "MyProject/.gitlab-ci.VERSION.yml"
needs:
- my-project-build-and-test
... and so on ...
I am still new to git, GitLab, and CI/pipelines. Any help would be appreciated! (I have little say in changing the mono-repo)
The following .gitlab-ci.yml will run the test_job only if the file version changes.
test_job:
script: echo hello world
rules:
- changes:
- version
See https://docs.gitlab.com/ee/ci/yaml/#ruleschanges
See also
Run jobs only/except for modifications on a path or file