I need to run deploy pipeline in GitLab CI, but only in master and if new tag added.
In my company we such versioning procedure.
1.You made improvements in your branch.
2.Then you make commit with name "something-0.1.1"
3.Then you add the same tag on this commit.
4.Then you merge your branch in master.
I need to create such conditions to pipeline so it will runs only after STEP 4 and if new tag was added, cause not every merge master should be deployed. Only if commit has tag.
deploy:
script:
- echo "$CI_COMMIT_TAG"
rules:
- if: '$CI_COMMIT_TAG =~ /.+-[0-9]+\.[0-9]+\.[0-9]*$/ && $CI_COMMIT_BRANCH == "main"'
The rules above doesn't work.
Pls any idea
I have some problems with understanding how and why "rules" in GitLab CI work.
I have written a minimal code showing the issue in a GitLab project: https://gitlab.com/daniel.grabowski/gitlab-ci-rules-problems It contains two directories ("files1/" and "files2/") with some files in them and a .gitlab-ci.yml file.
My configuration
Here's the CI configuration:
stages:
- build
workflow:
rules:
- if: $CI_PIPELINE_SOURCE == "push"
.job_tpl:
image: alpine:latest
stage: build
variables:
TARGET_BRANCH: $CI_DEFAULT_BRANCH
rules:
- if: $CI_COMMIT_BRANCH == $TARGET_BRANCH
changes:
- $FILES_DIR/**/*
variables:
JOB_ENV: "prod"
- if: $CI_COMMIT_BRANCH != $TARGET_BRANCH
changes:
- $FILES_DIR/**/*
when: manual
allow_failure: true
variables:
JOB_ENV: "dev"
script:
- echo "CI_COMMIT_BRANCH=$CI_COMMIT_BRANCH"
- echo "TARGET_BRANCH=$TARGET_BRANCH"
- echo "JOB_ENV=$JOB_ENV"
files1 job:
extends: .job_tpl
variables:
FILES_DIR: files1
files2 job:
extends: .job_tpl
variables:
FILES_DIR: files2
As you can see in the above code I'm using workflow to run only "branch pipelines" and have two "twin" jobs configured to watch for changes in one of the project's directories each. The TARGET_BRANCH variable is of course unnecessary in the demo project but i need something like this in the real one and it shows one of my problems. Additionally the jobs behave differently depending on the branch for which they are run.
My expectations
What I want to achieve is:
Each of the jobs should be added to a pipeline only when I push changes to files1/ or files2/ directory respectively.
When I push changes to a branch different then "main" a manual job responsible for the changed directory shoud be added to a pipeline.
When I merge changes to the "main" branch a job responsible for the changed directory shoud be added to a pipeline and it should be automatically started.
Test scenario
I'm creating a new branch from "main", make some change in the file1/test.txt and push the branch to GitLab.
what I expect: a pipeline created with only "files1 job" runnable manually
what I get: a pipeline with both jobs (both manual). Actually I've found explanation of such behaviour here: https://docs.gitlab.com/ee/ci/jobs/job_control.html#jobs-or-pipelines-run-unexpectedly-when-using-changes - "The changes rule always evaluates to true when pushing a new branch or a new tag to GitLab."
On the same branch I make another change in the file1/test.txt and make push.
what I expect: a pipeline created with only "files1 job" runnable manually
what I get: exactly what I expect since the branch isn't a "new" one
I create a Merge Request from my branch to main and make the merge.
what I expect: a pipeline created with only "files1 job" which starts automatically
what I get: a pipeline created with only "files1 job" but a manual one
My questions/problems
Can you suggest me any way to bypass the issue with "changes" evaluating always to "true" on new branches? Actually it behaves exactly as I want it if I don't use "rules" but let's assume I need "rules".
Why the jobs run as "manual" on the main branch in spite of the "if" condition in which both CI_COMMIT_BRANCH and TARGET_BRANCH variables are (or should be) set to "main". To debug it I'm printing those vars in job's "script" and when I run it on "main" pipeline I'm getting:
$ echo "CI_COMMIT_BRANCH=$CI_COMMIT_BRANCH"
CI_COMMIT_BRANCH=main
$ echo "TARGET_BRANCH=$TARGET_BRANCH"
TARGET_BRANCH=main
$ echo "JOB_ENV=$JOB_ENV"
JOB_ENV=dev
so theoretically CI should enter into the "automatic" job path.
Generally I find the CI "rules" quite inconvenient and confusing but as I understand it GitLab prefers them to "only/except" solution so I'm trying to refactor my CI/CD to use them which will fail if I don't find solution for the above difficulties :(
I have created a new CD/CD pipeline in GitLab via a .gitlab-ci.yml file in the repo root in a new project with a job structured like so:
...
test:
stage: test
script:
- pip install tox flake8
- tox -e py36,flake8
# Run only for merge requests on main branch
rules:
- if: '$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME == "main"'
...
GitLab does not trigger the pipeline, saying there is no .gitlab-ci.yml file in the repository. I had assumed that pipeline changes would apply to the merge request run that was triggered. I can understand why this isn't the case for security purposes in a public repository, but I would like to test pipeline changes in the merge request that I created for my self-hosted private GitLab instance.
Is this possible?
This was a programming error. I needed to use:
- if: '$CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "main"'
instead of:
- if: '$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME == "main"'
In our git process we have hothix process. The process is to create a hotfix brunch from master and make changes and then merge to master back.
The issue that I'm faŃing now is how to add job to pipelne if branch just created and exclude from pipeline for any further push?
When branch is created we need to set up custom environment. Create a sandbox, push code and push some test data. This is need only when branch is created. In case of changes and pushes, environment is available and no need in building it again. Just build code and push a release tag
I'm trying something like this:
job-name:
stage: build
script:
- myScript
rules:
- if: '$CI_COMMIT_BRANCH =~ /^hotfix.*/'
changes:
- force-app/**/*
when: never
- if: '$CI_COMMIT_BRANCH =~ /^hotfix.*/'
allow_failure: false
when: manual
And for sure it's not working as changes: not part of if policy:
Rules are evaluated in order until a match is found. If a match is found, the attributes are checked to see if the job should be added to the pipeline.
The CI pipeline runs on every commit in my Gitlab repository at work. Is there way to disable that and only run the CI pipeline on an open merge request to the master branch?
There is currently no configuration option to do that. Here are some things that can be used to "disable" a pipeline build.
Adding [ci skip] inside the commit message will not trigger a pipeline on push.
Using except and only options on all jobs inside the pipeline. To avoid duplication in this case, you can use Anchors.
Update: GitLab 11.7
When pushing to GitLab you can skip triggering a pipeline by passing ci.skip option to the push command: git push -o ci.skip
Update in 2020 because solution with only and except are candidates for deprecation : https://docs.gitlab.com/ee/ci/yaml/#onlyexcept-basic
Still in .gitlab-ci.yml you should use now rules : See https://docs.gitlab.com/ee/ci/yaml/#rules
I simply set this rule on my build job and the job is "blocked" and wait a manual trigger from the UI.
rules:
- when: manual
Note that we can create more advanced rules with conditions to trigger for exemple if we see a git tag.
https://docs.gitlab.com/ee/ci/yaml/#exclude-jobs-with-rules-from-certain-pipelines
Yes, but only if you set it on each job within the pipeline.
https://docs.gitlab.com/ee/ci/yaml/#onlyexcept-basic
job:
script: echo 'test'
only:
- merge_requests
That job will only run on commits that are part of a merge request, rather than every push of a commit to a branch. If you set every job to this setting then the pipeline will never run.
You can just add [ci skip] to the commit message and builds will be skipped
https://gitlab.com/gitlab-org/gitlab-ce/issues/14499
You could add something like this in the beginning of your .gitlab-ci.yaml
workflow:
rules:
- if: "$CI_COMMIT_BRANCH"
when: never
- if: '$CI_PIPELINE_SOURCE == "merge_request_event" && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"'
For more info look at Merge request pipelines
No. (Not yet !)
You can follow the expected feature development here.
https://gitlab.com/gitlab-org/gitlab-ce/issues/23902
However, you can automatically trigger pipelines using "maofr"'s code
https://gitlab.com/gitlab-org/gitlab-ce/issues/23902#note_88958643
For merge we can use this guide #onlyrefs--exceptrefs
merge_requests
For pipelines created when a merge request is created or updated. Enables merge request pipelines, merged results pipelines, and merge trains.
and for generally triggering pipeline we can use only: and variables: together:
script:
- . diff.sh $SMS_MESSAGE
only:
variables:
- $ACTION == "diff"
then triggering it is easy using curl:
curl -X POST \
-F token=$TOKEN \ # your token
-F ref=$REF_NAME \ # branch name
-F "variables[ACTION]=diff" \ # variable
https://gitlab.com/api/v4/projects/0/trigger/pipeline # your project endpoint
This part variables[ACTION]=diff will assign diff into ACTION and it will apply to .gitlab-ci.yml and only: section.
job:
stage: build
script:
- echo "Do your build here"
except:
- pushes
Use except pushes to stop create a new pipeline on every pushed commits. For reference check this
Update 2022 :
According to the documention of Gitlab on: https://docs.gitlab.com/ee/ci/yaml/#when
you can use when: manual to run the job only when triggered manually.
Example:
deploy_job:
stage: deploy
script:
- make deploy
when: manual
You don't need also rules for that.