I would like to trigger downstream pipeline but before that I need to eliminate the branches with rule method and I just want to trigger it only with specific branches. But I'm getting this error when I run the pipeline downstream pipeline cannot be created reference not found Did I miss something or can't I use rule and trigger methods in the same stage?
My stage:
test:
stage: test
variables:
branch: $CI_COMMIT_BRANCH
trigger:
project: test/project
strategy: depend
branch: $branch
allow_failure: false
rules:
- if: $CI_COMMIT_BRANCH == 'main' || $CI_COMMIT_BRANCH == 'test'
- when: never
Looks like the dash on the last line is not needed. Probably you meant this:
rules:
- if: $CI_COMMIT_BRANCH == 'main' || $CI_COMMIT_BRANCH == 'test'
when: never
I also recommend to check documentation for $CI_COMMIT_BRANCH:
The commit branch name. Available in branch pipelines, including
pipelines for the default branch. Not available in merge request
pipelines or tag pipelines.
You could try to use $CI_COMMIT_REF_NAME variable instead which is available in merge request or tag pipelines. You could try to debug this adding some echo commands (like - echo $CI_COMMIT_REF_NAME and - echo $branch to be sure that the branch really exists in the downstream project and variable are properly expanded.
But first of all you need to fix the syntax (that extra dash).
Consider the following pipeline:
image: alpine
stages:
- test
this-does-not-run:
stage: test
rules:
- if: $CI_PROJECT_DIR == "/builds/test"
script:
- echo "It works!"
this-runs:
stage: test
rules:
- if: $CI_PROJECT_DIR != "/builds/test"
script:
- echo "It doesn't work! CI_PROJECT_DIR- $CI_PROJECT_DIR"
And assume my repo is called test. The output of this CI will always be:
It doesn't work! CI_PROJECT_DIR- /builds/test
Obviously $CI_PROJECT_DIR == /builds/test, so I would expect the output to be It works!. Am I missing something, or is $CI_PROJECT_DIR not respected in rules?
The problem here is that variables in rules: must be expanded (evaluated by) the GitLab server at pipeline creation time, but $CI_PROJECT_DIR is determined by the runner's builds_dir setting, which is only known by the runner at runtime when the job actually runs:
References to unavailable variables are left intact. In this case, the runner attempts to expand the variable value at runtime. For example, a variable like CI_BUILDS_DIR is known by the runner only at runtime.
That is to say, like CI_BUILDS_DIR, you can't use CI_PROJECT_DIR or other runtime-only variables in rules:if:. Instead, you should use a variable that is known to the gitlab server at pipeline creation time, such as CI_PROJECT_NAME CI_PROJECT_ID, CI_PROJECT_URL, CI_PROJECT_PATH_SLUG or similar.
Try to wrap the expression in quotes. Afaik this is an expression and you would need to wrap it.
Eg:
rules:
- if: '$CI_PROJECT_DIR == "/builds/test"'
If you only have variables, then from experience you don't need to wrap it.
Eg:
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
In case the terminology is not standard, here is how I am using the below terms:
branch pipeline: A pipeline that is run when pushing to a branch.
MR pipeline: A pipeline that is run on a merge request, or pushes to a merge request branch.
I want to write a pipeline with two jobs, job_A and job_B. job_A should run on all pipelines. job_B should run only on merge request pipelines. One solution is to combine the workaround proposed in issue 194129, adding a workflow rule of - if: $CI with a merge_request_event rule, i.e.:
image: alpine
workflow:
rules:
- if: $CI
stages:
- stage_A
- stage_B
job_A:
stage: stage_A
script:
- echo "Hello from Job A"
job_B:
stage: stage_B
rules:
- if: $CI_PIPELINE_SOURCE == 'merge_request_event'
script:
- echo "Hello from Job B"
Now my pipeline runs in full on the MR -- which is what I wanted. However, two pipelines are being run now, the branch pipeline and the MR pipeline.
I want both job_A and job_B to run on MR pipelines even though job_A doesn't have the merge_request_event rule. But, I only want one pipeline to run when an MR is open -- the MR pipeline. How can I achieve this?
The correct answer is found in the gitlab docs.
workflow:
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
- if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS
when: never
- if: $CI_COMMIT_BRANCH
Try using only:variables. I had the same issue when mixing only/except with rules in one pipeline.
job_B:
stage: stage_B
only:
variables:
- $CI_PIPELINE_SOURCE == "merge_request_event"
script:
- echo "Hello from Job B"
I have some problems with understanding how and why "rules" in GitLab CI work.
I have written a minimal code showing the issue in a GitLab project: https://gitlab.com/daniel.grabowski/gitlab-ci-rules-problems It contains two directories ("files1/" and "files2/") with some files in them and a .gitlab-ci.yml file.
My configuration
Here's the CI configuration:
stages:
- build
workflow:
rules:
- if: $CI_PIPELINE_SOURCE == "push"
.job_tpl:
image: alpine:latest
stage: build
variables:
TARGET_BRANCH: $CI_DEFAULT_BRANCH
rules:
- if: $CI_COMMIT_BRANCH == $TARGET_BRANCH
changes:
- $FILES_DIR/**/*
variables:
JOB_ENV: "prod"
- if: $CI_COMMIT_BRANCH != $TARGET_BRANCH
changes:
- $FILES_DIR/**/*
when: manual
allow_failure: true
variables:
JOB_ENV: "dev"
script:
- echo "CI_COMMIT_BRANCH=$CI_COMMIT_BRANCH"
- echo "TARGET_BRANCH=$TARGET_BRANCH"
- echo "JOB_ENV=$JOB_ENV"
files1 job:
extends: .job_tpl
variables:
FILES_DIR: files1
files2 job:
extends: .job_tpl
variables:
FILES_DIR: files2
As you can see in the above code I'm using workflow to run only "branch pipelines" and have two "twin" jobs configured to watch for changes in one of the project's directories each. The TARGET_BRANCH variable is of course unnecessary in the demo project but i need something like this in the real one and it shows one of my problems. Additionally the jobs behave differently depending on the branch for which they are run.
My expectations
What I want to achieve is:
Each of the jobs should be added to a pipeline only when I push changes to files1/ or files2/ directory respectively.
When I push changes to a branch different then "main" a manual job responsible for the changed directory shoud be added to a pipeline.
When I merge changes to the "main" branch a job responsible for the changed directory shoud be added to a pipeline and it should be automatically started.
Test scenario
I'm creating a new branch from "main", make some change in the file1/test.txt and push the branch to GitLab.
what I expect: a pipeline created with only "files1 job" runnable manually
what I get: a pipeline with both jobs (both manual). Actually I've found explanation of such behaviour here: https://docs.gitlab.com/ee/ci/jobs/job_control.html#jobs-or-pipelines-run-unexpectedly-when-using-changes - "The changes rule always evaluates to true when pushing a new branch or a new tag to GitLab."
On the same branch I make another change in the file1/test.txt and make push.
what I expect: a pipeline created with only "files1 job" runnable manually
what I get: exactly what I expect since the branch isn't a "new" one
I create a Merge Request from my branch to main and make the merge.
what I expect: a pipeline created with only "files1 job" which starts automatically
what I get: a pipeline created with only "files1 job" but a manual one
My questions/problems
Can you suggest me any way to bypass the issue with "changes" evaluating always to "true" on new branches? Actually it behaves exactly as I want it if I don't use "rules" but let's assume I need "rules".
Why the jobs run as "manual" on the main branch in spite of the "if" condition in which both CI_COMMIT_BRANCH and TARGET_BRANCH variables are (or should be) set to "main". To debug it I'm printing those vars in job's "script" and when I run it on "main" pipeline I'm getting:
$ echo "CI_COMMIT_BRANCH=$CI_COMMIT_BRANCH"
CI_COMMIT_BRANCH=main
$ echo "TARGET_BRANCH=$TARGET_BRANCH"
TARGET_BRANCH=main
$ echo "JOB_ENV=$JOB_ENV"
JOB_ENV=dev
so theoretically CI should enter into the "automatic" job path.
Generally I find the CI "rules" quite inconvenient and confusing but as I understand it GitLab prefers them to "only/except" solution so I'm trying to refactor my CI/CD to use them which will fail if I don't find solution for the above difficulties :(
How do I run a .gitlab-ci.yml job only on a tagged Master branch?
job:
script:
- echo "Do something"
only:
- master
- tags
The above code will run if either condition exists: a Master branch or a tagged commit.
My goal is to have this run for a production deploy, but it would require that it be on the Master branch and that it be tagged (with a version). Otherwise, I'll have another job that will push to staging if its missing a tag.
This behavior will be introduced in version 12.
Open issue was recently update:
Jason Lenny #jlenny changed title from {-Update .gitlab-ci.yml to
support conjunction logic for build conditions-} to Conjunction logic
for build conditions MVC · 2 days ago
Jason Lenny #jlenny changed milestone to 12.0 · 2 days ago
(fingers crossed)
A solution is to use the except keyword to exclude all the branches, in conjunction with only to run on tags, in this way you run your pipeline only on tag in master branch:
only:
- tags
except:
- branches
I'm using version 11.3.4
Thanks to others like Matt Alioto who posted about the open issue (which is labeled Product Vision 2019 so hopefully they knock it out this year).
Specific to Carlson Cole's question, this would work:
job_for_master_no_tags:
stage: deploy
script:
- echo "Release to Staging"
only:
- master
job_for_master_tags_only:
stage: deploy
script:
- echo "Release to Production"
only:
- tags
except:
- /^(?!master).+#/ # Ruby RegEx for anything not starting with 'master'
To see how this RegEx works check out https://rubular.com/r/1en2eblDzRP5Ha
I tested this on GitLab version 11.7.0 and it works
Note: If you try to use - /^(?!master).+/ (without the #) it doesn't work - learned that the hard way 😕
I made it work with this working code snippet, all others were not working for me
only:
- tags # please mention the 's' compared to Sergio Tomasello's solution
except:
- branches
I use 11.4.3
I had the same problem. I wanted to trigger a deploy to our staging-environment on a push or merge, and only when applying a tag deploy it our production-environment.
We need 2 variables for this: $CI_COMMIT_BRANCH and $CI_COMMIT_TAG. With these variables we could deduct if the pipeline was triggered by a commit or a tag. Unfortunately the first variable is only set when committing on a branch, while the second variable only is set on applying a Tag. So this was no solution...
So I went for the next-best setup by only do a production-release when a tag is set by specified conventions and by a manual trigger.
This is my .gitlab-ci.yml file:
stages:
- deploy:staging
- deploy:prod
deploy-to-staging:
stage: deploy:staging
rules:
- if: $CI_COMMIT_BRANCH == 'master'
script:
- echo "Deploying to Staging..."
deploy-to-production:
stage: deploy:prod
rules:
- if: $CI_COMMIT_TAG =~ /^v(?:\d+.){2}(?:\d+)$/
when: manual
script:
- echo "Deploying to Production..."
If you really want to automate this, you have to do a little scripting to find out if the applied tag actually belongs to a commit that is on the master-branch. Check this comment on the GitLab issuetracker for more information: https://gitlab.com/gitlab-org/gitlab-foss/-/issues/31305#note_28580169
This behavior is not yet supported by gitlab-ci, although there is an open issue to add it.
In the meantime I've also heard anecdotal reports that
only:
- master
only:
- tags
will get the job done (as well as anecdotal reports that it won't).
Years later, still trying to launch a job on tags on master branch...
The issue at Gitlab has been closed : https://gitlab.com/gitlab-org/gitlab-foss/-/issues/27818
It's not possible to spot a tag on master branch as Git does not work this way. Branches and tags are separate references, each pointing to a commit. So, a tag has no relation with a branch.
My solution is to check the tag name to detect that it represents a PRODUCTION release :
deploy-prod:
stage: deploy-manual
only:
variables:
- $CI_COMMIT_TAG =~ /^v\d+.\d+.\d+-?.*$/
when: manual
The regexp matches semver tag names like :
v1.2.0
v2.0.0-beta.1
...
There is no proper build in solution in gitlab so far for this problem. To keep track of the development of a proper solution and to keep a working workaround updated I created: Gitlab CI: Run Pipeline job only for tagged commits that exist on protected branches
I have faced the same issue, this is how I tried to solve it
my_job:
stage: build
services:
- name: docker:dind
image: docker:latest
script:
- ...
rules:
- if: $CI_COMMIT_BRANCH == 'master' && $CI_COMMIT_TAG == null
cache: {}
this job runs only when there is a commit on the master branch (excluding any other commit on personal/feature branch). In the very same way I trigger builds on tags:
script:
- ...
rules:
- if: $CI_COMMIT_BRANCH == 'master' && $CI_COMMIT_TAG != null
Docs actually say that only and except are not being developed and that users should use rules instead.
Here is an example which will only run on a tag.
deploy-to-prod:
stage: deploy
rules:
- if: $CI_COMMIT_TAG
environment:
name: production
variables:
API_ENVIRONMENT: prod
when: manual
I have found workaround for this.
.gitlab-ci.yml
build-image-prod:
stage: image
only:
refs:
- tags
variables:
- $CI_COMMIT_TAG =~ /^v.*$/
script:
- ./scripts/not-on-main $CI_COMMIT_TAG && exit 0
- echo "Image building..."
scripts/not-on-main:
#!/bin/sh
if [ -z "$1" ]; then
echo "Missing argument with commit ref."
exit 0
fi
git fetch origin main
if git branch -r --contains "$1" | grep -q main; then
echo "Ref $1 is on main branch."
exit 1
else
echo "Ref $1 is not on main branch - exiting."
exit 0
fi
The job runs for each tag in repository if name of tag starts with "v". The helper script fetches main and checks if tag is on it. If the tag is not on main branch then the job exits without building.