I would like to trigger downstream pipeline but before that I need to eliminate the branches with rule method and I just want to trigger it only with specific branches. But I'm getting this error when I run the pipeline downstream pipeline cannot be created reference not found Did I miss something or can't I use rule and trigger methods in the same stage?
My stage:
test:
stage: test
variables:
branch: $CI_COMMIT_BRANCH
trigger:
project: test/project
strategy: depend
branch: $branch
allow_failure: false
rules:
- if: $CI_COMMIT_BRANCH == 'main' || $CI_COMMIT_BRANCH == 'test'
- when: never
Looks like the dash on the last line is not needed. Probably you meant this:
rules:
- if: $CI_COMMIT_BRANCH == 'main' || $CI_COMMIT_BRANCH == 'test'
when: never
I also recommend to check documentation for $CI_COMMIT_BRANCH:
The commit branch name. Available in branch pipelines, including
pipelines for the default branch. Not available in merge request
pipelines or tag pipelines.
You could try to use $CI_COMMIT_REF_NAME variable instead which is available in merge request or tag pipelines. You could try to debug this adding some echo commands (like - echo $CI_COMMIT_REF_NAME and - echo $branch to be sure that the branch really exists in the downstream project and variable are properly expanded.
But first of all you need to fix the syntax (that extra dash).
Related
While developing gitlab ci-cd pipeline i want to run pipeline manually for a particular branch for example branch name is "develop-tool". what i want is anything done on this branch should not trigger pipeline automatically. it should be manually. what i have tried is:
workflow:
rules:
- when: manual # Error: workflow:rules:rule when unknown value: manual
then i tried to do this:
- if: '$CI_PIPELINE_SOURCE == "web" && $CI_COMMIT_REF_NAME !~ /^.*-$develop-tool/'
- when: always
its taking job to some infinite loop and it keeps running until and unless you cancel it and run the job again. does anyone have any other way to achieve this goal.
Have the configuration skip pipelines when the source is not the web for that branch. So the only way it can be triggered on that branch is through the web UI.
workflow:
rules:
- if: '$CI_COMMIT_BRANCH == "develop-tool" && $CI_PIPELINE_SOURCE != "web"'
when: never
- when: always
I had a gitlab job defined as follows.
test-1:
stage: test
only:
variables:
- $RUN_TEST
except:
- tags
and then I changed that job to run based on gitlab rules.
test-1:
stage: test
rules:
- if: '$CI_PIPELINE_SOURCE != "schedule" && $CI_COMMIT_TAG == null'
- if: '$RUN_TESTS == "true" && $CI_COMMIT_TAG == null'
After this change, Whenever I commit some change to my branch, I am getting two pipeline runs in my gitlab project. One is for the latest commit I have made and second as detached.
How can I get rid of detached pipeline run?
Another interesting thing is, when I cancel the latest pipeline the detach pipeline also got cancelled.
For latest pipeline, value of CI_PIPELINE_SOURCE = push and for detach pipeline, value of CI_PIPELINE_SOURCE = merge_request_event.
The detached pipelines you are seeing are merge request pipelines. You can use workflow:rules to control when a pipeline is created. To prevent merge request pipelines:
workflow:
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
when: never
- when: always
Lets assume I have few stages
stages:
- mr:stage1
- mr:stage2
- mr:stage3
On all jobs I have rule:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
So I am getting pipeline like that
How can I split my 3 stages into 2 pipelines? For example I would like to have mr:stage1 and mr:stage2 in one pipeline and if this pipeline is successful, mr:stage3 will invoke in separate pipeline.
Thx for help
Each project or repository in GitLab has a single .gitlab-ci.yml file, and therefore a single pipeline. There is no way to have multiple pipelines like this.
You can include other yml files in your base .gitlab-ci.yml file, but this is solely for your convenience as the pipeline author. At runtime, the included files are copy and pasted into a single yml file, so there is still only ever a single pipeline.
However, that single pipeline can look drastically different depending on your use cases.
Let's say you have 3 jobs that only run on push events to a non-default branch. When you push to the default branch, these 3 jobs will not run. Let's say you have another 2 jobs that only run when there's a push event to the default branch. When you push here, these jobs will run, but if you push to a feature branch, they will not.
This scenario might look like this:
stages:
- build
- test
- deploy
Pull in Backend Dependencies nonprod:
stage: build
rules:
- if: $CI_PIPELINE_SOURCE === 'push' && $CI_COMMIT_REF_NAME !== $CI_DEFAULT_BRANCH
when: always
- when: never
script:
- # run your dependency manager to pull in backend dependencies, ie Composer for PHP, including dependencies only used in lower environments, like Unit Testing libraries, etc.
# ...
Pull in Frontend Dependencies nonprod:
stage: build
rules:
- if: $CI_PIPELINE_SOURCE === 'push' && $CI_COMMIT_REF_NAME !== $CI_DEFAULT_BRANCH
when: always
- when: never
script:
- # Run npm install, etc., including those only needed in lower environments
# ...
Pull in Backend Dependencies Prod:
stage: build
rules:
- if: $CI_PIPELINE_SOURCE === 'push' && ($CI_COMMIT_REF_NAME === $CI_DEFAULT_BRANCH || $CI_COMMIT_TAG !== '')
when: always
- when: never
script:
- # run your dependency manager to pull in backend dependencies, ie Composer for PHP, without dev dependencies
# ...
Run Unit Tests:
stage: tests
rules:
- if: $CI_PIPELINE_SOURCE === 'push' && $CI_COMMIT_REF_NAME !== $CI_DEFAULT_BRANCH
when: always
- when: never
script:
- # run our unit tests
Deploy to production:
stage: deploy
rules:
- if: $CI_PIPELINE_SOURCE === 'push' && $CI_COMMIT_REF_NAME === $CI_DEFAULT_BRANCH
when: always
- when: never
script:
- # deploy steps to prod
etc.
For the first two jobs, the rules say "If the event is a Push AND the branch isn't the default branch, always run this job. Otherwise, if any of these conditions isn't true, never run this job".
The Prod job's rules say "If the event is a Push AND the branch IS the default branch, OR it's a tag, run this job. Otherwise never run it."
Then we only run our unit test job for feature branches, and we only deploy to production for the default branch.
Therefore, depending on which branch/tag is pushed to, the pipeline instance will look very different. For a feature branch we'll have 2 build jobs, and a test job (2 stages). For the default branch, we'll have a single build job and a deploy job.
The same is true if you need to handle sources other than 'push'. For example, if you have a job that only runs when triggered via the API (or from another pipeline instance or another project's pipeline), you'd look to see if the $CI_PIPELINE_SOURCE variable holds the string trigger.
I want to see sonar results in the MR(merge request) command section when I create a MR.
My main expectations:
if there is an existing MR for the source branch, trigger detached pipeline (do not trigger feature pipeline. I need only that one for reviewing sonar results in MR commands)
if there isn't an existing MR for the source branch, just trigger the normal feature(source) branch pipeline
I tried to do it with the below example stage. But when I pushed the commit to the source pipeline, while MR is exist for source branch. I still getting double pipeline. Detach and source pipelines are running and I don't want to see both in same time, plus except not working with rules configuration. How can I integrate except section with rules part.
This is my gitlab-ci stage:
deploy:
stage: deployment
when: manual
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event" && $CUSTOM_VARIABLE == "true" || $CUSTOM_VARIABLE == "true"'
script:
- ....
- ....
except:
- tags
- main
I also tried below rules, if one of them fit my condition don't run the other one. But it still trigger both pipelines.
deploy:
stage: deployment
when: manual
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event" && $CUSTOM_VARIABLE == "true"'
when: on_success
- if: '$CI_PIPELINE_SOURCE == "push" && $CUSTOM_VARIABLE == "true"'
when: on_success
script:
- ....
- ....
except:
- tags
- main
Covered in workflow:rules templates, In this case, you can use the CI_OPEN_MERGE_REQUESTS variable to determine whether to run the pipeline for merge request or just the feature branch.
If you use both [pipelines for merge requests and branch pipelines], duplicate pipelines might run at the same time. To prevent duplicate pipelines, use the CI_OPEN_MERGE_REQUESTS variable.
Using workflow:rules you can do this for the entire pipeline, but the same principle can also be applied to individual jobs.
workflow:
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS'
when: never
- if: '$CI_COMMIT_BRANCH'
This means your pipeline will run:
for merge requests
for branch pipelines UNLESS there is an open merge requests
Imagine a simple .gitlab-ci.yml file with the following
stages:
- test
- build
test_job:
stage: test
rules:
- if: '$CI_COMMIT_BRANCH && $CI_COMMIT_BRANCH != $CI_DEFAULT_BRANCH'
script:
- exit 1
build_job:
stage: build
rules:
- if: '$CI_COMMIT_BRANCH && $CI_COMMIT_BRANCH != $CI_DEFAULT_BRANCH'
changes:
- web-app/**/*
script:
- echo "Building..."
The first time this pipeline is run, you will see both jobs, however the build_job will not be run because the test_job failed (exit 1).
Correcting the exit 1 (to exit 0, for example) and re-running, you will now only see the test_job because to gitlab, the web-app files havent changed, yet they havent successfully run previously.
So how do you ensure the build_job is run to success??
In the scenario you described, your second commit contains only 1 change to your .gitlab-ci.yml file, therefore your changes: rule correctly causes the build_job to be excluded from the pipeline as you have configured the pipeline.
For branch pipelines, GitLab will not consider previous commits when evaluating the changes: rules.
However, with pipelines for merge requests, all changes in the merge request are tested. Which sounds like the behavior you are expecting.
If your goal is for efficient pipelines tracking your features branches, your best course(s) of action might be something like this:
Always run the build job on branch pipelines
Only apply the changes: rule for pipelines for merge requests
Optionally, exclude branch pipelines when an MR is open
Optionally, adopt a workflow whereby which you open (draft) merge requests when creating your feature branches
An optimized CI configuration that does not skip build job on branches may look like this:
build_job:
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
changes: # in the case of merge requests, we can be more efficient
- web-app/**/*
- if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS'
when: never # skip branch pipelines when MR pipelines exist
- if: '$CI_COMMIT_BRANCH' # but in branch pipelines, build every time