I have a Gitlab CI config that kinda looks like this:
stages:
- test
- deploy
test:
stage: test
only:
- merge_request
- master
script:
- jest --coverage
deploy:
stage: deploy
only:
- master
dependencies:
- test
script:
- make deploy
I only want the tests to be run when a merge request is opened or if we merge to master because I'm only on the free plan on gitlab.com and I'd like to conserve my runner minutes.
If the unit tests ran for every commit we made, we'd always run out of minutes on the 3rd or 4th week.
For the most part, it works. The problem comes from pushing to master directly (which can happen every now and then); test runs twice and at the same time.
I couldn't find anything on Gitlab docs on how to properly approach this. Any help would be great.
I actually don't see why a direct push to master will run your tests twice except when you have an open merge-request which master as source branch.
You can prevent this from happening by using workflow. Furthermore you should use rules instead of only/except as they are not actively developed any more.
workflow:
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS'
when: never
- if: '$CI_COMMIT_BRANCH'
stages:
- test
- deploy
test:
stage: test
script:
- jest --coverage
rules:
- if: '$CI_COMMIT_BRANCH == "master" || $CI_PIPELINE_SOURCE == "merge_request_event"'
deploy:
stage: deploy
dependencies:
- test
script:
- make deploy
rules:
- if: '$CI_COMMIT_BRANCH == "master"'
Related
I'm new to this Gitlab CI/CD features, and I encountered the following issues.
I have these 2 jobs in my gitlab-ci.yml, the automation test and my deployment job.
automation_test_scheduled:
stage: test
script:
- yarn test:cypress
only:
- schedules
deploy_to_staging:
stage: deploy-staging
environment: staging
only:
- staging
I want to run my automation test automatically on a daily basis and I have created a new pipeline schedule against my staging branch.
however, when the scheduler is triggered, it also runs my deployment job, which is not needed because I only want my automation test to run in the scheduled pipeline.
Does this happen because my deploy_to_staging job has only: - staging rules? if so, how can I set my scheduled pipeline to only run the automation test without triggering another job?
If you wanted to do this with only/except, it would probably be sufficient to add
except:
- schedules
to your deployment job.
Though as
Though notably, the rules based system is preferred at this point.
This also allows for more expressive and detailed decisions when it comes to running jobs.
The simplest way to set the rules for the two jobs would be:
automation_test_scheduled:
stage: test
script:
- yarn test:cypress
rules:
- if: $CI_PIPELINE_SOURCE == "schedule"
deploy_to_staging:
stage: deploy-staging
environment: staging
rules:
- if: $CI_PIPELINE_SOURCE == "schedule"
when: never
- if: $CI_COMMIT_REF_SLUG == "staging"
And that might be all you need.
Though when it comes to rules, a particularly convenient way of handling them is defining some common rules for the configuration, and reusing these through yaml anchors. The following are some reusable definitions for your case:
.definitions:
rules:
- &if-scheduled
if: $CI_PIPELINE_SOURCE == "schedule"
- ¬-scheduled
if: $CI_PIPELINE_SOURCE == "schedule"
when: never
- &if-staging
if: $CI_COMMIT_REF_SLUG == "staging"
And after that you could use them in your jobs like this:
automation_test_scheduled:
stage: test
script:
- yarn test:cypress
rules:
- *if-scheduled
deploy_to_staging:
stage: deploy-staging
environment: staging
rules:
- *not-scheduled
- *if-staging
This way of handling the rules makes it a bit easier to have a overview, and to reuse rules, which in large configurations definitely makes sense
You should use rules instead of only as the latter is not in active development any more.
With that in mind you can change to the following rules clause using the predefined variables CI_COMMIT_REF_SLUG and CI_PIPELINE_SOURCE. The automation_test_scheduled is only run on the branch staging if triggered by a schedule and the deploy_to_staging job is run on any change on the staging branch.
automation_test_scheduled:
stage: test
script:
- yarn test:cypress
rules:
- if: '$CI_COMMIT_REF_SLUG == "staging" && $CI_PIPELINE_SOURCE == "schedule"'
deploy_to_staging:
stage: deploy-staging
environment: staging
rules:
- if: '$CI_COMMIT_REF_SLUG == "staging"'
I want my pipelines to run:
backend_tests job for any push with changes in backend/ folder,
frontend_tests job for any push with changes in frontend/ folder,
old_code_tests job for push with changes in backend/ folder, or when MR is created and there are changes in backend folder in the source branch
My problem is that with the following .gitlab-ci.yml the old_code_tests job will run for any push to the branch with open MR, if there are already changes in the backend folder - even if push did not introduced such changes. I have no idea how to avoid this. The job is created correctly when MR is being created.
In other words: if there already is MR and branch contains backend changes and I push frontend only changes, the old_code_test job should NOT run - and it unexpectedly runs with given configuration.
I do not want to run old_code_tests job if there is no MR for given branch - and it works.
stages:
- frontend_tests
- backend_tests
- old_code
old_code_test:
extends: .test_old_code_template
stage: old_code
needs: []
script:
- echo "Test old code"
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- if: '$CI_PIPELINE_SOURCE == "push"'
when: never
- changes:
- backend/**/*
backend_tests:
stage: backend_tests
needs: []
extends: .backend_template
script:
- echo "Test backend"
rules:
- if: '$CI_PIPELINE_SOURCE == "push" && $CI_OPEN_MERGE_REQUESTS'
when: never
- changes:
- backend/**/*
frontend_tests:
stage: frontend_tests
needs: []
extends: .frontend_template
script:
- echo "Frontend test"
rules:
- if: '$CI_PIPELINE_SOURCE == "push" && $CI_OPEN_MERGE_REQUESTS'
when: never
- changes:
- frontend/**/*
Try to use only keyword in your old_code stage:
only:
changes:
- backend/**/*
We would like to have a .gitlab-ci.yml which supports the default CI pipeline and the SAST pipeline only scheduled once a day.
lint, build, test-unit (on merge request)
test-sast (scheduled once a day)
What seems logic but didn't work is this configuration:
include:
- template: Security/SAST.gitlab-ci.yml
- template: Workflows/MergeRequest-Pipelines.gitlab-ci.yml
image: node:lts-alpine
stages:
- lint
- build
- test
lint:
stage: lint
script:
- npm i
- npm run lint
build:
stage: build
script:
- npm i
- npm run build
test-unit:
stage: test
script:
- npm i
- npm run test:unit
test-sast:
stage: test
script: [ "true" ]
rules:
- if: $CI_PIPELINE_SOURCE == "schedule"
when: always
- when: never
Then did some tests using the environment variable SAST_DISABLED which didn't work as well.
May be someone has a similiar setup and may help out with a working sample?
Your workflow:rules do not have an explicit allow for $CI_PIPELINE_SOURCE == "schedule"
This is what I use for merge request pipelines:
workflow:
rules:
# Do not start pipeline for WIP/Draft commits
- if: $CI_COMMIT_TITLE =~ /^(WIP|Draft)/i
when: never
# MergeRequest-Pipelines workflow
# For merge requests create a pipeline.
- if: $CI_MERGE_REQUEST_IID || $CI_PIPELINE_SOURCE == "merge_request_event"
# For tags, create a pipeline.
- if: $CI_COMMIT_TAG
# For default branch create a pipeline (this includes on schedules, pushes, merges, etc.).
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
# For other pipeline triggers
- if: $CI_PIPELINE_SOURCE =~ /^trigger|pipeline|web|api$/
I have a gitlab pipeline where there are two stages, one is build and the other one is deploy. The build stage is run when a commit is made. I want a way to run the deploy job when the merge request is merged to master. I tried several things but no luck. Can anyone help?
stages:
- build
- deploy
dotnet:
script: "echo This builds!"
stage: build
production:
script: "echo This deploys!"
stage: deploy
only:
refs:
- master
Try using the gitlab-ci.yml "rules" feature to check for the merge request event.
Your current gitlab-ci.yml will run your "dotnet" job every commit, merge request, schedule, and manually triggered pipeline.
https://docs.gitlab.com/ee/ci/yaml/#workflowrules
dotnet:
script: "echo This builds!"
stage: build
rules:
- if: '$CI_COMMIT_REF_NAME != "master" && $CI_PIPELINE_SOURCE == "push" || $CI_PIPELINE_SOURCE == "merge_request_event"'
production:
script: "echo This deploys!"
stage: deploy
rules:
- if: '$CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_REF_NAME == "master"'
If you want your job run on only after merging the merge request, then you can trigger a job based on the commit message, as shown below.
rules:
- if: '$CI_COMMIT_MESSAGE =~ /See merge request/'
Basically, all the merge request comes with a "See merge request" commit message so you can depend on that message to trigger your job.
You can run the pipeline after merge by using Gitlab ci predefined variable $CI_MERGE_REQUEST_APPROVED this will return true after merge and available from gitlab v14.1.
If you want to run the pipeline when merge request is created to main branch you can use $CI_MERGE_REQUEST_TARGET_BRANCH_NAME with $CI_MERGE_REQUEST_APPROVED.
you can add the rule like this in your job.
rules:
- if: $CI_MERGE_REQUEST_APPROVED && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "main"
CI_MERGE_REQUEST_APPROVED variable seems to be designed for check approve status, not for triggering pipeline. (Issue, https://gitlab.com/gitlab-org/gitlab/-/issues/375908)
When you use merge request predefined variables, gitlab creates pipeline at merge request created, not at merge request approved.
Instead, use CI_COMMIT_BRANCH predefined variables with regular expression and restrict commit at master branch using branch access rights.
stages:
- build
- deploy
dotnet:
stage: build
rules:
- if: $CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == "master"
script: "echo This builds!"
production:
stage: deploy
rules:
- if: $CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == "master"
script: "echo This deploys!"
Edit answer for incorrect example. (edited at 2023. 01. 26)
Assume that we have rc and master branch.
We can configure master branch as protected, and use only for merge request.
In this case, we can define gitlab-ci file like below.
stages:
- build
- deploy
dotnet:
stage: build
rules:
- if: $CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == "rc"
script: "echo This builds!"
production:
stage: deploy
rules:
- if: $CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == "master"
script: "echo This deploys!"
Thanks for notification #Kappacake
I have a pipeline with 3 stages: build, deploy-test and deploy-prod. I want stages to have following behavior:
always run build
run deploy-test automatically when on master or manually when on other branches
run deploy-prod manually, only available on master branch
My pipeline configuration seems to achieve that but I have a problem when trying to merge branches into master. I don't want to execute deploy-test stage on every branch before doing merge. Right now I am required to do that as the merge button is disabled with a message Pipeline blocked. The pipeline for this merge request requires a manual action to proceed. The setting Pipelines must succeed in project is disabled.
I tried adding additional rule to prevent deploy-test stage from running in merge requests but it didn't change anything:
rules:
- if: '$CI_MERGE_REQUEST_ID'
when: never
- if: '$CI_COMMIT_BRANCH == "master"'
when: on_success
- when: manual
Full pipeline configuration:
stages:
- build
- deploy-test
- deploy-prod
build:
stage: build
script:
- echo "build"
deploy-test:
stage: deploy-test
script:
- echo "deploy-test"
rules:
- if: '$CI_COMMIT_BRANCH == "master"'
when: on_success
- when: manual
deploy-prod:
stage: deploy-prod
script:
- echo "deploy-prod"
only:
- master
The only way I got it to work was to set ☑️ Skipped pipelines are considered successful in Setttings > General > Merge requests > Merge Checks
and marking the manual step as "allow_failure"
upload:
stage: 'upload'
rules:
# Only allow uploads for a pipeline source whitelisted here.
# See: https://docs.gitlab.com/ee/ci/jobs/job_control.html#common-if-clauses-for-rules
- if: $CI_COMMIT_BRANCH
when: 'manual'
allow_failure: true
After this clicking the Merge when Pipeline succeeds button …
… will merge the MR without any manual interaction:
I've opened a merge request from branch "mybranch" into "master" with the following .gitlab-ci.yml:
image: alpine
stages:
- build
- deploy-test
- deploy-prod
build:
stage: build
script:
- echo "build"
# run deploy-test automatically when on master or manually when on other branches
# Don't run on merge requests
deploy-test:
stage: deploy-test
script:
- echo "deploy-test"
rules:
- if: $CI_MERGE_REQUEST_ID
when: never
- if: '$CI_COMMIT_BRANCH == "master"'
when: on_success
- when: manual
# run deploy-prod manually, only available on master branch
deploy-prod:
stage: deploy-prod
script:
- echo "deploy-prod"
rules:
- if: '$CI_COMMIT_BRANCH == "master"'
when: manual
Notes:
only is deprecated, so I replaced it with if
I added Alpine image to make the jobs run faster (slimmer container); it doesn't affect the logic
When I pushed changes to branch "mybranch", GitLab did the following:
showed a blue "Merge when pipeline succeeds" button on my MR
ran "build" stage
skipped "deploy-prod" stage (only available on "master" branch)
gave me a manual "play" button to run the job on "mybranch"
at this point, the pipeline status is "blocked" and the MR is showing "Pipeline blocked. The pipeline for this merge request requires a manual action to proceed"
now I manually start the "deploy-test" stage by selecting the Play icon in the Pipelines screen
pipeline status indicator changes to "running" and then to "passed"
my merge request shows the pipeline passed and gives me the green "Merge" button
There are a number of variables that are available to the pipeline on runtime - Predefined variables reference
Some are available specifically for pipelines associated with merge requests - Predefined variables for merge request pipelines
You can utilize one or more of these variables to determine if you would want to run the deploy-test job for that merge request.
For example, you could use mention the phrase "skip_cicd" in your merge request title, access it with CI_MERGE_REQUEST_TITLE variable and create a rule. Your pipeline would look somewhat like this (please do test the rule, I have edited the pipeline off the top of my head and could be wrong) -
stages:
- build
- deploy-test
- deploy-prod
build:
stage: build
script:
- echo "build"
deploy-test:
stage: deploy-test
script:
- echo "deploy-test"
rules:
- if: '$CI_MERGE_REQUEST_TITLE == *"skip_cicd"*'
when: never
- if: '$CI_COMMIT_BRANCH == "master"'
when: on_success
- when: manual
deploy-prod:
stage: deploy-prod
script:
- echo "deploy-prod"
only:
- master