I am trying to run the below script as gitlab-ci.yml but somehow I am not able to trigger the CICD pipeline with a merge request.
Steps I am following:
Create a branch from feature/new and name it as feature/abc
Make test changes to feature/abc
Create Merge request with target branch as feature/new.
stages: # List of stages for jobs, and their order of execution
- deployment
- testing
workflow:
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event" && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "feature/new"'
deployment-dev: # This job runs in the deploy stage.
stage: deployment
image: google/cloud-sdk
services:
- docker:dind
script:
- echo hello
environment:
name: main/$CI_COMMIT_REF_NAME
when: on_success
testing-dev: # This job also runs in the test stage.
stage: testing
image: google/cloud-sdk
services:
- docker:dind
script:
- echo testing in progress
Related
I am trying to setup CI in gitlab so
the second job (pushdev) will be available for running manually only after the devjob has run successfully.
the third job pushtostage will only run iff file has changed.
the way the jobs are setup, second and third jobs alway run. What is missing in the pipeline spec
devjob:
image: node:16
stage: publishdev
script:
- echo "running validation checks"
- npm run validate
rules:
- changes:
- ./src/myfile.txt
- when: manual
# - this jobs needs to run after "devjob" has run successfully
# and myfile.txt has changed
# - "needs" artifacts from the "lint" job
pushdev:
image: node:16
stage: publishdev
needs: [ "devjob", "lint"]
script:
- echo "Pushing changes after validation to dev"
- npm run pushdev
rules:
- changes:
- ./src/myfile.txt
when: on_success
- when: manual
pushtostage:
image: node:16
stage: pushstage
script:
- echo "Pushing changes to stage"
rules:
- changes:
- ./src/myfile.txt
- when: manual
I change your sample to look like this:
stages:
- publishdev
- pushstage
default:
image: ubuntu:20.04
lint:
stage: publishdev
script:
- echo "lint job"
devjob:
stage: publishdev
script:
- echo "running validation checks"
rules:
- changes:
- README.md
when: manual
allow_failure: false
pushdev:
stage: publishdev
needs: [ "devjob", "lint"]
script:
- echo "Pushing changes after validation to dev"
rules:
- changes:
- README.md
when: manual
allow_failure: false
pushtostage:
stage: pushstage
script:
- echo "Pushing changes to stage"
rules:
- changes:
- README.md
when: manual
allow_failure: false
I add allow_failure: false, because allow_failure when manual job default is true.
I merge your rules. because GitLab rules one - is one rule:
Rules are evaluated when the pipeline is created, and evaluated in order until the first match.
your .gitlab-ci.yml first job devjob is manual, so it is always a success, and your second job pushdev first rule changes and when: on_success always match, so it always run.
I change your .gitlab-ci.yml, first job devjob merge your rules when file change and set it is manual job and not allow_failure. and so on.
the sample code in Files · try-rules-stackoverflow-72594854-manual · GitLab PlayGround / Workshop / Tryci · GitLab
I have been racking my brain around this and I feel like I am going nowhere. I have a project whose code is hosted on Gitlab. My project has two branches, develop and main. I make changes on my develop branch, push these changes which updates my staging instance. When I am ready to deploy live changes, I merge with main which is meant to trigger a deployment to my live instance. This is my gitlab-ci.yml file
default:
image: docker:19.03.10
tags:
- docker
.deployment_script: &deployment_script
- echo "Starting deployment to ECS ..."
- echo "So many lines here that deploy code to aws ecs"
services:
- docker:dind
variables:
REPOSITORY_URL: **********.dkr.ecr.**-****-*.amazonaws.com/********/****
before_script:
- echo "Creating image tag ..."
- IMAGE_TAG="$(echo $CI_COMMIT_SHA | head -c 8)"
stages:
- build
- deploy
build:
stage: build
script:
- echo $DEPLOY_ENVIRONMENT ...
- echo "The script here build my docker container and publishes to aws ecr"
- echo "Build is only meant to be ran when pushing to develop"
rules:
- if: '$CI_COMMIT_BRANCH == "develop"'
when: always
deploy-staging:
environment: staging
dependencies:
- "build"
image: registry.gitlab.com/gitlab-org/cloud-deploy/aws-base:latest
script:
- *deployment_script
rules:
- if: '$CI_COMMIT_BRANCH == "develop"'
when: manual
deploy-production:
environment: production
image: registry.gitlab.com/gitlab-org/cloud-deploy/aws-base:latest
script:
- *deployment_script
rules:
- if: '$CI_COMMIT_BRANCH == "main"'
when: manual
Every time I deploy to the develop branch, the pipelines are triggered and I successfully deploy my staging application. However, no matter how many times I push to main, nothing is triggered. I have no idea why. I hope someone can point me in the right direction too figure this out.
I want my pipelines to run:
backend_tests job for any push with changes in backend/ folder,
frontend_tests job for any push with changes in frontend/ folder,
old_code_tests job for push with changes in backend/ folder, or when MR is created and there are changes in backend folder in the source branch
My problem is that with the following .gitlab-ci.yml the old_code_tests job will run for any push to the branch with open MR, if there are already changes in the backend folder - even if push did not introduced such changes. I have no idea how to avoid this. The job is created correctly when MR is being created.
In other words: if there already is MR and branch contains backend changes and I push frontend only changes, the old_code_test job should NOT run - and it unexpectedly runs with given configuration.
I do not want to run old_code_tests job if there is no MR for given branch - and it works.
stages:
- frontend_tests
- backend_tests
- old_code
old_code_test:
extends: .test_old_code_template
stage: old_code
needs: []
script:
- echo "Test old code"
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- if: '$CI_PIPELINE_SOURCE == "push"'
when: never
- changes:
- backend/**/*
backend_tests:
stage: backend_tests
needs: []
extends: .backend_template
script:
- echo "Test backend"
rules:
- if: '$CI_PIPELINE_SOURCE == "push" && $CI_OPEN_MERGE_REQUESTS'
when: never
- changes:
- backend/**/*
frontend_tests:
stage: frontend_tests
needs: []
extends: .frontend_template
script:
- echo "Frontend test"
rules:
- if: '$CI_PIPELINE_SOURCE == "push" && $CI_OPEN_MERGE_REQUESTS'
when: never
- changes:
- frontend/**/*
Try to use only keyword in your old_code stage:
only:
changes:
- backend/**/*
I am trying to put together a fairly complex pipeline with several jobs that run sequentially in our different environments. This is to run our Terraform changes across our infra. The sequence of jobs should run automatically across our infraci environment which is only ever rolled out to via CI, then stop and require a button click to start the deployment to our dev environment which has actual (albeit dev) users. Of course I don't want to write the same code over and over again so I've tried to be as DRY as possible. Here is my gitlab-ci.yml:
---
# "variables" & "default" are used by all jobs
variables:
TF_ROOT: '${CI_PROJECT_DIR}/terraform'
TF_CLI_CONFIG_FILE: .terraformrc
AWS_STS_REGIONAL_ENDPOINTS: regional
AWS_DEFAULT_REGION: eu-west-2
ASG_MODULE_PATH: module.aws_asg.aws_autoscaling_group.main_asg
default:
image:
name: hashicorp/terraform:light
entrypoint: ['']
cache:
paths:
- ${TF_ROOT}/.terraform
tags:
- nonlive # This tag matches the group wide GitLab runner.
before_script:
- cd ${TF_ROOT}
# List of all stages (jobs within the same stage are executed concurrently)
stages:
- init
- infraci_plan
- infraci_taint
- infraci_apply
- dev_plan
- dev_taint
- dev_apply
# "Hidden" jobs we use as templates to improve code reuse.
.default:
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
.plan:
extends: .default
stage: ${CI_ENVIRONMENT_NAME}_plan
script:
- terraform workspace select ${CI_ENVIRONMENT_NAME}
- terraform plan
rules:
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH && $CI_ENVIRONMENT_NAME != "infraci"'
when: manual
allow_failure: false
.taint:
extends: .default
stage: ${CI_ENVIRONMENT_NAME}_taint
script: terrafrom taint ${ASG_MODULE_PATH}
needs:
- ${CI_ENVIRONMENT_NAME}_plan
.apply:
extends: .default
stage: ${CI_ENVIRONMENT_NAME}_apply
script: terraform apply -auto-approve
# Create actual jobs
## init - runs once per pipeline
init:
stage: init
script: terraform init
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
when: always
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH && $CI_PIPELINE_SOURCE == "web"'
when: manual
## infraci - auto deploy
infraci_plan:
extends: .plan
environment:
name: infraci
infraci_taint:
extends: .taint
environment:
name: infraci
infraci_apply:
extends: .apply
environment:
name: infraci
## dev - manual deployment
dev_plan:
extends: .plan
environment:
name: dev
dev_taint:
extends: .taint
environment:
name: dev
dev_apply:
extends: .apply
environment:
name: dev
Unfortunately this fails validation with the following error:
infraci_plan job: chosen stage does not exist; available stages are .pre, init, infraci_plan, infraci_taint, infraci_apply, dev_plan, dev_taint, dev_apply, .post
My assumption is that it's to do with interpolating CI_ENVIRONMENT_NAME in the hidden jobs but not actually setting the value until the jobs where the jobs are actually defined.
If that's the case though what's a way to get the setup I need without a severe amount of duplication?
You are right, it is not possible to use a variable in stage. The only way I to see you need to define the stage directly in your job and remove the stage in .plan.
infraci_plan:
extends: .plan
stage: infraci_plan
environment:
name: infraci
I have a pipeline with 3 stages: build, deploy-test and deploy-prod. I want stages to have following behavior:
always run build
run deploy-test automatically when on master or manually when on other branches
run deploy-prod manually, only available on master branch
My pipeline configuration seems to achieve that but I have a problem when trying to merge branches into master. I don't want to execute deploy-test stage on every branch before doing merge. Right now I am required to do that as the merge button is disabled with a message Pipeline blocked. The pipeline for this merge request requires a manual action to proceed. The setting Pipelines must succeed in project is disabled.
I tried adding additional rule to prevent deploy-test stage from running in merge requests but it didn't change anything:
rules:
- if: '$CI_MERGE_REQUEST_ID'
when: never
- if: '$CI_COMMIT_BRANCH == "master"'
when: on_success
- when: manual
Full pipeline configuration:
stages:
- build
- deploy-test
- deploy-prod
build:
stage: build
script:
- echo "build"
deploy-test:
stage: deploy-test
script:
- echo "deploy-test"
rules:
- if: '$CI_COMMIT_BRANCH == "master"'
when: on_success
- when: manual
deploy-prod:
stage: deploy-prod
script:
- echo "deploy-prod"
only:
- master
The only way I got it to work was to set ☑️ Skipped pipelines are considered successful in Setttings > General > Merge requests > Merge Checks
and marking the manual step as "allow_failure"
upload:
stage: 'upload'
rules:
# Only allow uploads for a pipeline source whitelisted here.
# See: https://docs.gitlab.com/ee/ci/jobs/job_control.html#common-if-clauses-for-rules
- if: $CI_COMMIT_BRANCH
when: 'manual'
allow_failure: true
After this clicking the Merge when Pipeline succeeds button …
… will merge the MR without any manual interaction:
I've opened a merge request from branch "mybranch" into "master" with the following .gitlab-ci.yml:
image: alpine
stages:
- build
- deploy-test
- deploy-prod
build:
stage: build
script:
- echo "build"
# run deploy-test automatically when on master or manually when on other branches
# Don't run on merge requests
deploy-test:
stage: deploy-test
script:
- echo "deploy-test"
rules:
- if: $CI_MERGE_REQUEST_ID
when: never
- if: '$CI_COMMIT_BRANCH == "master"'
when: on_success
- when: manual
# run deploy-prod manually, only available on master branch
deploy-prod:
stage: deploy-prod
script:
- echo "deploy-prod"
rules:
- if: '$CI_COMMIT_BRANCH == "master"'
when: manual
Notes:
only is deprecated, so I replaced it with if
I added Alpine image to make the jobs run faster (slimmer container); it doesn't affect the logic
When I pushed changes to branch "mybranch", GitLab did the following:
showed a blue "Merge when pipeline succeeds" button on my MR
ran "build" stage
skipped "deploy-prod" stage (only available on "master" branch)
gave me a manual "play" button to run the job on "mybranch"
at this point, the pipeline status is "blocked" and the MR is showing "Pipeline blocked. The pipeline for this merge request requires a manual action to proceed"
now I manually start the "deploy-test" stage by selecting the Play icon in the Pipelines screen
pipeline status indicator changes to "running" and then to "passed"
my merge request shows the pipeline passed and gives me the green "Merge" button
There are a number of variables that are available to the pipeline on runtime - Predefined variables reference
Some are available specifically for pipelines associated with merge requests - Predefined variables for merge request pipelines
You can utilize one or more of these variables to determine if you would want to run the deploy-test job for that merge request.
For example, you could use mention the phrase "skip_cicd" in your merge request title, access it with CI_MERGE_REQUEST_TITLE variable and create a rule. Your pipeline would look somewhat like this (please do test the rule, I have edited the pipeline off the top of my head and could be wrong) -
stages:
- build
- deploy-test
- deploy-prod
build:
stage: build
script:
- echo "build"
deploy-test:
stage: deploy-test
script:
- echo "deploy-test"
rules:
- if: '$CI_MERGE_REQUEST_TITLE == *"skip_cicd"*'
when: never
- if: '$CI_COMMIT_BRANCH == "master"'
when: on_success
- when: manual
deploy-prod:
stage: deploy-prod
script:
- echo "deploy-prod"
only:
- master