I have a job in gitlab ci/cd defined:
authenticate:
stage: authenticate
image: google/cloud-sdk:debian_component_based
variables:
ENVIRONMENT: staging
rules:
- if: $CI_COMMIT_BRANCH == "main"
when: always
variables:
ENVIRONMENT: production
- if: $CI_COMMIT_BRANCH == "staging"
when: always
variables:
ENVIRONMENT: staging
- if: $CI_COMMIT_BRANCH == "development"
when: always
variables:
ENVIRONMENT: development
environment:
name: $ENVIRONMENT
I cannot set gitlab environment, because when i push to development branch, $CI_ENVIRONMENT_NAME is staging instead of development
Related
I'm writing a GitLab CI workflow in order to be re-used accross by project, using Docker for build and deployment.
On each project, I would like to only specify the needed changes to trigger this workflow under a merge request like so:
# .gitlab-ci.yml
include:
- { project: internal, ref: main, file: workflow.gitlab-ci.yml }
.workflow:changes:
- Dockerfile
- docker-compose.yml
- 'package*.json'
I was able to achieve it with the following:
# workflow.gitlab-ci.yml
build:
stage: build
script: docker build
rules:
- if: $CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_TAG =~ /^v[0-9]+\.[0-9]+\.[0-9]+$/
- if: $CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
changes: !reference [.workflow:changes]
test:
stage: test
script: docker test
rules:
- if: $CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
changes: !reference [.workflow:changes]
However, I also want to store my repeated rules conditions onto configuration I can reference to them like described on the official documentation:
# workflow.gitlab-ci.yml
.rules:production:
rules:
- if: $CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_TAG =~ /^v[0-9]+\.[0-9]+\.[0-9]+$/
.rules:staging:
rules:
- if: $CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
.rules:review:
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
changes: !reference [.workflow:changes]
build:
stage: build
script: docker build
rules:
- !reference [.rules:production, rules]
- !reference [.rules:staging, rules]
- !reference [.rules:review, rules]
test:
stage: test
script: docker test
rules:
- !reference [.rules:staging, rules]
- !reference [.rules:review, rules]
The .rules:review section has the exact same configuration as when it was placed under the build and test section of the first example. However, it give me the following invalid configuration error:
Unable to create pipeline
jobs:build:rules:rule:changes should be an array or a hash
Is that a GitLab CI limitation of the !reference keyword? Is there any workaround?
Note: The other .rules:production and .rules:staging configuration block are working properly.
I have this gitlab-ci.yml:
build-docker:
stage: build-docker
rules:
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
changes:
- app/Dockerfile
- app/requirements.txt
when: always
- when: manual
allow_failure: true
image:
name: alpine
entrypoint: [""]
script:
- echo 'Git Pulling, building and restarting'
deploy:
stage: deploy
rules:
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
changes:
- app/**/*
when: always
- when: manual
allow_failure: true
image:
name: alpine
entrypoint: [""]
script:
- echo 'Git Pulling and restarting'
My problem is that I doesn't need to run deploy if the changed files are only app/Dockerfile and/or app/requirements.txt (because the build job already ran and does the same as the deploy stage, and more), but I need it to run if changes happen on any other file inside app folder.
I already tried this in the deploy stage:
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
changes:
- "app/!(Dockerfile)"
- "app/!(requirements.txt)"
- app/**/*
when: always
- when: manual
allow_failure: true
But this doesn't work as expected.
My gitlab ci pipeline always blocks the terraform deploy, requiring manual action to start it. Is it possible to make it automatic instead?
From terraform gitlab yaml example
stages:
- validate
- test
- build
- deploy
- cleanup
sast:
stage: test
include:
- template: Terraform/Base.gitlab-ci.yml # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Terraform/Base.gitlab-ci.yml
fmt:
extends: .terraform:fmt
needs: []
validate:
extends: .terraform:validate
needs: []
build:
extends: .terraform:build
deploy:
extends: .terraform:deploy
dependencies:
- build
environment:
name: $TF_STATE_NAME
action: start
when: on_success
destroy:
extends: .terraform:destroy
environment:
name: $TF_STATE_NAME
action: stop
when: manual
Based on the documentation, when: on_success should automatically run the deploy command when the build stage succeeds. However, it still requires manual actions. Removing the when command is the same, it always requires a manual action to start the deploy.
Given I'm using gitlab's terraform template, is this hard coded to require manual actions to enable a deploy?
It's been a little while since I've worked on GitLab, but the template you reference has it as a rule:
.terraform:deploy: &terraform_deploy
stage: deploy
script:
- cd "${TF_ROOT}"
- gitlab-terraform apply
resource_group: ${TF_STATE_NAME}
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
when: manual
Which is different from just the when keyword that you're using.
What if you tried overriding with with your own rule?
deploy:
extends: .terraform:deploy
dependencies:
- build
environment:
name: $TF_STATE_NAME
action: start
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
when: on_success
Or better yet, just create/manage your own template from a repo of your own. Then you can modify the rules in there and delete the when: manual piece.
workflow:
rules:
- if : '$CI_COMMIT_BRANCH == "Sprint-Release-Branch"'
when: never
- if : '$CI_PIPELINE_SOURCE == "merge_request_event" && $CI_PIPELINE_SOURCE == "push"'
when: never
- when: always
stages:
- Stage1
- Stage2
- Stage3
Task1:
stage: Stage1
script:
- echo "Stage1"
rules:
- if: '$CI_COMMIT_TAG =~ /^\d+\.\d+\.DEV\d+/'
tags:
- Runner
Task2:
stage: Stage1
script:
- echo "Checking code standard as per "Coding Standards""
rules:
- if: '$CI_COMMIT_TAG =~ /^\d+\.\d+\.DEV\d+/'
allow_failure: true
tags:
- Runner
Task3:
stage: Stage2
script:
- echo "Stage2"
when: manual
tags:
- Runner
Task4:
stage: Stage3
script:
- echo "Stage3"
when: manual
tags:
- Runner
Above is my Gitlab CI file, where i am trying
pipeline should not add jobs when there is merge & push requests happened on "Sprint-Release-Branch"
but whenever any merge request done "feature branches" onto "Sprint-Release-Branch" jobs which are defined as "when: manual" get added in pipeline.
So in my situation, Dev team is creating different feature branch for different user-stories, and then merging those features branches onto Sprint-Release-Branch having above yml file. So multiple jobs are getting added in pipeline continuously for every merge_request which are defined with "manual" trigger
How can i make optimized my yml so that jobs having manual trigger should not get added in pipeline.
which are defined as "when: manual" get added in pipeline.
You have to repeat the whole logic when you overwrite it.
Task3:
...
rules:
- if : '$CI_COMMIT_BRANCH == "Sprint-Release-Branch"'
when: never
- if : '$CI_PIPELINE_SOURCE == "merge_request_event" && $CI_PIPELINE_SOURCE == "push"'
when: never
- when: manual
Also, it's better to use when: on_success, not always.
Do something like the following with yaml anchors:
.myrules: &myrules
if: $CI_COMMIT_BRANCH == "Sprint-Release-Branch" || ($CI_PIPELINE_SOURCE == "merge_request_event" && $CI_PIPELINE_SOURCE == "push")
when: never
workflow:
rules:
- *myrules
- when: on_success
Task3:
...
rules:
- *myrules
- when: manual
I am trying to put together a fairly complex pipeline with several jobs that run sequentially in our different environments. This is to run our Terraform changes across our infra. The sequence of jobs should run automatically across our infraci environment which is only ever rolled out to via CI, then stop and require a button click to start the deployment to our dev environment which has actual (albeit dev) users. Of course I don't want to write the same code over and over again so I've tried to be as DRY as possible. Here is my gitlab-ci.yml:
---
# "variables" & "default" are used by all jobs
variables:
TF_ROOT: '${CI_PROJECT_DIR}/terraform'
TF_CLI_CONFIG_FILE: .terraformrc
AWS_STS_REGIONAL_ENDPOINTS: regional
AWS_DEFAULT_REGION: eu-west-2
ASG_MODULE_PATH: module.aws_asg.aws_autoscaling_group.main_asg
default:
image:
name: hashicorp/terraform:light
entrypoint: ['']
cache:
paths:
- ${TF_ROOT}/.terraform
tags:
- nonlive # This tag matches the group wide GitLab runner.
before_script:
- cd ${TF_ROOT}
# List of all stages (jobs within the same stage are executed concurrently)
stages:
- init
- infraci_plan
- infraci_taint
- infraci_apply
- dev_plan
- dev_taint
- dev_apply
# "Hidden" jobs we use as templates to improve code reuse.
.default:
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
.plan:
extends: .default
stage: ${CI_ENVIRONMENT_NAME}_plan
script:
- terraform workspace select ${CI_ENVIRONMENT_NAME}
- terraform plan
rules:
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH && $CI_ENVIRONMENT_NAME != "infraci"'
when: manual
allow_failure: false
.taint:
extends: .default
stage: ${CI_ENVIRONMENT_NAME}_taint
script: terrafrom taint ${ASG_MODULE_PATH}
needs:
- ${CI_ENVIRONMENT_NAME}_plan
.apply:
extends: .default
stage: ${CI_ENVIRONMENT_NAME}_apply
script: terraform apply -auto-approve
# Create actual jobs
## init - runs once per pipeline
init:
stage: init
script: terraform init
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
when: always
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH && $CI_PIPELINE_SOURCE == "web"'
when: manual
## infraci - auto deploy
infraci_plan:
extends: .plan
environment:
name: infraci
infraci_taint:
extends: .taint
environment:
name: infraci
infraci_apply:
extends: .apply
environment:
name: infraci
## dev - manual deployment
dev_plan:
extends: .plan
environment:
name: dev
dev_taint:
extends: .taint
environment:
name: dev
dev_apply:
extends: .apply
environment:
name: dev
Unfortunately this fails validation with the following error:
infraci_plan job: chosen stage does not exist; available stages are .pre, init, infraci_plan, infraci_taint, infraci_apply, dev_plan, dev_taint, dev_apply, .post
My assumption is that it's to do with interpolating CI_ENVIRONMENT_NAME in the hidden jobs but not actually setting the value until the jobs where the jobs are actually defined.
If that's the case though what's a way to get the setup I need without a severe amount of duplication?
You are right, it is not possible to use a variable in stage. The only way I to see you need to define the stage directly in your job and remove the stage in .plan.
infraci_plan:
extends: .plan
stage: infraci_plan
environment:
name: infraci