Gitlab-ci lost environment variables - gitlab

I'm developing a pipeline on GitLab-ci, in the first job I use gittools/gitversion the obtain the semantic version of my software.
Here a small piece of code of /gitversion-ci-cd-plugin-extension.gitlab-ci.yml (Full documentation here https://gitversion.net/docs/reference/build-servers/gitlab)
.gitversion_function:
image:
name: gittools/gitversion
entrypoint: ['']
stage: .pre
.....
.....
artifacts:
reports:
#propagates variables into the pipeline level
dotenv: thisversion.env
Then a simplified version of my pipeline is as follows
stages:
- .pre
- install_dependencies
- build
- deploy
include:
- local: '/gitversion-ci-cd-plugin-extension.gitlab-ci.yml'
determineversion:
extends: .gitversion_function
install_dependencies:
image: node:16.14
stage: install_dependencies
script:
- echo ${PACKAGE_VERSION}
build:
image: node:16.14
stage: build
script:
- echo $PACKAGE_VERSION
deploy:
image: bitnami/kubectl
stage: deploy
needs: ['build']
script:
- echo $PACKAGE_VERSION
The problem is that the environment variable $PACKAGE_VERSION works in the first two jobs install_dependencies and build.
echo $PACKAGE_NAME; //0.0.1
But when the jobs deploy is executed the environment variable is not expanded by pipeline and I obtain literally this
echo $PACKAGE_NAME; //$PACKAGE_NAME

I found the problem.
In the last job of my pipeline, I use needs (https://docs.gitlab.com/ee/ci/yaml/#needs) to establish dependencies between jobs.
The problem is that artifact is not automatically passed because there is no a dependency between determineversion and deploy, to fix I do this:
...
deploy:
image: bitnami/kubectl
stage: deploy
needs: ['determineversion', 'build'] # <------
script:
- echo $PACKAGE_VERSION
...
I added determineversion as a dependency of deploy, in this way $PACKAGE_VERSION is printed correctly

Related

How to read a value from env file to decide whether to execute GitLab CI stage?

I have a variable stored into env file :
stages:
- build
- execute
build:
stage: build
script:
- Add-Content properties.env -Value PROD="TRUE"
artifacts:
reports:
dotenv: properties.env
tags:
- windows
Now i wanted to read this value in a stage and then decide whether to execute that stage or not.
I did the following, but it isn't working:
execute:
stage: execute
rules:
- if: $PROD == "TRUE"
when: always
script:
- echo "happy"
tags:
- windows
dependencies:
- build
Any help ?
Thanks.
You can try and use the approach described in "Pass an environment variable to another job":
build:
stage: build
script:
- echo "BUILD_VERSION=hello" >> build.env
artifacts:
reports:
dotenv: build.env
deploy_one:
stage: deploy
script:
- echo "$BUILD_VERSION" # Output is: 'hello'
dependencies:
- build
environment:
name: customer1
deployment_tier: production

Gitlab CI/CD: use multiple when conditions

I have like this gitlab ci cd configuration file:
image: docker:git
stages:
- develop
- production
default:
before_script:
- apk update && apk upgrade && apk add git curl
deploy:
stage: develop
script:
- echo "Hello World"
backup:
stage: develop
when:
- manual
- on_success
remove:
stage: develop
when:
- delayed
- on_success
start_in: 30 minutes
In my case job deploy runs automaticaly and job backup must runs manually only when successfully completed job deploy. But in my case this configuration doesn't works and I get error with message:
Found errors in your .gitlab-ci.yml:
jobs:backup when should be one of:
on_success
on_failure
always
manual
delayed
How I can use multiple when option arguments in my case?
Basically you can't because when does not expect an array. You can work around it though with needs. But this solution does only work if you run your jobs in different stages.
image: docker:git
stages:
- deploy
- backup
- remove
deploy:develop:
stage: deploy
script:
- exit 1
backup:develop:
stage: backup
script:
- echo "backup"
when: manual
needs: ["deploy:develop"]
remove:develop:
stage: remove
script:
- echo "remove"
when: delayed
needs: ["backup:develop"]
start_in: 30 minutes

How to reuse job in .gitlab-ci.yml

I currently have two jobs in my CI file which are nearly identical.
The first is for manually compiling a release build from any git branch.
deploy_internal:
stage: deploy
script: ....<deploy code>
when: manual
The second is to be used by the scheduler to release a daily build from develop branch.
scheduled_deploy_internal:
stage: deploy
script: ....<deploy code from deploy_internal copy/pasted>
only:
variables:
- $MY_DEPLOY_INTERNAL != null
This feels wrong to have all that deploy code repeated in two places. It gets worse. There are also deploy_external, deploy_release, and scheduled variants.
My question:
Is there a way that I can combine deploy_internal and scheduled_deploy_internal such that the manual/scheduled behaviour is retained (DRY basically)?
Alternatively: Is there is a better way that I should structure my jobs?
Edit:
Original title: Deploy job. Execute manually except when scheduled
You can use YAML anchors and aliases to reuse the script.
deploy_internal:
stage: deploy
script:
- &deployment_scripts |
echo "Deployment Started"
bash command 1
bash command 2
when: manual
scheduled_deploy_internal:
stage: deploy
script:
- *deployment_scripts
only:
variables:
- $MY_DEPLOY_INTERNAL != null
Or you can use extends keyword.
.deployment_script:
script:
- echo "Deployment started"
- bash command 1
- bash command 2
deploy_internal:
extends: .deployment_script
stage: deploy
when: manual
scheduled_deploy_internal:
extends: .deployment_script
stage: deploy
only:
variables:
- $MY_DEPLOY_INTERNAL != null
Use GitLab's default section containing a before_script:
default:
before_script:
- ....<deploy code>
job1:
stage: deploy
script: ....<code after than deploy>
job2:
stage: deploy
script: ....<code after than deploy>
Note: the default section fails to function as such if you try to execute a job locally with the gitlab-runner exec command - use YAML anchors instead.

Build to multiple build servers on a single Gitlab merge request?

I need some direction here. I'm reading whatever documentation I can find online but it's not hitting the right synapses or I haven't found the right link yet. On a merge request to a deployable environment, I want to kick off a build on two separate machines. Both machines are IBM Is, running different versions of the OS. I'd like for these builds and subsequent deploys to happen independently of each other.
My .yml file has the entries for the build for the two machines (QQDEV & BNADEV), but the builds occur sequentially, not in parallel. The picture below is what Gitlab draws.
To me, from the above picture, it looks like both build_BNADEV and build_QQDEV are going to run the deploy jobs DEV_BNADEV and DEV_QQDEV. I want build_BNADEV to run DEV_BNADEV, et al, and that is a separate issue aside from the parallel builds.
What do I need here? Another runner? Another pipeline? Just looking for general pointers and direction here.
Here is my YAML.
stages:
- build
- deploy
build_QQDEV:
variables:
THING: "This is a THING for build for QQDEV"
script:
- "bash ./GitLabCI/GitLabCI.Build.sh qqdev"
stage: build
only:
- DEV
- QA
- UAT
- PROD
build_BNADEV:
variables:
THING: "This is a THING for build for BNADEV"
script:
- "bash ./GitLabCI/GitLabCI.Build.sh bnadev"
stage: build
only:
- DEV
- QA
DEV_QQDEV:
variables:
THING: "This is a THING for deploy_DEV_QQDEV"
ASPGRP: "*NONE"
script:
- "bash ./GitLabCI/GitLabCI.Deploy.sh QQDEV EPDEV1_5 /home/quikq/1.5/dev"
stage: deploy
environment:
name: DEV
only:
- DEV
DEV_BNADEV:
variables:
THING: "This is a THING for deploy_DEV_BNADEV"
REBUILD_DEPLOYMENT: "0"
ASPGRP: "DATADEV"
script:
- "bash ./GitLabCI/GitLabCI.Deploy.sh BNADEV EPDEV1_5 /home/quikq/1.5/dev"
stage: deploy
environment:
name: DEV
only:
- DEV
QA_QQDEV:
variables:
THING: "This is a THING for deploy_QA_QQDEV"
ASPGRP: "*NONE"
script:
- "bash ./GitLabCI/GitLabCI.Deploy.sh QQDEV EPQA1_5 /home/quikq/1.5/qa"
stage: deploy
environment:
name: QA
only:
- QA
QA_BNADEV:
variables:
THING: "This is a THING for deploy_QA_BNADEV"
REBUILD_DEPLOYMENT: "0"
ASPGRP: "DATADEV"
script:
- "bash ./GitLabCI/GitLabCI.Deploy.sh BNADEV EPQA1_5 /home/quikq/1.5/qa"
stage: deploy
environment:
name: QA
only:
- QA
UAT_QQ:
variables:
THING: "This is a THING for deploy_UAT_QQ"
ASPGRP: "*NONE"
script:
- "bash ./GitLabCI/GitLabCI.Deploy.sh QQ EPUAT1_5 /home/quikq/1.5/uat"
stage: deploy
environment:
name: UAT
only:
- UAT
UAT_QQBNA:
variables:
THING: "This is a THING for deploy_UAT_QQBNA"
ASPGRP: "*NONE"
script:
- "bash ./GitLabCI/GitLabCI.Deploy.sh QQBNA EPUAT1_5 /home/quikq/1.5/uat"
stage: deploy
environment:
name: UAT
only:
- UAT
PROD_QQ:
variables:
THING: "This is a THING for deploy_PROD_QQ"
ASPGRP: "*NONE"
script:
- "bash ./GitLabCI/GitLabCI.Deploy.sh QQ EPPROD1_5 /home/quikq/1.5/prod"
stage: deploy
environment:
name: PROD
only:
- PROD
PROD_QQBNA:
variables:
THING: "This is a THING for deploy_PROD_QQBNA"
ASPGRP: "*NONE"
script:
- "bash ./GitLabCI/GitLabCI.Deploy.sh QQBNA EPPROD1_5 /home/quikq/1.5/prod"
stage: deploy
environment:
name: PROD
only:
- PROD

Sending to production or review environments depending on branch

We're using Gitlab-CI but we have some troubles to have review and production environments at the same time.
We have several stages in our .gitlab-ci.yml but here I'll focus on the deploy stage:
deploy:
stage: deploy
script:
- some commands
environment:
name: review/$CI_BUILD_REF_NAME
url: http://$CI_BUILD_REF_SLUG.$DEPLOY_SERVER
on_stop: stop_deploy
only:
- /^feature-[cw]\/.*$/
deploy:
stage: deploy
script:
- some other commands
environment:
name: production
only:
- prod
stop_deploy:
stage: deploy
variables:
GIT_STRATEGY: none
script:
- some clean commands
when: manual
environment:
name: review/$CI_BUILD_REF_NAME
action: stop
only:
- /^feature-[cw]\/.*$/
The issue is that the first job is not run on the branches whose name starts with feature-c/. However when removing the second job, the first job is run on those branches.
The job that deploys to production is correctly run when pushed to prod.
So why does the first job is not run when the second job is defined? Where does the conflict comes from?
Thanks!
The answer is quite simple; they can't have the same name :) Name one deploy-review and the other deploy-prod and its fixed.

Resources