I need some direction here. I'm reading whatever documentation I can find online but it's not hitting the right synapses or I haven't found the right link yet. On a merge request to a deployable environment, I want to kick off a build on two separate machines. Both machines are IBM Is, running different versions of the OS. I'd like for these builds and subsequent deploys to happen independently of each other.
My .yml file has the entries for the build for the two machines (QQDEV & BNADEV), but the builds occur sequentially, not in parallel. The picture below is what Gitlab draws.
To me, from the above picture, it looks like both build_BNADEV and build_QQDEV are going to run the deploy jobs DEV_BNADEV and DEV_QQDEV. I want build_BNADEV to run DEV_BNADEV, et al, and that is a separate issue aside from the parallel builds.
What do I need here? Another runner? Another pipeline? Just looking for general pointers and direction here.
Here is my YAML.
stages:
- build
- deploy
build_QQDEV:
variables:
THING: "This is a THING for build for QQDEV"
script:
- "bash ./GitLabCI/GitLabCI.Build.sh qqdev"
stage: build
only:
- DEV
- QA
- UAT
- PROD
build_BNADEV:
variables:
THING: "This is a THING for build for BNADEV"
script:
- "bash ./GitLabCI/GitLabCI.Build.sh bnadev"
stage: build
only:
- DEV
- QA
DEV_QQDEV:
variables:
THING: "This is a THING for deploy_DEV_QQDEV"
ASPGRP: "*NONE"
script:
- "bash ./GitLabCI/GitLabCI.Deploy.sh QQDEV EPDEV1_5 /home/quikq/1.5/dev"
stage: deploy
environment:
name: DEV
only:
- DEV
DEV_BNADEV:
variables:
THING: "This is a THING for deploy_DEV_BNADEV"
REBUILD_DEPLOYMENT: "0"
ASPGRP: "DATADEV"
script:
- "bash ./GitLabCI/GitLabCI.Deploy.sh BNADEV EPDEV1_5 /home/quikq/1.5/dev"
stage: deploy
environment:
name: DEV
only:
- DEV
QA_QQDEV:
variables:
THING: "This is a THING for deploy_QA_QQDEV"
ASPGRP: "*NONE"
script:
- "bash ./GitLabCI/GitLabCI.Deploy.sh QQDEV EPQA1_5 /home/quikq/1.5/qa"
stage: deploy
environment:
name: QA
only:
- QA
QA_BNADEV:
variables:
THING: "This is a THING for deploy_QA_BNADEV"
REBUILD_DEPLOYMENT: "0"
ASPGRP: "DATADEV"
script:
- "bash ./GitLabCI/GitLabCI.Deploy.sh BNADEV EPQA1_5 /home/quikq/1.5/qa"
stage: deploy
environment:
name: QA
only:
- QA
UAT_QQ:
variables:
THING: "This is a THING for deploy_UAT_QQ"
ASPGRP: "*NONE"
script:
- "bash ./GitLabCI/GitLabCI.Deploy.sh QQ EPUAT1_5 /home/quikq/1.5/uat"
stage: deploy
environment:
name: UAT
only:
- UAT
UAT_QQBNA:
variables:
THING: "This is a THING for deploy_UAT_QQBNA"
ASPGRP: "*NONE"
script:
- "bash ./GitLabCI/GitLabCI.Deploy.sh QQBNA EPUAT1_5 /home/quikq/1.5/uat"
stage: deploy
environment:
name: UAT
only:
- UAT
PROD_QQ:
variables:
THING: "This is a THING for deploy_PROD_QQ"
ASPGRP: "*NONE"
script:
- "bash ./GitLabCI/GitLabCI.Deploy.sh QQ EPPROD1_5 /home/quikq/1.5/prod"
stage: deploy
environment:
name: PROD
only:
- PROD
PROD_QQBNA:
variables:
THING: "This is a THING for deploy_PROD_QQBNA"
ASPGRP: "*NONE"
script:
- "bash ./GitLabCI/GitLabCI.Deploy.sh QQBNA EPPROD1_5 /home/quikq/1.5/prod"
stage: deploy
environment:
name: PROD
only:
- PROD
Related
prod is configured as a protected environment. After approval of deploy job I would like to run tests automatically to verify successful deployment.
deploy:
stage: deploy
script:
- echo "deploy"
environment:
name: prod
action: start
test:
stage: test
script:
- echo "test"
environment:
name: prod
action: verify
when: on_success
The .gitlab.ci template above will generate the following pipeline:
After approving the deployment to prod the test job still require a manual action. Why is that? How do you run automated tests using environment and action: verify?
Removing the environment section from the test job gives the desired funcitonality, but this defeats the purpose of action: verify?
deploy:
stage: deploy
script:
- echo "deploy"
environment:
name: prod
action: start
test:
stage: test
script:
- echo "test"
when: on_success
I'm developing a pipeline on GitLab-ci, in the first job I use gittools/gitversion the obtain the semantic version of my software.
Here a small piece of code of /gitversion-ci-cd-plugin-extension.gitlab-ci.yml (Full documentation here https://gitversion.net/docs/reference/build-servers/gitlab)
.gitversion_function:
image:
name: gittools/gitversion
entrypoint: ['']
stage: .pre
.....
.....
artifacts:
reports:
#propagates variables into the pipeline level
dotenv: thisversion.env
Then a simplified version of my pipeline is as follows
stages:
- .pre
- install_dependencies
- build
- deploy
include:
- local: '/gitversion-ci-cd-plugin-extension.gitlab-ci.yml'
determineversion:
extends: .gitversion_function
install_dependencies:
image: node:16.14
stage: install_dependencies
script:
- echo ${PACKAGE_VERSION}
build:
image: node:16.14
stage: build
script:
- echo $PACKAGE_VERSION
deploy:
image: bitnami/kubectl
stage: deploy
needs: ['build']
script:
- echo $PACKAGE_VERSION
The problem is that the environment variable $PACKAGE_VERSION works in the first two jobs install_dependencies and build.
echo $PACKAGE_NAME; //0.0.1
But when the jobs deploy is executed the environment variable is not expanded by pipeline and I obtain literally this
echo $PACKAGE_NAME; //$PACKAGE_NAME
I found the problem.
In the last job of my pipeline, I use needs (https://docs.gitlab.com/ee/ci/yaml/#needs) to establish dependencies between jobs.
The problem is that artifact is not automatically passed because there is no a dependency between determineversion and deploy, to fix I do this:
...
deploy:
image: bitnami/kubectl
stage: deploy
needs: ['determineversion', 'build'] # <------
script:
- echo $PACKAGE_VERSION
...
I added determineversion as a dependency of deploy, in this way $PACKAGE_VERSION is printed correctly
I have like this gitlab ci cd configuration file:
image: docker:git
stages:
- develop
- production
default:
before_script:
- apk update && apk upgrade && apk add git curl
deploy:
stage: develop
script:
- echo "Hello World"
backup:
stage: develop
when:
- manual
- on_success
remove:
stage: develop
when:
- delayed
- on_success
start_in: 30 minutes
In my case job deploy runs automaticaly and job backup must runs manually only when successfully completed job deploy. But in my case this configuration doesn't works and I get error with message:
Found errors in your .gitlab-ci.yml:
jobs:backup when should be one of:
on_success
on_failure
always
manual
delayed
How I can use multiple when option arguments in my case?
Basically you can't because when does not expect an array. You can work around it though with needs. But this solution does only work if you run your jobs in different stages.
image: docker:git
stages:
- deploy
- backup
- remove
deploy:develop:
stage: deploy
script:
- exit 1
backup:develop:
stage: backup
script:
- echo "backup"
when: manual
needs: ["deploy:develop"]
remove:develop:
stage: remove
script:
- echo "remove"
when: delayed
needs: ["backup:develop"]
start_in: 30 minutes
I currently have two jobs in my CI file which are nearly identical.
The first is for manually compiling a release build from any git branch.
deploy_internal:
stage: deploy
script: ....<deploy code>
when: manual
The second is to be used by the scheduler to release a daily build from develop branch.
scheduled_deploy_internal:
stage: deploy
script: ....<deploy code from deploy_internal copy/pasted>
only:
variables:
- $MY_DEPLOY_INTERNAL != null
This feels wrong to have all that deploy code repeated in two places. It gets worse. There are also deploy_external, deploy_release, and scheduled variants.
My question:
Is there a way that I can combine deploy_internal and scheduled_deploy_internal such that the manual/scheduled behaviour is retained (DRY basically)?
Alternatively: Is there is a better way that I should structure my jobs?
Edit:
Original title: Deploy job. Execute manually except when scheduled
You can use YAML anchors and aliases to reuse the script.
deploy_internal:
stage: deploy
script:
- &deployment_scripts |
echo "Deployment Started"
bash command 1
bash command 2
when: manual
scheduled_deploy_internal:
stage: deploy
script:
- *deployment_scripts
only:
variables:
- $MY_DEPLOY_INTERNAL != null
Or you can use extends keyword.
.deployment_script:
script:
- echo "Deployment started"
- bash command 1
- bash command 2
deploy_internal:
extends: .deployment_script
stage: deploy
when: manual
scheduled_deploy_internal:
extends: .deployment_script
stage: deploy
only:
variables:
- $MY_DEPLOY_INTERNAL != null
Use GitLab's default section containing a before_script:
default:
before_script:
- ....<deploy code>
job1:
stage: deploy
script: ....<code after than deploy>
job2:
stage: deploy
script: ....<code after than deploy>
Note: the default section fails to function as such if you try to execute a job locally with the gitlab-runner exec command - use YAML anchors instead.
We're using Gitlab-CI but we have some troubles to have review and production environments at the same time.
We have several stages in our .gitlab-ci.yml but here I'll focus on the deploy stage:
deploy:
stage: deploy
script:
- some commands
environment:
name: review/$CI_BUILD_REF_NAME
url: http://$CI_BUILD_REF_SLUG.$DEPLOY_SERVER
on_stop: stop_deploy
only:
- /^feature-[cw]\/.*$/
deploy:
stage: deploy
script:
- some other commands
environment:
name: production
only:
- prod
stop_deploy:
stage: deploy
variables:
GIT_STRATEGY: none
script:
- some clean commands
when: manual
environment:
name: review/$CI_BUILD_REF_NAME
action: stop
only:
- /^feature-[cw]\/.*$/
The issue is that the first job is not run on the branches whose name starts with feature-c/. However when removing the second job, the first job is run on those branches.
The job that deploys to production is correctly run when pushed to prod.
So why does the first job is not run when the second job is defined? Where does the conflict comes from?
Thanks!
The answer is quite simple; they can't have the same name :) Name one deploy-review and the other deploy-prod and its fixed.