Gitlab CI: create multiple builds from single commit - gitlab

My current gitlab configuration is very simple as below
stages:
- build
before_script:
- some commands here
build-after-commit:
stage: build
script:
- some command here
artifacts:
expire_in: 1 day
when: on_success
name: name here
paths:
- build/*.zip
I want to run build-after-commit part twice with different settings. I am expecting something like this
stages:
- build
before_script:
- some commands here
build-after-commit:
stage: build
script:
- some command here
artifacts:
expire_in: 1 day
when: on_success
name: name1 here
paths:
- build/*.zip
# run it again with different settings
stage: build
script:
- Different script here
artifacts:
expire_in: 1 day
when: on_success
name: name2 here
paths:
- build/*.zip
So basically, in the second run the script will be different and the name of the output file will be different. How can I do this?

The straightforward approach would be to just have another job in the build stage.
E.g.
stages:
- build
before_script:
- some commands here
build-after-commit:
stage: build
script:
- some command here
artifacts:
expire_in: 1 day
when: on_success
name: name1 here
paths:
- build/*.zip
build-after-commit2:
stage: build
script:
- Different script here
artifacts:
expire_in: 1 day
when: on_success
name: name2 here
paths:
- build/*.zip
If you define build-after-commit2 in the same stage (build) it will even be run in parallel to build-after-commit.
In this case, I don't think having two jobs is bad design, as they are actually quite different from each other i.e. different script and different artifact name.

Related

Trigger GitLab pipeline to enviroment X only when a pipeline from environment Y finishes

Currently I have this script in my .gitlab-ci.yml file:
image: node:16
cache:
paths:
- .npm
- cache/Cypress
- node_modules
stages:
- build
- deploy
- test
install:dependencies:
stage: build
script:
- yarn install
artifacts:
paths:
- node_modules/
only:
- merge_requests
test:unit:
stage: test
script: yarn test --ci --coverage
needs: ["install:dependencies"]
artifacts:
when: always
paths:
- coverage
expire_in: 30 days
only:
- merge_requests
deploy-to-vercel:
stage: deploy
image: node:16
script:
- npm i -g vercel
- DEPLOYMENT_URL=$(vercel -t $VERCEL_TOKEN --confirm)
- echo $DEPLOYMENT_URL > vercel_deployment_url.txt
- cat vercel_deployment_url.txt
artifacts:
when: on_success
paths:
- vercel_deployment_url.txt
only:
- merge_requests
I need to trigger a pipeline to an environment called playground but only when a pipeline from test enviroment is finished, when a pipeline to master happens, I don't to mirror to the playground environment.
Everything is deployed to vercel, and the project is powered by Next JS.

The best way to run tests on different environments and different test sets using gitlab ci/cd

I have a project written in Java. Code is on Gitlab.
I would like to be able to run my tests (written in selenium + testng) in gitlab CI on different environments and different test suites (all.xml, login.xml, order.xml etc).
What is the best way to do this using gitab-ci.yml file? Currently I have the file configured as below and everything works. I can run both tests on DEV and PROD environments and different test suites. But the problem is a lot of duplicate code in the gitlab-ci.yml file. How to do it better?
My gitab-ci.yml file:
image: myImage/maven-chrome:jdk-17
stages:
- test
LOGIN_DEV:
stage: test
script:
- mvn $MAVEN_OPTS clean test -D surefire.suiteXmlFiles=login.xml -DTestEnvironment=DEV
artifacts:
paths:
- results
expire_in: 1 day
when: manual
when: manual
REGISTER_DEV:
stage: test
script:
- mvn $MAVEN_OPTS clean test -D surefire.suiteXmlFiles=register.xml -DTestEnvironment=DEV
artifacts:
paths:
- results
expire_in: 1 day
when: manual
when: manual
ALL_DEV:
stage: test
script:
- mvn $MAVEN_OPTS clean test -D surefire.suiteXmlFiles=all.xml -DTestEnvironment=DEV
artifacts:
paths:
- results
expire_in: 1 day
when: manual
when: manual
LOGIN_PROD:
stage: test
script:
- mvn $MAVEN_OPTS clean test -D surefire.suiteXmlFiles=login.xml -DTestEnvironment=PROD
artifacts:
paths:
- results
expire_in: 1 day
when: manual
when: manual
REGISTER_PROD:
stage: test
script:
- mvn $MAVEN_OPTS clean test -D surefire.suiteXmlFiles=register.xml -DTestEnvironment=PROD
artifacts:
paths:
- results
expire_in: 1 day
when: manual
when: manual
ALL_PROD:
stage: test
script:
- mvn $MAVEN_OPTS clean test -D surefire.suiteXmlFiles=all.xml -DTestEnvironment=PROD
artifacts:
paths:
- results
expire_in: 1 day
when: manual
when: manual

How can I build/deploy twice for a single branch?

I need to set up a demo server, which is a copy of the production server, but pointed at a different API. I want to run 2 separate build/deploy whenever the main branch is updated to accomplish this, as I need to run the demo build (Vue) to use different env variables pointing at the demo API (which will also need a dual deploy). Is this possible, and how would I go about it? Here's the existing:
stages:
- build
- deploy
- test
include:
- template: Security/SAST.gitlab-ci.yml
- template: Security/Secret-Detection.gitlab-ci.yml
build-main:
image: node:12
stage: build
only:
- main
script:
- yarn global add #quasar/cli
- rm package-lock.json
- yarn
- npm run build:prod
artifacts:
expire_in: 1 hour
paths:
- dist
deploy-main:
stage: deploy
only:
- main
script:
- echo $CI_PROJECT_DIR
- whoami
- sudo rsync -rav --exclude '.git' $CI_PROJECT_DIR/dist/spa/. /var/www/console
tags:
- deploy
build-beta:
image: node:12
stage: build
only:
- beta
script:
- yarn global add #quasar/cli
- rm package-lock.json
- yarn
- npm run build:beta
artifacts:
expire_in: 1 hour
paths:
- dist
deploy-beta:
stage: deploy
only:
- beta
script:
- echo $CI_PROJECT_DIR
- whoami
# - sudo /usr/local/bin/rsync -rav --exclude '.git' $CI_PROJECT_DIR/dist/spa/. /var/www/console.beta
- sudo rsync -rav --exclude '.git' $CI_PROJECT_DIR/dist/spa/. /var/www/console.beta
tags:
- deploy
build-dev:
image: node:12
stage: build
only:
- dev
script:
- yarn global add #quasar/cli
- rm package-lock.json
- yarn
- npm run build:dev
artifacts:
expire_in: 1 hour
paths:
- dist
deploy-dev:
stage: deploy
only:
- dev
script:
- echo $CI_PROJECT_DIR
- whoami
- sudo rsync -rav --exclude '.git' $CI_PROJECT_DIR/dist/spa/. /var/www/console.dev
tags:
- deploy
sast:
stage: test
artifacts:
reports:
sast: gl-sast-report.json
paths:
- 'gl-sast-report.json'
You can do something like the following ,that will create 2 build jobs and 2 deploy jobs, that are linked together using needs:
stages:
- build
- deploy
build-main:
stage: build
script: echo
only:
- main
artifacts:
expire_in: 1 hour
paths:
- dist
deploy-main:
stage: deploy
script: echo
only:
- main
needs:
- job: build-main
artifacts: true
build-demo:
stage: build
script: echo
only:
- main
artifacts:
expire_in: 1 hour
paths:
- dist
deploy-demo:
stage: deploy
script: echo
only:
- main
needs:
- job: build-demo
artifacts: true
you might also want to extract common jobs to hidden jobs to simplify your pipeline, for instance:
stages:
- build
- deploy
.build:
stage: build
artifacts:
expire_in: 1 hour
paths:
- dist
build-main:
extends: .build
only: main
# other specific codes
Also you might want to improve readability and management of workflow rules like the following, which will centralize rules logics:
workflow:
rules:
- if: $CI_COMMIT_REF_NAME == $CI_DEFAULT_BRANCH
variables:
DEPLOY_PROD: "true"
DEPOLOY_DEMO: "true"
# Some other conditional variables
- if: $CI_COMMIT_REF_NAME == "dev"
variables:
DEPLOY_DEV: "true"
- when: always
build-main:
rules:
- if: $DEPLOY_PROD
# other jobs

Get artifacts from previous GIT jobs

I have a 3 stages in pipeline, each job in all 3 stages are creating a xml data files. These jobs which runs in parallel.
I want to merge all xml data file in 4th stage. Below is my yml code
stages:
- deploy
- test
- execute
- artifact
script:
- XYZ
artifacts:
name: datafile.xml
paths:
- data/
Problem: how i can collect all xmls from previous jobs to merge it? Files names are unique.
Here is a .gitlab-ci.yml file that collects artifacts into a final artifact (takes a file generated by earlier stages, and puts them all together).
The key is the needs attribute which takes the artifacts from the earlier jobs (with artifacts: true).
stages:
- stage_one
- stage_two
- generate_content
apple:
stage: stage_one
script: echo apple > apple.txt
artifacts:
paths:
- apple.txt
banana:
stage: stage_two
script: echo banana > banana.txt
artifacts:
paths:
- banana.txt
put_it_all_together:
stage: generate_content
needs:
- job: apple
artifacts: true
- job: banana
artifacts: true
script:
- cat apple.txt banana.txt > fruit.txt
artifacts:
paths:
- fruit.txt

GitLab CI: Two independent scheduled jobs

Consider the following gilab-ci.yml script:
stages:
- build_for_ui_automation
- independent_job
variables:
LC_ALL: "en_US.UTF-8"
LANG: "en_US.UTF-8"
before_script:
- gem install bundler
- bundle install
build_for_ui_automation:
dependencies: []
stage: build_for_ui_automation
artifacts:
paths:
- fastlane/screenshots
- fastlane/logs
- fastlane/test_output
- fastlane/report.xml
script:
- bundle exec fastlane ui_automation
tags:
- ios
only:
- schedules
allow_failure: false
# This should be added and trigerred independently from "build_for_ui_automation"
independent_job:
dependencies: []
stage: independent_job
artifacts:
paths:
- fastlane/screenshots
- fastlane/logs
- fastlane/test_output
- fastlane/report.xml
script:
- bundle exec fastlane independent_job
tags:
- ios
only:
- schedules
allow_failure: false
I'd like to be able to schedule these two jobs independently, but following the rules:
build_for_ui_automation runs every day at 5 AM
independent_job runs every day at 5 PM
However, with the current setup I can only trigger the whole pipeline, which will go through both jobs in sequence.
How can I have a schedule triggering only a single job?
To build off #Naor Tedgi's answer, you can define a variable in your pipeline schedules. For example, set SCHEDULE_TYPE = "build_ui" in the schedule for build_for_ui_automation and SCHEDULE_TYPE = "independent" in the schedule for independent_job. Then your .gitlab-ci.yml file could be modified as:
stages:
- build_for_ui_automation
- independent_job
variables:
LC_ALL: "en_US.UTF-8"
LANG: "en_US.UTF-8"
before_script:
- gem install bundler
- bundle install
build_for_ui_automation:
dependencies: []
stage: build_for_ui_automation
artifacts:
paths:
- fastlane/screenshots
- fastlane/logs
- fastlane/test_output
- fastlane/report.xml
script:
- bundle exec fastlane ui_automation
tags:
- ios
only:
refs:
- schedules
variables:
- $SCHEDULE_TYPE == "build_ui"
allow_failure: false
# This should be added and trigerred independently from "build_for_ui_automation"
independent_job:
dependencies: []
stage: independent_job
artifacts:
paths:
- fastlane/screenshots
- fastlane/logs
- fastlane/test_output
- fastlane/report.xml
script:
- bundle exec fastlane independent_job
tags:
- ios
only:
refs:
- schedules
variables:
- $SCHEDULE_TYPE == "independent"
allow_failure: false
where note the syntax change in the only sections to execute the jobs only during schedules and when the schedule variable is matching.
in gitlab inside your project go to
CI/CD -> Schedules
press new Schedule button config the tasks as you want set the time and interval
now at the end add a variable for each one of them
now edit your gitlab.yml by adding that variable at the only section
as shown here below
https://docs.gitlab.com/ee/ci/variables/#environment-variables-expressions

Resources