Consider the following gilab-ci.yml script:
stages:
- build_for_ui_automation
- independent_job
variables:
LC_ALL: "en_US.UTF-8"
LANG: "en_US.UTF-8"
before_script:
- gem install bundler
- bundle install
build_for_ui_automation:
dependencies: []
stage: build_for_ui_automation
artifacts:
paths:
- fastlane/screenshots
- fastlane/logs
- fastlane/test_output
- fastlane/report.xml
script:
- bundle exec fastlane ui_automation
tags:
- ios
only:
- schedules
allow_failure: false
# This should be added and trigerred independently from "build_for_ui_automation"
independent_job:
dependencies: []
stage: independent_job
artifacts:
paths:
- fastlane/screenshots
- fastlane/logs
- fastlane/test_output
- fastlane/report.xml
script:
- bundle exec fastlane independent_job
tags:
- ios
only:
- schedules
allow_failure: false
I'd like to be able to schedule these two jobs independently, but following the rules:
build_for_ui_automation runs every day at 5 AM
independent_job runs every day at 5 PM
However, with the current setup I can only trigger the whole pipeline, which will go through both jobs in sequence.
How can I have a schedule triggering only a single job?
To build off #Naor Tedgi's answer, you can define a variable in your pipeline schedules. For example, set SCHEDULE_TYPE = "build_ui" in the schedule for build_for_ui_automation and SCHEDULE_TYPE = "independent" in the schedule for independent_job. Then your .gitlab-ci.yml file could be modified as:
stages:
- build_for_ui_automation
- independent_job
variables:
LC_ALL: "en_US.UTF-8"
LANG: "en_US.UTF-8"
before_script:
- gem install bundler
- bundle install
build_for_ui_automation:
dependencies: []
stage: build_for_ui_automation
artifacts:
paths:
- fastlane/screenshots
- fastlane/logs
- fastlane/test_output
- fastlane/report.xml
script:
- bundle exec fastlane ui_automation
tags:
- ios
only:
refs:
- schedules
variables:
- $SCHEDULE_TYPE == "build_ui"
allow_failure: false
# This should be added and trigerred independently from "build_for_ui_automation"
independent_job:
dependencies: []
stage: independent_job
artifacts:
paths:
- fastlane/screenshots
- fastlane/logs
- fastlane/test_output
- fastlane/report.xml
script:
- bundle exec fastlane independent_job
tags:
- ios
only:
refs:
- schedules
variables:
- $SCHEDULE_TYPE == "independent"
allow_failure: false
where note the syntax change in the only sections to execute the jobs only during schedules and when the schedule variable is matching.
in gitlab inside your project go to
CI/CD -> Schedules
press new Schedule button config the tasks as you want set the time and interval
now at the end add a variable for each one of them
now edit your gitlab.yml by adding that variable at the only section
as shown here below
https://docs.gitlab.com/ee/ci/variables/#environment-variables-expressions
Related
Currently I have this script in my .gitlab-ci.yml file:
image: node:16
cache:
paths:
- .npm
- cache/Cypress
- node_modules
stages:
- build
- deploy
- test
install:dependencies:
stage: build
script:
- yarn install
artifacts:
paths:
- node_modules/
only:
- merge_requests
test:unit:
stage: test
script: yarn test --ci --coverage
needs: ["install:dependencies"]
artifacts:
when: always
paths:
- coverage
expire_in: 30 days
only:
- merge_requests
deploy-to-vercel:
stage: deploy
image: node:16
script:
- npm i -g vercel
- DEPLOYMENT_URL=$(vercel -t $VERCEL_TOKEN --confirm)
- echo $DEPLOYMENT_URL > vercel_deployment_url.txt
- cat vercel_deployment_url.txt
artifacts:
when: on_success
paths:
- vercel_deployment_url.txt
only:
- merge_requests
I need to trigger a pipeline to an environment called playground but only when a pipeline from test enviroment is finished, when a pipeline to master happens, I don't to mirror to the playground environment.
Everything is deployed to vercel, and the project is powered by Next JS.
I have some stages and jobs linked to it.
install-rpm:
stage: deployrpm
allow_failure: true
extends:
- .iftestawsdeploy
image:
name: amazon/aws-cli
entrypoint: [""]
variables:
terraform_version: 1.1.7
terragrunt_version: 0.36.6
kubectl_version: 1.21.11
before_script:
- *install_dependencies
- *install_myproject_rpm
script:
- *run_myproject_deploy
- exit 0
after_script:
- *forward_myproject_resources
- ls dist1
- exit 0
artifacts:
paths:
- ./dist1
tags:
- ec2runner
cleanup-aws:
stage: cleanup
allow_failure: true
extends:
- .iftestawsdeploy
image:
name: amazon/aws-cli
entrypoint: [""]
variables:
terraform_version: 1.1.7
terragrunt_version: 0.36.6
kubectl_version: 1.21.11
before_script:
- *install_dependencies
- *install_myproject_rpm
- *use_myproject_resources
script:
- *teardown_myproject
after_script:
- *nuke_SUB_AWS_resources
tags:
- dev
I want to set an option before the job cleanupaws above or the cleanup stage so that when previous stages completed, this job/stage should wait till someone clicks on continue or approve button.
So that, we can do our R&D tasks if any failure in deployment and later we can resume cleanup tasks.
Please suggest how to setup that.
Create a stage that preceeds the stage you want to be started manually and create a single manual job in that stage.
stages:
# ...
- cleanup_approval
- cleanup
manual_cleanup:
# this job must be run manually to allow cleanup stage to run
stage: cleanup_approval
when: manual
script:
- echo noop
allow_failure: false
cleanup-aws:
stage: cleanup
# runs after `manual_cleanup` job in the previous stage has been manually run
# ...
I can't find a satisfying solution for my case.
I want to start a job manually only when a certain previous job has failed. The job in question dose a validation. I want to make the next job manual so that the user acknowledges that something wasn't good and make him investigate the problem and continue only if he deems that the fail can be ignored.
stages:
- test
- validate
- build
lint:
stage: test
allow_failure: true
script:
- npm run lint
check:reducer:
stage: test
allow_failure: true
script:
- chmod +x ./check-reducers.py
- ./check-reducers.py $CI_PROJECT_ID $CI_COMMIT_BRANCH
except:
- master
- development
fail:pause:
stage: validate
allow_failure: true
script:
- echo The 'validate:reducer' job has failed
- echo Check the job and decide if this should continue
when: manual
needs: ["check:reducer"]
build:
stage: build
script:
- cp --recursive _meta/ $BUILD_PATH
- npm run build
artifacts:
name: "build"
expire_in: 1 week
paths:
- $BUILD_PATH
needs: ["fail:pause"]
I would like that if check:reducer fails, fail:pause to wait for the user input. If check:reducer exits with 0, fail:pause should start automatically or buildshould start.
Unfortunately, this isn't possible as the when keyword is evaluated at the very start of the pipeline (I.e., before any job execution has run), so you cannot set the when condition based on the previous job status.
This is possible if you use a generated gitlab-ci.yml as a child workflow.
stages:
- test
- approve
- deploy
generate-config:
stage: test
script:
- ./bin/run-tests.sh
- ./bin/generate-workflows.sh $?
artifacts:
paths:
- deploy-gitlab-ci.yml
trigger-workflows:
stage: deploy
trigger:
include:
- artifact: deploy-gitlab-ci.yml
job: generate-config
The generate-workflows.sh script writes out a deploy-gitlab-ci.yml that either has the approval job or not based on the return code of the run-test.sh passed as the first argument to the script.
You can make it easier on yourself using includes, where you either include the approve step or not in the generated deploy-gitlab-ci.yml file, and make the steps in the deploy optionally need the approal.
approve-gitlab-ci.yml
approve:
stage: approve
when: manual
script:
- echo "Approved!"
deploy-gitlab-ci.yml
deploy:
stage: deploy
needs:
- job: approve
- optional: true
Then the deploy-gitlab-ci.yml is simply an includes with the jobs to run:
includes:
- approve-gitlab-ci.yml
- deploy-gitlab-ci.yml
I have a .gitlab-ci.yml that looks like this:
image: "python:3.7"
.python-tag:
tags:
- python
before_script:
- python --version
- pip install -r requirements.txt
- export PYTHONPATH=${PYTHONPATH}:./src
- python -c "import sys;print(sys.path)"
stages:
- Static Analysis
- Local Tests
- Integration Tests
- Deploy
mypy:
stage: Static Analysis
extends:
- .python-tag
script:
- mypy .
pytest-smoke:
stage: Local Tests
extends:
- .python-tag
script:
- pytest -m smoke
int-tests-1:
stage: Integration Tests
when: manual
allow_failure: false
trigger:
project: tests/gitlab-integration-testing-integration-tests
strategy: depend
int-tests-2:
stage: Integration Tests
when: manual
allow_failure: false
trigger:
project: tests/gitlab-integration-testing-integration-tests
strategy: depend
deploy:
stage: Deploy
extends:
- .python-tag
script:
- echo "Deployed!"
The Integrations stage has multiple jobs in it that take a decent chunk of time to run. It is unusual that all of the integration tests need to be run. This is why we stuck a manual flag on these, and the specific ones needed will be manually run.
How do I make it so that the Deploy stage requires that one or more of the jobs in Integration Tests has passed? I can either do all like I have now or I can do none by removing allow_failure: false from the integration test jobs.
I want to require that at least once has passed.
if each job generate an artifcact only when the job is successful
artifacts:
paths:
- success.txt
script:
# generate the success.txt file
you should be able to test if the file exist in the next stage
you need to add this (below) in the next stage to be able to read the file:
artifacts:
paths:
- success.txt
My current gitlab configuration is very simple as below
stages:
- build
before_script:
- some commands here
build-after-commit:
stage: build
script:
- some command here
artifacts:
expire_in: 1 day
when: on_success
name: name here
paths:
- build/*.zip
I want to run build-after-commit part twice with different settings. I am expecting something like this
stages:
- build
before_script:
- some commands here
build-after-commit:
stage: build
script:
- some command here
artifacts:
expire_in: 1 day
when: on_success
name: name1 here
paths:
- build/*.zip
# run it again with different settings
stage: build
script:
- Different script here
artifacts:
expire_in: 1 day
when: on_success
name: name2 here
paths:
- build/*.zip
So basically, in the second run the script will be different and the name of the output file will be different. How can I do this?
The straightforward approach would be to just have another job in the build stage.
E.g.
stages:
- build
before_script:
- some commands here
build-after-commit:
stage: build
script:
- some command here
artifacts:
expire_in: 1 day
when: on_success
name: name1 here
paths:
- build/*.zip
build-after-commit2:
stage: build
script:
- Different script here
artifacts:
expire_in: 1 day
when: on_success
name: name2 here
paths:
- build/*.zip
If you define build-after-commit2 in the same stage (build) it will even be run in parallel to build-after-commit.
In this case, I don't think having two jobs is bad design, as they are actually quite different from each other i.e. different script and different artifact name.