Run jobs in parallel in .gitlab.ci.yml - gitlab

Question

Basically I currently have 5 test licenses on a server. I have a pipeline that runs test scripts when I trigger it manually. It connects to the license server and acquires a floating license . This approach is good for now but soon I will be wanting to expand this so that when the application needs testing, I can run multiple pipelines in parallel to have multiple tests running. Here's the thing, I'm wanting to only sometimes run them in parallel based on what I need to test and I would want to manually trigger each pipeline. For example, one day I might only want to run the tests scripts on one job so this requires one pipeline running. Another day I might want to run 3 jobs at the same time or another day I may want to run 5 jobs throughout the day but may overlap with a running pipeline so it needs to run its own pipeline.
My question is, how do I go about setting this up in a gitlab yml file.
If anyone could also provide a basic example that would be helpful!

Answer 1

As a general rule, any job within the same stage is run in parallel. Within those jobs, you can define rules that specify when a given job runs. Note that needing to manually kick off jobs will cause your overall pipeline to be listed as "blocked" when the manual jobs are reached. Similarly, you'll need to set some jobs as allow_failure: true or they will block the next stage from executing. Example pipeline:
stages:
- build
- test
build_job:
image: alpine:latest
stage: build
script:
- echo "hello world"
test_job_1:
image: alpine:latest
stage: test
rules:
- when: manual
script:
- echo "test 1"
test_job_2:
image: alpine:latest
stage: test
rules:
- when: manual
script:
- echo "test 2"
When you run the above pipeline, you will have to manually click "play" on both jobs to start them.
I will note though, that this feels like an anti-pattern. It sounds like you want to conserve your 5 test licenses to make sure that you don't, for example, have 6 jobs running and have a job fail due to no free licenses. If so, this is one of the exact use-cases that the resource_group keyword is meant to address. Jobs added to the same resource group do not run concurrently (even if they are in different pipelines), so you could have a test_1 resource group, test_2 resource group, etc, and those jobs would always run in parallel automatically, but would never run more than one instance of that job at once even across different pipelines. This allows you ensure you only use 5 licenses, but your tests still run automatically and you don't need to manually trigger them, which also allows you to more easily define downstream jobs to run when your tests pass.