I have a template file which has few stages like:
template.yaml
stages:
- a2
- a3
Now, I have another file .gitlab-ci.yaml which should include stages from above template file, but also needs few more stages:
include:
- project: 'path to repo'
ref: main
file: "/template.yml"
stages:
- a1
job1:
stage: a1
job2:
stage: a2
I get the following error:
chosen stage does not exist; available stages are .pre, a1, .post
Is it possible to do something similar without making any changes in the template file ?
Related
Suppose we have a .gitlab-ci.yml file that reads in a common.gitlab-ci.complete.yml file:
include: common.gitlab-ci.complete.yml
Suppose that common.gitlab-ci.complete.yml has the following stages:
stages:
- build
- unit-test
- mutation-test
In the .gitlab-ci.yml file, how do we ignore the unit-test and mutation-test stages? Would it be something like:
include: common.gitlab-ci.complete.yml
stages:
- build
#- unit-test
#- mutation-test
Would these stages override the one in the common.gitlab-ci.complete.yml file?
(I have not yet checked this code for accuracy. Treat it as pseudocode.)
You're asking to remove all jobs in a stage. If you look at the documentation for the include: keyword, you'll note that the YAML is always merged with your current CI script. So no, you can't just add a statement like "Disable all jobs in stage A".
Here's an idea... recall that
If a stage is defined but no jobs use it, the stage is not visible in the pipeline...
https://docs.gitlab.com/ee/ci/yaml/#stages
So what you could do is add a mechanism for disabling all jobs in that stage. Then the stage will disappear as well. That mechanism might be checking a variable in rules.
So if you wrote the common.gitlab-ci.complete.yml with templates that check a variable to disable the job...
stages:
- build
- unit-test
- package
.unit-test-stage:
stage: unit-test
when: on_success
rules:
- if: $DISABLE_UNIT_TESTS
when: never
unit-test-job-a:
extends: .unit-test-stage
script:
- echo "do stuff"
unit-test-job-b:
extends: .unit-test-stage
script:
- echo "do more stuff"
Then maybe your top-level .gitlab-ci.yml would simply set that known variable in its global variables section like this:
include: common.gitlab-ci.complete.yml
variables:
DISABLE_UNIT_TESTS: 1
In Azure Devops, I have two individual yaml pipelines one for build and another for release. I would want to link the build artifact in the yaml release pipeline ,like the one we have for classic pipelines. Is there anyway we can achieve the same for yaml release pipelines.
198982-release.png
I'm from the Microsoft for Founders Hub team. If you want to be able to access your artifact across different stages in your pipeline, you can use output variables to pass it to the next stage in your YAML. Please follow the below script.
trigger: none
pool:
vmImage: 'ubuntu-latest'
stages:
- stage: A
jobs:
- job: A1
steps:
- script: echo ##vso[task.setvariable variable=shouldrun;isOutput=true]true
name: printvar
- stage: B
dependsOn: A
jobs:
- job: B1
condition: in(stageDependencies.A.A1.result, 'Succeeded', 'SucceededWithIssues', 'Skipped')
steps:
- script: echo hello from Job B1
- job: B2
condition: eq(stageDependencies.A.A1.outputs['printvar.shouldrun'], 'true')
steps:
- script: echo hello from Job B2
I am unable to find a solution for how to ignore changes made in .gitlab-ci.yml to trigger a job. So far I have tried the below options:
except:
changes:
- .gitlab-ci.yml
and
only
- Branch A
but every time i make changes in .gitlab.ci-yml file, jobs for Stage B get added in pipeline and show as skipped.
Below are the jobs defined in .gitlab-ci.yml. Do you have any suggestion here?
I do not want Stage B jobs get added in pipeline when:
i) push made against the .gitlab-ci.yml (either manual changing file or git push command)
ii) any merge request for .gitlab-ci.yml
stages:
- A
- B
Stage A:
stage: A
script:
- echo "TEST"
rules:
- if: '$CI_COMMIT_TAG =~ /^\d+\.\d+\.DEV\d+/'
tags:
- runner
Stage B:
stage: B
script:
- echo "TEST"
when: manual
tags:
- runner
With this setup the Stage B is not added if .gitlab-ci.yml is modified:
stages:
- A
- B
Stage A:
stage: A
script:
- echo "Stage A"
tags:
- runner
Stage B:
stage: B
script:
- echo "Stage B"
rules:
- changes:
- ".gitlab-ci.yml"
when: never
- when: manual
tags:
- runner
Otherwise Stage B is showed in the pipeline and can be run manually. Tested with GitLab CI CE 14.1.0.
Could you try using workflow rule? It should determine if the pipeline is created.
P.S: Someone complained a couple of years ago about not being able to activate manual jobs after the exception, but it looks like it was a bug. I can't find the issue mentioned in the post
Edit:
This conf skips any commit with changes README.md or .gitlab-ci.yml:
workflow:
rules:
- if:
changes:
- README.md
- .gitlab-ci.yml
when: never
I have a repo QA/tests which I want to run all the jobs when there is a push to this repo.
I used a script to generate the jobs dynamically:
job-generator:
stage: generate
tags:
- kuber
script:
- scripts/generate-job.sh > generated-job.yml
artifacts:
paths:
- generated-job.yml
main:
trigger:
include:
- artifact: generated-job.yml
job: job-generator
strategy: depend
At the next step, I have another repo products/first which I want to run a specific job in QA/tests at every push in the products/first so I tried:
stages:
- test
tests:
stage: test
variables:
TARGET: first
trigger:
project: QA/tests
branch: master
strategy: depend
Then I tried to define a global TARGET: all variable in my main gitlab-ci.yml and override it with the TARGET: first in the above YAML.
generate-job.sh:
#!/bin/bash
PRODUCTS=("first" "second" "third")
for P in "${PRODUCTS[#]}"; do
cat << EOF
$P:
stage: test
tags:
- kuber
script:
- echo -e "Hello from $P"
rules:
- if: '"$TARGET" == "all"'
when: always
- if: '"$TARGET" == $P'
when: always
EOF
done
But no results. the downstream pipeline doesn't have any job at all!
Any idea?
I am not sure if this is now helpful, but this looks like an over complicated approach from the outside. I have to say i have limited knowledge and my answer is based on assumption:
the QA/tests repository contains certain test cases for all repositories
QA/tests has the sole purpose of containing the tests, not an overview over the projects etc.
My Suggestion
As QA/tests is only containing tests which should be executed against each project, i would create a docker image out of it which is contains all the tests and can actually execute them. (lets calls it qa-tests:latest)
Within my projects i would add a step which uses this images, with my source code of the project and executes the tests:
qa-test:
image: qa-tests:latest
script:
- echo "command to execute scripts"
# add rules here accordingly
this would solve the issue with each push into the repositories. For an easier usage, i could create a QA-Tests.gitlab-ci.yml file which can be included by the sub-projects with
include:
- project: QA/tests
file: QA-Tests.gitlab-ci.yml
this way you do not need to do updates with in the repositories if the ci snippet changes.
Finally to trigger the execution on each push, you only need to trigger the pipelines of the subprojects from the QA/tests.
Disclaimer
As i said, i have only a limited few, as the goal is described but not the motivation. With this approach you remove some of the directive calls - mainly the ones triggering from sub projects to QA/tests. And it generates a clear structure, but it might not fit your needs.
I solved it with:
gitlab-ci.yml:
variables:
TARGET: all
job-generator:
stage: generate
tags:
- kuber
script:
- scripts/generate-job.sh > generated-job.yml
artifacts:
paths:
- generated-job.yml
main:
variables:
CHILD_TARGET: $TARGET
trigger:
include:
- artifact: generated-job.yml
job: job-generator
strategy: depend
and use CHILD_TARGET in my generate-job.sh:
#!/bin/bash
PRODUCTS=("first" "second" "third")
for P in "${PRODUCTS[#]}"; do
cat << EOF
$P:
stage: test
tags:
- kuber
script:
- echo -e "Hello from $P"
rules:
- if: '\$CHILD_TARGET == "all"'
when: always
- if: '\$CHILD_TARGET == "$P"'
when: always
EOF
done
So I could call it from other projects like this:
stages:
- test
e2e-tests:
stage: test
variables:
TARGET: first
trigger:
project: QA/tests
branch: master
strategy: depend
I built a .gitlab-ci-yaml which looks like this and triggers two child-pipelines:
stages:
- prepare
- triggers
include: 'global-gitlab-ci.yaml'
...
frontend:
stage: triggers
trigger:
include: frontend-gitlab-ci.yaml
backend:
stage: triggers
trigger:
include: backend-gitlab-ci.yaml
the child-pipelines both look like this:
stages:
- build
- test
include: 'global-gitlab-ci.yaml'
test_frontend:
stage: test
image: ...
script:
- ...
build_frontend:
stage: build
image: ...
script:
- ...
When I run this pipelines, I get the following:
The frontend-pipeline (as well as the backend-pipeline) only shows one job - the second one is ignored.
What's the matter? Is the child-pipline not supposed to contain a complete pipeline with several jobs?
(Btw global-gitlab-ci.yaml just contains default: definitions)