fail gitlab multijob pipeline if test job fails in another repository - gitlab

I have a gitlab ci pipeline in my application repo, A, which calls an end to end testing Repo T to run its tests. The repo A pipeline succesfully triggers the tests from Repo T but if the test job fails in T, the job calling the test job in T from A still passes. How do I get repo A to track the result of Repo T's test job, and pass/fail its pipeline based off of the test jobs in T?
.gitlab-ci.yml for testing Repo T:
stages:
- test
test:
stage: test
image: markhobson/maven-chrome:jdk-11
artifacts:
paths:
- target/surefire-reports
script:
- mvn test
only:
- triggers
.gitlab-ci.yml from application repo A:
job1:
stage: unit-tests ...
job2:
stage: build ...
...
trigger-e2e-repo:
stage: e2e-testing
image: markhobson/maven-chrome
script:
- "curl -X POST -F token=repo-T-token -F ref=repo-T-branch https://repo-A/api/v4/projects/repo-T-id/trigger/pipeline"
only:
- repo-A-branch

Since GitLab 11.8 you can trigger a pipeline via bridge job.
In GitLab 11.8, GitLab provides a new CI/CD configuration syntax to make this task easier, and avoid needing GitLab Runner for triggering cross-project pipelines.
With bridge jobs it is possible to mirror the status of the trigger pipeline to the calling pipeline.
You can mirror the pipeline status from the triggered pipeline to the source bridge job by using strategy: depend.
Example in your case:
trigger-e2e-repo:
stage: e2e-testing
trigger:
project: repo-T
strategy: depend
If the triggered pipeline with the test jobs fails, the calling pipeline also fails.
If you only want to execute a particular job in your repository "Repo T" when executed by a bridge job, then you should use only: pipeline (only) or rules: -if '$CI_PIPELINE_SOURCE == "pipeline"' (rules:if) instead of only: triggers.

I wasn't able to use the bridge job property of mirroring a downstream job result as the version of my gitlab is before 11.8. I did manage to get it to work by creating a trigger for repo A, and making a call from repo T to repo A with this new second trigger. The remaining jobs in repo A will only be activated by triggers and setting of variables (JOB in this case ) as laid out below:
.gitlab-ci.yml for repo T:
test:
stage: test
script:
- mvn test
- "curl -X POST -F token=repo-A-token -F ref=$BRANCH -F variables[JOB]=build https://project.com/api/v4/projects/project_id/trigger/pipeline"
.gitlab-ci.yml in A
job1:
...
except:
- triggers
job2:
...
except:
- triggers
...
trigger-e2e-repo:
stage: e2e-testing
script:
- "curl -X POST -F token=repo-B-token -F ref=repo-B-branch -F variables[BRANCH]=$CI_COMMIT_REF_NAME -F https://project-B/api/v4/projects/project-B-id/trigger/pipeline"
except:
- triggers
build_application_for_prod:
stage: build_prod
script:
- "curl -X POST -F token=repo-A-token -F ref=$CI_COMMIT_REF_NAME -F variables[JOB]=deploy -F variables[SEND]=true https://foo/api/v4/projects/proj_A_id/trigger/pipeline"
only:
variables:
- $JOB == "build"
deploy_production_environment:
stage: deploy_prod
script:
- ...
only:
variables:
- $JOB == "deploy"
Note I also had to add the except statements for the jobs before the end to end tests in repo A so that they won't rerun and loop when repo A's API trigger is called later on.

Related

GitLab Pipeline error using extends keyword

I got an error on GitLab Pipeline when I commit the next .gitlab-ci.yml for a repository.
Pipeline executed to build solution, deploy to Artifactory and trigger and API call
Deploy job have to be executed manually, and there are two different job options to execute.
stages:
- build
- deploy
- trigger
variables:
APP_PROJECT_ID: ${CUSTOMER_RELEASED}
build_job:
stage: build
tags:
- dotnet
script:
- echo "build"
only:
- tags
allow_failure: false
.deploy_job_base:
stage: deploy
needs: [build_job]
tags:
- dotnet
script:
- echo "deploy"
dependencies:
- build_job
only:
- tags
deploy_job_sport:
extends: .deploy_job_base
after_script:
- $APP_PROJECT_ID = "2096"
when: manual
allow_failure: false
deploy_job_others:
extends: .deploy_job_base
after_script:
- $APP_PROJECT_ID = "0"
when: manual
allow_failure: false
.trigger_base:
stage: trigger
script:
- echo "Customer Project ID '{$APP_PROJECT_ID}'"
- echo "Call API..."
trigger_sport:
extends: .trigger_base
needs: [deploy_job_sport]
trigger_others:
extends: .trigger_base
needs: [deploy_job_others]
Lint status is correct
but I get that error GitLab Pipeline when I commit changes:
Found errors in your .gitlab-ci.yml: 'trigger_sport' job needs
'deploy_job_sport' job but 'deploy_job_sport' is not in any previous
stage 'trigger_others' job needs 'deploy_job_others' job but
'deploy_job_others' is not in any previous stage
If I remove trigger_sport and trigger_others job and create only one job, it works fine but I don't know how I can target the two manual jobs (deploy_job_sport and deploy_job_others) to a single job.
Do you have any idea?
Thanks in advance.
I think this is related to the fact that you're using only: tags in your template for deploy jobs and the build job also is limited to run when commits contain a tag.
But the trigger template is missing this limitation which most likely causes this error when pushing a commit without a tag because the pipeline creation would add trigger_XY to the pipeline, which has dependencies to the previous deploy_XY jobs.
When updating your job template for trigger jobs to the following this error should be resolved:
.trigger_base:
stage: trigger
script:
- echo "Customer Project ID '{$APP_PROJECT_ID}'"
- echo "Call API..."
only:
- tags

Gitlab CI: How do I use the environment variable from one stage as needs:project ref in another

I have two jobs in the same project: job A and job B.
job A creates an environment variable EXTERNAL_PROJ_REF=some_tag and exports it through a .env file.
job B needs to download artifacts from an external_project and package them with other artifacts from the current project. I want to be able to dynamically choose the commit reference from which these external artifacts get downloaded. I am trying to use the environment variable EXTERNAL_PROJ_REF as the ref for external_project needed by job B.
job A:
stage: build
script:
- echo "EXTERNAL_PROJ_REF=`./generate_variable.sh`" > build.env # evaluates to EXTERNAL_PROJ_REF=some_tag
artifacts:
reports:
dotenv: build.env
job B:
stage: package
script:
- ./do_packaging_job.sh
needs:
- job: job A
artifacts: true
- project: external_project
ref: $EXTERNAL_PROJ_REF
job: external_job
artifacts: true
When I run this pipeline though, job B instantly fails with the following error:
This job depends on other jobs with expired/erased artifacts:
If I hardcode ref to some_tag, the job does not fail, and I can confirm the EXTERNAL_PROJ_REF is successfully passed to job B.
job B:
stage: package
script:
- echo "Ref = $EXTERNAL_PROJ_REF" # Correctly prints "Ref = some_tag"
- ./do_packaging_job.sh
needs:
- job: job A
artifacts: true
- project: external_project
ref: some_tag # hardcoded so the job doesn't fail
job: external_job
artifacts: true
However, when I have ref:$EXTERNAL_PROJ_REF, the pipeline fails. Can somebody tell me if I'm missing something?
A simple solution I use for sharing variables between two GitLab CI jobs:
stages:
- build
- package
job A:
stage: build
script:
- echo "EXTERNAL_PROJ_REF=`./generate_variable.sh`" > build.env
artifacts:
paths:
- build.env
job B:
stage: package
before_script:
- export $(cat build.env | xargs)
script:
- ./do_packaging_job.sh
I finally realized Gitlab does not support what I want to do, at least not this way. According to this link, a variable passed from a different job can only be used in before_script, script or after_script sections of a job; it cannot be used to configure jobs. I cannot use it the needs section of job B.
Luckily, I found a simple workaround using the Gitlab API. I have API access to external_project, so I just use wget to download the artifact I need from the dynamically selected commit reference. Afterwards, I directly pass the artifact to job B.
job A:
stage: build
script:
# Dynamically create commit reference
- EXTERNAL_PROJ_REF=`./generate_commit_ref.sh`
# Download artifact with Gitlab API
- wget --header "PRIVATE-TOKEN:${EXTERNAL_PROJ_TOKEN}" --output-document outputFileName.file "${CI_API_V4_URL}/projects/${EXTERNAL_PROJ_ID}/jobs/artifacts/${EXTERNAL_PROJ_REF}/raw/${EXTERNAL_ARTIFACT_PATH}?job=${EXTERNAL_JOB_NAME}"
# btw CI_API_V4_URL is a gitlab defined variable
artifacts:
paths:
- outputFileName.file
job B:
stage: package
script:
- ./do_packaging_job.sh
needs:
# Now this packaging job only needs job A. It doesn't need the external job anymore
- job: job A
artifacts: true

Gitlab CI doesnt trigger pipelines on commit to the specified branch

I am trying to trigger a pipeline on any type of change/commit (push/merge/webIDE etc) to a specific branch called test branch. Not sure why it doesn't trigger at all even after I make a change to a file in that specific branch.
default:
tags:
- specs
stages:
- validate
Trigger job:
stage: validate
script:
- curl -X POST -F token=$ENTDV_PL_TRIGGER_TOKEN -F "ref=master" https://gitlab.com/api/v4/projects/1252972/trigger/pipeline
variables:
CI_DEBUG_TRACE: "true"
rules:
- if: '$CI_COMMIT_BRANCH == "testbranch"'
If you want Trigger job to only be run when changes are made on testbranch, you can modify your Trigger job to the following:
Trigger job:
stage: validate
script:
- curl -X POST -F token=$ENTDV_PL_TRIGGER_TOKEN -F "ref=master" https://gitlab.com/api/v4/projects/1252972/trigger/pipeline
variables:
CI_DEBUG_TRACE: "true"
only:
- testbranch
The Gitlab CI documentation is actually quite good. Here is a link to how "only" works: https://docs.gitlab.com/ee/ci/yaml/index.html#only--except
From the docs: "Use the only:refs and except:refs keywords to control when to add jobs to a pipeline based on branch names or pipeline types."
You can do some other neat things like specifying only merge_requests or using regex for triggering jobs and what not.

"Only" or "rules" keywords weirdly remove jobs from the CI pipeline

I am trying to use "rules" and "only" keywords to define my pipeline behaviors between merge requests, pushes into dev branch and pushes into master branch.
I noticed several weird behaviors in the Gitlab CI, let's see in my merge_requests pipelines.
With this gitlab-ci.yml file, without any rule, all the jobs are displayed and run.
image: "python:3.8"
stages:
- test_without_only_policy
- test_with_only_policy
test_without_only_policy:
stage: test_without_only_policy
when: manual
script:
- echo "Yay, I am in the pipeline"
test_with_only_policy:
stage: test_with_only_policy
script:
- echo "I am always in the pipeline"
Everything is working as expected, great 👍
With this gitlab-ci.yml file, without an "only" policy in the 2nd job, the 1st job without rules disappears.
image: "python:3.8"
stages:
- test_without_only_policy
- test_with_only_policy
test_without_only_policy:
stage: test_without_only_policy
when: manual
script:
- echo "No, I am not in the pipeline anymore"
test_with_only_policy:
stage: test_with_only_policy
only:
- merge_requests
script:
- echo "I am always in the pipeline"
Why did the 1st job without the rules or "only" policy disappear?
With this gitlab-ci.yml file, with a "rules" keyword in the 2nd job, the 1st job without rules disappears.
image: "python:3.8"
stages:
- test_without_only_policy
- test_with_only_policy
test_without_only_policy:
stage: test_without_only_policy
when: manual
script:
- echo "No, I am not in the pipeline anymore"
test_with_only_policy:
stage: test_with_only_policy
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
when: manual
script:
- echo "I am always in the pipeline"
Why did the 1st job without the rules or "only" policy disappear?
Thank you for your help, I don't understand why my job disappears when I add rules in other jobs.
According to the documentation for Merge Request Pipelines, if you have a pipeline where only some jobs use only/except/rules with merge_requests, only those jobs will be in the pipeline if it is a Merge Request pipeline. The other jobs will be left out.
Here's the example from the docs:
build:
stage: build
script: ./build
only:
- main
test:
stage: test
script: ./test
only:
- merge_requests
deploy:
stage: deploy
script: ./deploy
only:
- main
In this example, the only job that specifies it should run for Merge Request pipelines is the test job. For standard push pipelines, the build and deploy jobs will run, but when a new merge request is created, a change is pushed to a branch that is the source branch on an existing merge request, or you hit the Run Pipeline button on the Pipelines tab for a Merge Request, it will run a Merge Request pipeline.
Here's another example with a different scenario:
A:
stage: some_stage
only:
- branches
- tags
- merge_requests
script:
- script.sh
B:
stage: some_other_stage
only:
- branches
- tags
- merge_requests
script:
- script.sh
C:
stage: third_stage
only:
- merge_requests
script:
- script.sh
In this example, jobs A and B run for all pipeline types push, tags, merge_requests, etc., but job C only runs for merge_request pipelines.
That's why your test_without_only_policy job will never be in a pipeline where test_with_only_policy runs. test_with_only_policy specifically runs for Merge Request events, but test_without_only_policy does not.

Create 2 Pipelines for a Node Project in GitLab

I'm trying to run 2 pipelines for a project in GitLab, but I can't find any way to do it.
In gitlab CI you can't create multiple pipelines for one project explicitly. There are cases where multiple pipelines will run simultaneously, such as when you have jobs that run only for merge requests and other jobs that do not run on merge requests.
That said, there are ways to obtain the effect of running multiple series of jobs independently from one another.
The hacky way, before gitlab-ce 12.2
If you want to start 2 pipelines for the same project you can use pipeline triggers. This method is limited to 2 pipelines and gitlab CI is not meant to be used this way. Usually, triggers are used to start pipelines on another project.
All in your .gitlab-ci.yml:
stages:
- start
- build
###################################################
# First pipeline #
###################################################
start_other_pipeline:
stage: start
script:
# Trigger a pipeline for current project on current branch
- curl --request POST --form "token=$CI_JOB_TOKEN" --form ref=$CI_COMMIT_REF_NAME $CI_API_V4_URL/projects/$CI_PROJECT_ID/trigger/pipeline
except:
- pipelines
build_first_pipeline:
stage: build
script:
- echo "Building first pipeline"
except:
- pipelines
###################################################
# Second pipeline #
###################################################
# Will run independently of the first pipeline.
build_second_pipeline:
stage: build
script:
- echo "Building second pipeline"
only:
- pipelines
To clean up this mess of a .gitlab-ci.yml, you can use the include keyword:
# .gitlab-ci.yml
include:
- '/first-pipeline.yml'
- '/second-pipeline.yml'
stages:
- start
- build
# This starts the second pipeline. The first pipeline is already running.
start_other_pipeline:
stage: start
script:
# Trigger a pipeline for current project on current branch
- curl --request POST --form "token=$CI_JOB_TOKEN" --form ref=$CI_COMMIT_REF_NAME $CI_API_V4_URL/projects/$CI_PROJECT_ID/trigger/pipeline
except:
- pipelines
# first-pipeline.yml
build_first_pipeline:
stage: build
script:
- echo "Building first pipeline"
except:
- pipelines
# second-pipeline.yml
build_second_pipeline:
stage: build
script:
- echo "Building second pipeline"
only:
- pipelines
The reason this works is the use only and except in the jobs. The jobs marked with
except:
- pipelines
do not run when the pipeline has started because of a trigger coming from another pipeline, so they don't run in the second pipeline. On the other hand,
only:
- pipelines
does the exact opposite, therefore those jobs run only when the pipeline is triggered by another pipeline, so they only run in the second pipeline.
The probably right way, depending on your needs ;)
In gitlab CE 12.2, it is possible to define Directed Acyclic Graphs to specify the order that your jobs run. This way, a job can start as soon as the job it depends on (using needs) finishes.
As of GitLab 12.7 it is also possible to use parent-child pipelines for this:
# .gitlab-ci.yml
trigger_child:
trigger:
include: child.yml
do_some_stuff:
script: echo "doing some stuff"
# child.yml
do_some_more_stuff:
script: echo "doing even more stuff"
The trigger_child job completes successfully once the child pipeline has been created.

Resources