I have gitlab project with ci file:
stages:
- build
- run
build:
stage: build
script:
- (some stuff)
tags:
- my_runner_tag
except:
- triggers
when: manual
run:
stage: run
script:
- (some stuff)
tags:
- my_runner_tag
except:
- triggers
when: manual
Jobs are created on every source code change, and they can be run only manually, using gitlab interface.
Now, i want to have possibility to trigger stage run with Gitlab API. Trying:
curl -X POST \
> -F token=xxxxxxxxxxxxxxx \
> -F ref=master \
> https://gitlab.xxxxx.com/api/v4/projects/5/trigger/pipeline
Returns:
{"message":{"base":["No stages / jobs for this pipeline."]}}
Seems, like i have to define stage to trigger, but i can't find a way to pass it via api call.
you are using the wrong endpoint, to do it, you need to follow the path below
list all of your pipelines and get the newest one
GET /projects/:id/pipelines
list the jobs from this pipeline
GET /projects/:id/pipelines/:pipeline_id/jobs
After that you can trigger your job
POST /projects/:id/jobs/:job_id/play
you are telling your build to run at all times except for the time they are being triggered (api call is also considered as a trigger).
change your job definition to the following:
run:
stage: run
script:
- (some stuff)
tags:
- my_runner_tag
when: manual
Related
In the gitlab ce omnibus 14.10.2 instance, there is a project in which it is necessary to make a pipeline in gitlab-ci.yml that would check for the existence of dynamical environment deployments
and if they are not there, then do nothing, and if they are, it would update them or manually deploy by a trigger from the webui.
My example:
#...
check-review:
variables:
GIT_STRATEGY: none
stage: dev/review/check
only:
- /^(feature|hotfix)\/(.*)/m
dependencies: []
allow_failure: true
script:
- curl -sLf ${CI_COMMIT_REF_SLUG}.domain.io --output /dev/null
when: always
ansible/deploy:
stage: dev/review
only:
- /^(feature|hotfix)\/(.*)/m
except:
- master
- staging
environment:
name: $CI_COMMIT_REF_NAME
url: https://${CI_COMMIT_REF_SLUG}.domain.io
action: start
on_stop: ansible/undeploy
when: on_success
dependencies:
- vendor # build back
script:
- make deploy
ansible/undeploy:
stage: dev/review
variables:
GIT_STRATEGY: none
environment:
name: $CI_COMMIT_REF_NAME
url: https://${CI_COMMIT_REF_SLUG}.domain.io
action: stop
when: manual
needs:
- job: ansible/deploy
script:
- make delete_dev_stand
This condition need because dev teams workinig on many branches and not everyone need to be deployed on dev envs, on the other hand with manual update only - on every update/fix code qa need to go gitlab webui & trigger job to deploy/update deployment in environments https://${CI_SERVER_HOST}/${CI_PROJECT_PATH}/-/environments/
Maybe I`ve miss something in gitlab-ci ref, only solution that I found was to make failure pipline on check stage/job but this not happy to see so many failed pipelines.
It's been two months so you probably have a solution already, but here's something you could try if it fits your use case
Approach:
Use pipeline triggers and a script that calls the Gitlab API
1. Find out if the environment exists
a. Create a script called get-deployment-strategy.sh and add it to your project
b. Create a private token in your user preferences with API read permission and add it to your CI/CD variables (GITLAB_API_PRIVATE_READ_TOKEN=token)
#!/bin/bash
# Private Token (created in user preferences) with API READ permission added to CI/CD vars
HEADER="PRIVATE-TOKEN: $GITLAB_API_PRIVATE_READ_TOKEN"
# To get the project which the environment should be read from
API_URL="https://gitlab.com/api/v4/projects/$CI_PROJECT_ID/environments"
# URL query parameter to filter for environment with specific name
QUERY_STRING="name=$DEPLOYMENT_TARGET/$DEPLOYMENT_NAME"
# Call Gitlab API (Environments API)
API_RESPONSE="$(curl -G --header "$HEADER" "$API_URL" --data-urlencode "$QUERY_STRING")"
# The response is an empty array (string) if there is no environment with the specific name
if [ "$API_RESPONSE" = "[]" ]; then
# Deployment environment does not exist
echo "DO_STRATEGY_A"
else
echo "DO_STRATEGY_B"
fi
2. Use trigger pipelines within your check job
a. Create a trigger token in your CI/CD settings and add it to your CI/CD variables
b. Call the trigger pipeline in your check job and pass it a "deployment strategy variable"
Something like this:
check-job:
before_script:
- export DEPLOYMENT_STRATEGY=$(./get-deployment-strategy.sh)
script:
- |
curl \
--fail \
--request POST \
--form token=$DEPLOY_TRIGGER_TOKEN \
--form ref=$CI_COMMIT_BRANCH \
--form variable[DEPLOYMENT_STRATEGY]=$DEPLOYMENT_STRATEGY \
"https://gitlab.com/api/v4/projects/$CI_PROJECT_ID/trigger/pipeline"
3. Make the deploy jobs activate only on specific DEPLOYMENT_STRATEGY
# Only runs for DO_STRATEGY_A (environment does not exist)
deploy/strategy-A:
rules:
- if: '$CI_PIPELINE_SOURCE="trigger" && $DEPLOYMENT_STRATEGY=="DO_STRATEGY_A"'
# Only runs for DO_STRATEGY_B (environment exists)
deploy/strategy-B:
rules:
- if: '$CI_PIPELINE_SOURCE="trigger" && $DEPLOYMENT_STRATEGY=="DO_STRATEGY_B"'
I am trying to trigger a pipeline on any type of change/commit (push/merge/webIDE etc) to a specific branch called test branch. Not sure why it doesn't trigger at all even after I make a change to a file in that specific branch.
default:
tags:
- specs
stages:
- validate
Trigger job:
stage: validate
script:
- curl -X POST -F token=$ENTDV_PL_TRIGGER_TOKEN -F "ref=master" https://gitlab.com/api/v4/projects/1252972/trigger/pipeline
variables:
CI_DEBUG_TRACE: "true"
rules:
- if: '$CI_COMMIT_BRANCH == "testbranch"'
If you want Trigger job to only be run when changes are made on testbranch, you can modify your Trigger job to the following:
Trigger job:
stage: validate
script:
- curl -X POST -F token=$ENTDV_PL_TRIGGER_TOKEN -F "ref=master" https://gitlab.com/api/v4/projects/1252972/trigger/pipeline
variables:
CI_DEBUG_TRACE: "true"
only:
- testbranch
The Gitlab CI documentation is actually quite good. Here is a link to how "only" works: https://docs.gitlab.com/ee/ci/yaml/index.html#only--except
From the docs: "Use the only:refs and except:refs keywords to control when to add jobs to a pipeline based on branch names or pipeline types."
You can do some other neat things like specifying only merge_requests or using regex for triggering jobs and what not.
I have a gitlab ci pipeline in my application repo, A, which calls an end to end testing Repo T to run its tests. The repo A pipeline succesfully triggers the tests from Repo T but if the test job fails in T, the job calling the test job in T from A still passes. How do I get repo A to track the result of Repo T's test job, and pass/fail its pipeline based off of the test jobs in T?
.gitlab-ci.yml for testing Repo T:
stages:
- test
test:
stage: test
image: markhobson/maven-chrome:jdk-11
artifacts:
paths:
- target/surefire-reports
script:
- mvn test
only:
- triggers
.gitlab-ci.yml from application repo A:
job1:
stage: unit-tests ...
job2:
stage: build ...
...
trigger-e2e-repo:
stage: e2e-testing
image: markhobson/maven-chrome
script:
- "curl -X POST -F token=repo-T-token -F ref=repo-T-branch https://repo-A/api/v4/projects/repo-T-id/trigger/pipeline"
only:
- repo-A-branch
Since GitLab 11.8 you can trigger a pipeline via bridge job.
In GitLab 11.8, GitLab provides a new CI/CD configuration syntax to make this task easier, and avoid needing GitLab Runner for triggering cross-project pipelines.
With bridge jobs it is possible to mirror the status of the trigger pipeline to the calling pipeline.
You can mirror the pipeline status from the triggered pipeline to the source bridge job by using strategy: depend.
Example in your case:
trigger-e2e-repo:
stage: e2e-testing
trigger:
project: repo-T
strategy: depend
If the triggered pipeline with the test jobs fails, the calling pipeline also fails.
If you only want to execute a particular job in your repository "Repo T" when executed by a bridge job, then you should use only: pipeline (only) or rules: -if '$CI_PIPELINE_SOURCE == "pipeline"' (rules:if) instead of only: triggers.
I wasn't able to use the bridge job property of mirroring a downstream job result as the version of my gitlab is before 11.8. I did manage to get it to work by creating a trigger for repo A, and making a call from repo T to repo A with this new second trigger. The remaining jobs in repo A will only be activated by triggers and setting of variables (JOB in this case ) as laid out below:
.gitlab-ci.yml for repo T:
test:
stage: test
script:
- mvn test
- "curl -X POST -F token=repo-A-token -F ref=$BRANCH -F variables[JOB]=build https://project.com/api/v4/projects/project_id/trigger/pipeline"
.gitlab-ci.yml in A
job1:
...
except:
- triggers
job2:
...
except:
- triggers
...
trigger-e2e-repo:
stage: e2e-testing
script:
- "curl -X POST -F token=repo-B-token -F ref=repo-B-branch -F variables[BRANCH]=$CI_COMMIT_REF_NAME -F https://project-B/api/v4/projects/project-B-id/trigger/pipeline"
except:
- triggers
build_application_for_prod:
stage: build_prod
script:
- "curl -X POST -F token=repo-A-token -F ref=$CI_COMMIT_REF_NAME -F variables[JOB]=deploy -F variables[SEND]=true https://foo/api/v4/projects/proj_A_id/trigger/pipeline"
only:
variables:
- $JOB == "build"
deploy_production_environment:
stage: deploy_prod
script:
- ...
only:
variables:
- $JOB == "deploy"
Note I also had to add the except statements for the jobs before the end to end tests in repo A so that they won't rerun and loop when repo A's API trigger is called later on.
I'm trying to run 2 pipelines for a project in GitLab, but I can't find any way to do it.
In gitlab CI you can't create multiple pipelines for one project explicitly. There are cases where multiple pipelines will run simultaneously, such as when you have jobs that run only for merge requests and other jobs that do not run on merge requests.
That said, there are ways to obtain the effect of running multiple series of jobs independently from one another.
The hacky way, before gitlab-ce 12.2
If you want to start 2 pipelines for the same project you can use pipeline triggers. This method is limited to 2 pipelines and gitlab CI is not meant to be used this way. Usually, triggers are used to start pipelines on another project.
All in your .gitlab-ci.yml:
stages:
- start
- build
###################################################
# First pipeline #
###################################################
start_other_pipeline:
stage: start
script:
# Trigger a pipeline for current project on current branch
- curl --request POST --form "token=$CI_JOB_TOKEN" --form ref=$CI_COMMIT_REF_NAME $CI_API_V4_URL/projects/$CI_PROJECT_ID/trigger/pipeline
except:
- pipelines
build_first_pipeline:
stage: build
script:
- echo "Building first pipeline"
except:
- pipelines
###################################################
# Second pipeline #
###################################################
# Will run independently of the first pipeline.
build_second_pipeline:
stage: build
script:
- echo "Building second pipeline"
only:
- pipelines
To clean up this mess of a .gitlab-ci.yml, you can use the include keyword:
# .gitlab-ci.yml
include:
- '/first-pipeline.yml'
- '/second-pipeline.yml'
stages:
- start
- build
# This starts the second pipeline. The first pipeline is already running.
start_other_pipeline:
stage: start
script:
# Trigger a pipeline for current project on current branch
- curl --request POST --form "token=$CI_JOB_TOKEN" --form ref=$CI_COMMIT_REF_NAME $CI_API_V4_URL/projects/$CI_PROJECT_ID/trigger/pipeline
except:
- pipelines
# first-pipeline.yml
build_first_pipeline:
stage: build
script:
- echo "Building first pipeline"
except:
- pipelines
# second-pipeline.yml
build_second_pipeline:
stage: build
script:
- echo "Building second pipeline"
only:
- pipelines
The reason this works is the use only and except in the jobs. The jobs marked with
except:
- pipelines
do not run when the pipeline has started because of a trigger coming from another pipeline, so they don't run in the second pipeline. On the other hand,
only:
- pipelines
does the exact opposite, therefore those jobs run only when the pipeline is triggered by another pipeline, so they only run in the second pipeline.
The probably right way, depending on your needs ;)
In gitlab CE 12.2, it is possible to define Directed Acyclic Graphs to specify the order that your jobs run. This way, a job can start as soon as the job it depends on (using needs) finishes.
As of GitLab 12.7 it is also possible to use parent-child pipelines for this:
# .gitlab-ci.yml
trigger_child:
trigger:
include: child.yml
do_some_stuff:
script: echo "doing some stuff"
# child.yml
do_some_more_stuff:
script: echo "doing even more stuff"
The trigger_child job completes successfully once the child pipeline has been created.
The CI pipeline runs on every commit in my Gitlab repository at work. Is there way to disable that and only run the CI pipeline on an open merge request to the master branch?
There is currently no configuration option to do that. Here are some things that can be used to "disable" a pipeline build.
Adding [ci skip] inside the commit message will not trigger a pipeline on push.
Using except and only options on all jobs inside the pipeline. To avoid duplication in this case, you can use Anchors.
Update: GitLab 11.7
When pushing to GitLab you can skip triggering a pipeline by passing ci.skip option to the push command: git push -o ci.skip
Update in 2020 because solution with only and except are candidates for deprecation : https://docs.gitlab.com/ee/ci/yaml/#onlyexcept-basic
Still in .gitlab-ci.yml you should use now rules : See https://docs.gitlab.com/ee/ci/yaml/#rules
I simply set this rule on my build job and the job is "blocked" and wait a manual trigger from the UI.
rules:
- when: manual
Note that we can create more advanced rules with conditions to trigger for exemple if we see a git tag.
https://docs.gitlab.com/ee/ci/yaml/#exclude-jobs-with-rules-from-certain-pipelines
Yes, but only if you set it on each job within the pipeline.
https://docs.gitlab.com/ee/ci/yaml/#onlyexcept-basic
job:
script: echo 'test'
only:
- merge_requests
That job will only run on commits that are part of a merge request, rather than every push of a commit to a branch. If you set every job to this setting then the pipeline will never run.
You can just add [ci skip] to the commit message and builds will be skipped
https://gitlab.com/gitlab-org/gitlab-ce/issues/14499
You could add something like this in the beginning of your .gitlab-ci.yaml
workflow:
rules:
- if: "$CI_COMMIT_BRANCH"
when: never
- if: '$CI_PIPELINE_SOURCE == "merge_request_event" && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"'
For more info look at Merge request pipelines
No. (Not yet !)
You can follow the expected feature development here.
https://gitlab.com/gitlab-org/gitlab-ce/issues/23902
However, you can automatically trigger pipelines using "maofr"'s code
https://gitlab.com/gitlab-org/gitlab-ce/issues/23902#note_88958643
For merge we can use this guide #onlyrefs--exceptrefs
merge_requests
For pipelines created when a merge request is created or updated. Enables merge request pipelines, merged results pipelines, and merge trains.
and for generally triggering pipeline we can use only: and variables: together:
script:
- . diff.sh $SMS_MESSAGE
only:
variables:
- $ACTION == "diff"
then triggering it is easy using curl:
curl -X POST \
-F token=$TOKEN \ # your token
-F ref=$REF_NAME \ # branch name
-F "variables[ACTION]=diff" \ # variable
https://gitlab.com/api/v4/projects/0/trigger/pipeline # your project endpoint
This part variables[ACTION]=diff will assign diff into ACTION and it will apply to .gitlab-ci.yml and only: section.
job:
stage: build
script:
- echo "Do your build here"
except:
- pushes
Use except pushes to stop create a new pipeline on every pushed commits. For reference check this
Update 2022 :
According to the documention of Gitlab on: https://docs.gitlab.com/ee/ci/yaml/#when
you can use when: manual to run the job only when triggered manually.
Example:
deploy_job:
stage: deploy
script:
- make deploy
when: manual
You don't need also rules for that.