I would like to know how to manually trigger specific jobs in a project's CI pipeline.
Since there is only one gitlab-ci.yml file, I can define many jobs to be executed one after the other sequentially. But what if I want to start a manual CI pipeline that only carries out one job?
As I understand it, every time the pipeline will run, it will run all jobs, unless I use many only and similar parameters. For instance, when I have this simple pipeline config:
stages:
- build
build:
stage: build
script:
- npm i
- npm run build
- echo "successful build"
What do I do if I want to only run an echo job that runs a simple echo "hello" script, but does only that and only when I manually run it? There are no 'triggers' for a job like that afaik.
Is this even a possibility?
Thanks for the clarification!
Apparently, the solution is pretty simple, just needed to add a when: manual paramater to the job:
echo:
stage: echo
script:
- echo 'this is a manual job'
when: manual
Once that's done, the job can be triggered independently right here:
Related
From what I have just ran into, the "needs" line in a gitlab-ci.yml file only checks to see if the job that is defined in the "needs" line is being run - not if it passes or fails.
I ran the below code in my pipeline and the "build-latest" job runs even if the "test-
tag" job fails.
I only want the "build-latest" job to run if the "test-tag" job passes.
How is this achieved?
build-latest:
stage: publish
image:
name: gcr.io/go-containerregistry/crane:debug
entrypoint: [""]
rules:
#- if: $CI_COMMIT_TAG != null
- if: $CI_COMMIT_REF_NAME == "add-latest-tagging"
when: always
needs:
- test-tag
script:
- crane auth login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
The issue lies with the fact that you added
when: always
It is true that since you specified needs, the build-latest job will need the job test-tag to execute first.
After test-tag job concluded it will evaluate if it should execute the build-latest job.
By adding the always clause to the build-latest job will force it to execute, even if the test-tag fails. Provided test-tag job has at least concluded
Long story sort, you should remove the when always clause
If you want a job to run only when one or more previous jobs pass, then you want to put it in a separate stage.
Not sure how you've broken up the jobs without more of the CI file, but assuming:
test-tag job is in stage: test
stage: publish comes after test
Then it should work the way you want simply by removing the needs: option from your build-latest job.
Can someone please tell me how to run a job in detached mode ? A job in my pipeline takes 30 minutes to complete, I want the pipeline to proceed without waiting for this job to complete.
For example:
stages:
- build
- build2
- test
newservice:
stage: build
script:
- echo "build is done"
newservice1:
stage: build2
script:
- echo "build1 is done"
- sleep 60
mygotservice:
stage: test
needs: ["newservice"]
script:
- echo "test is done"
I want the pipeline to proceed ahead without waiting for newservice1.
It depends on many reasons...
running tests may take time.
Where is the server? Where to get resources?
There are many possible reasons and it is not just about the git...
I have a YAML file as below. Let’s say the *.md file is committed, the build does not work, but the test works. Here how can I make the test depend on the build? Like if the build doesn’t work, the test shouldn’t work.
Thanks in advance.
build:
stage: build
script:
- echo "Build is running"
only:
changes:
- Dockerfile
- requirements.txt
- ./configs/*
test:
stage: test
script:
- echo "Test is running"
- echo "$CI_JOB_STAGE"
dependencies:
- build
That should be what stages defines
Use stages to define stages that contain groups of jobs.
stages is defined globally for the pipeline.
Use stage in a job to define which stage the job is part of.
The order of the stages items defines the execution order for jobs:
Jobs in the same stage run in parallel.
Jobs in the next stage run after the jobs from the previous stage complete successfully.
For example:
stages:
- build
- test
- deploy
All jobs in build execute in parallel.
If all jobs in build succeed, the test jobs execute in parallel.
If all jobs in test succeed, the deploy jobs execute in parallel.
If all jobs in deploy succeed, the pipeline is marked as passed.
If any job fails, the pipeline is marked as failed and jobs in later stages do not start.
Jobs in the current stage are not stopped and continue to run.
So, in your case:
stages:
- build
- test
test won't run if build fails.
Here is my .gitlab-ci.yml file:
script1:
only:
refs:
- merge_requests
- master
changes:
- script1/**/*
script: echo 'script1 done'
script2:
only:
refs:
- merge_requests
- master
changes:
- script2/**/*
script: echo 'script2 done'
I want script1 to run whenever there is a change in script1 directory; likewise script2.
I tested these with a change in script1, a change in script2, change in both the directories, and no change in either of these directories.
Former 3 cases are passing as expected but 4th case, the one with no change in either directory, is failing.
In the overview, Gitlab gives the message
Could not retrieve the pipeline status. For troubleshooting steps, read thedocumentation.
In the Pipelines tab, I have an option to Run pipeline. Clicking on that gives the error
An error occurred while trying to run a new pipeline for this Merge Request.
If there is no job, I want the pipeline to succeed.
Gitlab pipelines do not have any independent validity outside of jobs. A pipeline, by definition, consists of one or more jobs. In your example 4 above no jobs are created. The simplest hack you can add to your pipeline is a job which always runs:
dummyjob:
script: exit 0
In a project I'm running two stages with these jobs:
build
compile & test
generate sonar report
deploy
deploy to staging environment [manual]
deploy to production [manual]
The jobs in the deploy stage depend on the outputs of the compile & test job. However the generate sonar report job is not required to finish before I can start any job in the deploy stage. Nevertheless, GitLab insists that all jobs in the build phase have finished before I can launch any job in the deploy phase.
Is there a way I can tell GitLab that the generate sonar report job should not block subsequent pipeline stages? I already tried allow_failure: true on this job but this does not seem to have the desired effect. This job takes a long time to finish and I really don't want to wait for it all the time before being able to deploy.
We have similar situation, and while we do use allow_failure: true, this does not help when the Sonar job simply takes a long time to run, whether it fails or succeeds.
Since you are not wanting your deploy stage to actually be gated by the outcome of the generate sonar report job, then I suggest moving the generate sonar report job to the deploy stage, so your pipeline would become:
build
compile & test
deploy
deploy to staging environment [manual]
deploy to production [manual]
generate sonar report [allow_failure: true]
This way the generate sonar report job does not delay your deploy stage jobs
The other benefit of running generate sonar report after build & test is that you can save coverage reports from the build & test job as Gitlab job artifacts, and then have generate sonar report job consume them as dependencies, so Sonar can monitor your coverage, too
Finally, we find it useful to separate build & test into build, then test, so we can separate build failures from test failures - and we can then also run multiple test jobs in parallel, all in the same test stage, etc. Note you will need to convey the artifacts from the build job to the test job(s) via Gitlab job artifacts & dependencies if you choose to do this
From my point of view, it depends on your stage semantics. You should try to decide what is mostly important in your pipeline: clarity on stages or get the job done.
GitLab has many handy features like needs keyword you can use it to specify direct edges on the dependency graph.
stages:
- build
- deploy
build:package:
stage: build
script:
- echo "compile and test"
- mkdir -p target && echo "hello" > target/file.txt
artifacts:
paths:
- ./**/target
build:report:
stage: build
script:
- echo "consume the target artifacts"
- echo "waiting for 120 seconds to continue"
- sleep 120
- mkdir -p target/reports && echo "reporting" > target/reports/report.txt
artifacts:
paths:
- ./**/target/reports
deploy:
stage: deploy
needs: ["build:package"]
script:
- echo "deploy your package on remote site"
- cat target/file.txt
Unless I'm mistaken, this is currently not possible, and there is an open feature proposal, and another one similar to add what you are suggesting.