I’m looking for a way to trigger a cloud function (HTTP Trigger) when the CircleCI is job success? I want to post an artifact URL for the cloud function when the circle-ci job success (this is for an android project)
You can use the when: on_success function to determine when the job step will run. More information on that here: https://circleci.com/docs/2.0/configuration-reference/#run
As for the HTTP Trigger, you can run any bash file or command from the command output. Here's what it might look like.
- run:
command: |
echo Hitting endpoint
curl mywebsite.com
when: on_success
Related
Context
After creating a general post-receive for a GitLab server, I noticed it gets triggered directly after a new commit is detected in any repository. However, I would like the post-receive script to do something with the build status of the GitLab Runner CI on the commit that triggered the post-receive script.
Approach
Based on this question and answer, I wrote a post-receive script that gets the commit and repository, and I tried to get the build status from that commit from within the GitLab docker:
#!/bin/bash
read oldrev newrev refname
echo "Previous Commit: $oldrev"
echo "New/latest Commit: $newrev"
echo "Repository name: $refname"
# Get build status of $newrev
curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/17/$refname/commits/$newrev/statuses
However, that API call does not work from within the Docker environment (which is from where the post-receive script runs).
Docker GitLab Build Status File locations
I also found the build status badges inside the Docker, they are located in: /opt/gitlab/embedded/service/gitlab-rails/public/assets/. However I do not (yet) know how to decode their filenames. For example, the build status badge accompanying Job #3, of commit: 9514d16aafc1d741ba6a9ff47718d632fa8d435b has filename: icons-6d7d4be41eac996c72b30eac2f28399ac8c6eda840a6fe8762fc1b84b30d5a2d.svg. Basically I do not know to which commit/repository that build status badge belongs.
On the other hand, I have found the location of the job logs in the hashed path of the repository:
/var/opt/gitlab/gitlab-rails/shared/artifacts/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35/2021_10_09/1/1/job.log
/var/opt/gitlab/gitlab-rails/shared/artifacts/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35/2021_10_14/3/3/job.log
Which each in turn contain their respective commit and branch as:
Checking out 9514d16a as master...
So in principle I could scan the repository path and accompanying job logs until I found the job.log that contains the commit of the post-receive script (for e.g. 5 minutes, to account for the delay between the commit and the starting of the GitLab Runner CI), and then search for the build status output in that job.log (e.g. Job succeeded) (for e.g. 60 minutes to allow for long jobs). However, that seems like a convoluted work-around.
Question
Hence, I was wondering, *Is there a better/faster/robuster method to get the GitLab Runner CI build status of the commit that triggered the general post-receive script of a GitLab server, inside that triggered instance/run of the post-receive script?
I have a gitlab pipeline where I have 3 jobs in same stage and should run in parallel. After all jobs are completed, I need to know the following information for each job:
Job Name, Status (pass/fail), started at, finished at
I am using below api in the after_script in .gitlab-ci.yml
curl https://gitlab.com/api/v4/job?job_token=$CI_JOB_TOKEN"
But is always gives me status as 'running'. How can I get the correct status whether the job is passed or failed?
You don't need to use the API in that case.
In after_script section, you can use CI_JOB_STATUS environment variable (available from Gitlab Runner 13.5). From the documentation :
The status of the job as each runner stage is executed. Use with after_script. Can be success, failed, or canceled.
I've got "running" from CI_JOB_STATUS
To fix that set ENV variable for job or globally: FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY: 1
More details you can find here https://gitlab.com/gitlab-org/gitlab-runner/-/issues/27693
I'm working with SVN but I would like to move on to Git, and more specifically to Gitlab.
I have the following structure:
MyStructure/
customer/
client1/
delivery.sh
MyFiletoSend.sh
client2/
delivery.sh
MyFiletoSend2.sh
Currently, the "delivery.sh" will send the modifications (rsync) of the file "MyFiletoSend.sh" to the server "client1".
Can I run the "delivery.sh" via Gitlab automatically after/before the git push only on the files modified in this push?
Example:
I have a modification to make to the file "MyFiletoSend.sh" from client1/
I make my change
commit and push
Gitlab is running "delivery.sh" on my "client1/" file.
The file "MyFiletoSend.sh" is sent to the server of "client1" without touching "client2".
Yes, it is possible
but first of all you need to understand how gitlab ci works. Read this article https://docs.gitlab.com/ee/ci/yaml/
You will create a step in your pipeline that will do what you want after you push the code (in master or in any other branch/mr)
and about the job? you have to create one, you can use this code to help you
https://gist.github.com/hnlq715/6c222ba0fd868bae7e4dfd3af61bf26e
Assuming your delivery.sh scripts have all the rsync logic required, GitLab has built-in logic to detect changes in files and execute bash commands in response. You can create a separate job for each client, which can run in parallel in the same stage. This approach is also auditable in that it will clearly show you which clients got updated and with which version of the file.
update-client-1:
stage: update-clients
only:
changes:
# Detect change only in MyFiletoSend.sh:
- customer/client1/MyFiletoSend.sh
# Detect any change in the customer folder:
- customer/client1/*
script:
- cd customer/client1
- delivery.sh
update-client-2:
stage: update-clients
only:
changes:
- customer/client2/*
script:
- cd customer/client2
- delivery.sh
# repeat for all remaining clients
For more information: https://docs.gitlab.com/ee/ci/yaml/#onlychangesexceptchanges
I have a job in my pipeline that has a script with two very important steps:
mvn test to run JUnit tests against my code
junit2html to convert the XML result of the tests to a HTML format (only possible way to see the results as my pipelines aren't done through MRs) that is uploaded to GitLab as an artifact
docker rm to destroy a container created earlier in the pipeline
My problem is that when my tests fail, the script stops immediately at mvn test, so the junit2html step is never reached, meaning the test results are never uploaded in the event of failure, and docker rm is never executed either, so the container remains and messes up subsequent pipelines as a result.
What I want is to be able to keep a job going till the end even if the script fails at some point. Basically, the job should still count as failed in GitLab CI / CD, but its entire script should be executed. How can I configure this?
In each step that you need to continue even if the step fails, you can add a flag to your .gitlab-ci.yml file in that step. For example:
...
Unit Tests:
stage: tests
only:
- branches
allow_failure: true
script:
- ...
It's that allow_failure: true flag that will continue the pipeline even if that specific step fails. Gitlab CI Documentation about allow_failure is here: https://docs.gitlab.com/ee/ci/yaml/#allow_failure
Update from comments:
If you need the step to keep going after a failure, and be aware that something failed, this has worked well for me:
./script_that_fails.sh || FAILED=true
if [ $FAILED ]
then ./do_something.sh
fi
Having in gitlab-ci a job like the following one:
static_test_service:
stage: test code
script:
- docker run --rm -v $(pwd):/data -w /data dparra0007/sonar-scanner:20171010-1 sonar-scanner
-Dsonar.projectKey=$CI_PROJECT_NAMESPACE:$CI_PROJECT_NAME
-Dsonar.projectName=$CI_PROJECT_NAME
-Dsonar.branch=$CI_COMMIT_REF_NAME
-Dsonar.projectVersion=$CI_JOB_ID
-Dsonar.sources=./greetingapi/src
-Dsonar.java.binaries=./greetingapi/target
-Dsonar.gitlab.project_id=$CI_PROJECT_ID
-Dsonar.gitlab.commit_sha=$CI_COMMIT_SHA
-Dsonar.gitlab.ref_name=$CI_COMMIT_REF_NAME
I would need to fail the gitlab job when the sonarqube analysis fails. But in that case, the error in analysis is reported but not sending a fail status to the job in Gitlab CI and the step always finish with success.
It seems that there is no way to raise any event from "docker run" to be managed by gitlab job.
Any idea on how to force to fail the job if the sonarqube analysis fails?
Thanks,
To break the CI build for a failed Quality Gate, you have write script based on the following steps
1.Search in /report-task.txt the values of the CE Task URL (ceTaskUrl) and CE Task Id (ceTaskId)
2.Call /api/ce/task?id=XXX where XXX is the CE Task Id retrieved from step 1 Ex:- https://yourSonarURL/api/ce/task?id=Your ceTaskId
3.Wait for sometime until the status is SUCCESS, CANCELED or FAILED from Step 2
4.If it is FAILED, break the build (Here failure is unable to generate sonar report)
5.If successful,then Use the analysisId from the JSON returned by /api/ce/task? id=XXX(step2)and Immediately call /api/qualitygates/project_status?analysisId=YYY to check the status of the quality gate.
Ex:- https://yourSonarURL/api/qualitygates/project_status?analysisId=Your analysisId
6.Step 5 gives the status of the critical, major and minor error threshold limit
7.Based on the limit break the build.
I faced this problem with GitLab and Sonar where Sonar was failing the QualityAnalysis but GitLab job was still passing with
INFO: ANALYSIS SUCCESSFUL, you can find the results at:
Now the problem is below missing config in sonar.properties
sonar.qualitygate.wait=true
sonar.qualitygate.timeout=1800
So basically, the SonarScan takes time to do the analysis and by default it won't wait for the analysis to complete and may returns default SUCCESSFUL ANALYSIS result to GitLab
With the mentioned configuration, we are explicitly asking GitLab to wait for the qualitygate to finish and gave some timeout as well (in case analysis takes long time to finish)
Now we see the GitLab job fails with below
ERROR: QUALITY GATE STATUS: FAILED - View details