I have written few jobs in .gitlab-ci.yml and my question is similar to this one SO Question. However, the answers provided and accepted doesn't work for my scenario.
The job has an after_script section which executes when the main task completes or fails.
Problem: I send an email alert based on whether the main task succeeded or failed, but I can't find any Gitlab CI variable that indicates the result of the job to clarify in the alert email.
How can I tell, inside the after_script section, whether the main task has succeeded or failed?"
If I use when: on_failure, then my question is when can I define my when: on_success job, since these jobs will depend on the job right before the one - so I can only execute one of these. I've been trying to find variables in Gitlab Variables for this, but couldn't find.
Also, in my after script - I can write if condition, but I am checking if someone can provide better alternate soltion
Problem: I send an email alert based on whether the main task succeeded or failed, but I can't find any Gitlab CI variable that indicates the result of the job to clarify in the alert email.
Also, in my after script - I can write if condition, but I am checking if someone can provide better alternate solution
May be you will be interested in new stage to provide email notification job.
Example of full .gitlab-ci.yml:
stages:
- build
- notify
build_job:
stage: build
script:
- echo "Your awesome script"
allow_failure: false # it's here just clarify the workflow, it's false by default
email_alarm:
stage: notify
when: on_failure
script:
- echo "Your alarm notification to email"
email_success:
stage: notify
when: on_success
script:
- echo "Your success notification to email"
So, here it is: email_alarm will be executed only if build_job has been failed. And email_success will be executed only if build_job succeed finished.
If you need to provide some artifacts from build_job to email - use gitlab artifacts in build_job and email_alarm/email_success jobs.
Or you can set your custom conditions such as:
scripts:
- ./script_that_fails.sh > /dev/null 2>&1 || FAILED=true
- if [ $FAILED ]
then ./do_something.sh
fi
Note the answer is another question - since v13.5, there's a CI_JOB_STATUS variable available in after_script.
Related
I wrote a pipeline task with a variable passed like this
jobs:
- job: buildandpush
pool:
vmImage: 'ubuntu-latest'
steps:
- script: |
echo sanity check
echo $NOTING_SERVICE_ORIGIN
echo $NOTING_SERVICE_ORIGIN_2
env:
NOTING_SERVICE_ORIGIN: dummy-string-111
NOTING_SERVICE_ORIGIN_2: dummy-string-222
What I see printed is:
sanity check
https://some-url-we-used-in-the-past/
dummy-string-222
I did not ever add any variables through Azure DevOps UI. The value https://some-url-we-used-in-the-past/ is no longer anywhere in the codebase. I could not find anything interesting in Azure Pipelines docs.
Is Azure Pipelines caching NOTING_SERVICE_ORIGIN somewhere?
Ended up finding someone else already defined the variable for the pipeline. Surprising was that it took precedence over the same variable passed explicitly on the spot.
Is it possible to trigger another pipeline from the pipeline completion trigger if there is a failure in the triggering pipeline? Seems there is no configuration/property available by default as per the documentation. Just wanted to check whether there is any possible way with the pipeline completion trigger.
If the initial pipeline fails to trigger, all subsequent pipelines would logically fail to trigger. Try having your initial pipeline start with a stage that will never fail, and if that pipeline fails, you can set it to trigger the subsequent pipelines after the first one fails but gets triggered succesfully.
Is it possible to trigger another pipeline from the pipeline completion trigger if there is a failure in the triggering pipeline?
There is no such configuration/property available to achieve trigger another pipeline from the pipeline completion trigger if there is a failure in the triggering pipeline.
To resole this issue, you could try to add powershell task to use the REST API Builds - Queue：
POST https://dev.azure.com/{organization}/{project}/_apis/build/builds?api-version=6.1-preview.7
You could check this thread for the detailed scripts.
And set this powershell task with condition Only when a previous task has failed:
In this case, regardless of whether the previous task fails, the REST API will be called at the end of the pipeline to trigger the build.
I was able to manage my requirement through the pipeline completion trigger itself. It is possible if we define stages in the triggering pipeline. I'm posting the answer if someone else looking for the same approach.
Need to define the triggering pipeline definition with stages. Also, we need to make sure that at least one stage should be successful every time. I already have few stages defined and hence this is totally matching with my requirement.
Triggering pipeline YAML definition: (pipeline name: pipeline1)
trigger: none
pr: none
pool:
vmImage: 'ubuntu-latest'
stages:
- stage: stage_1
displayName: Stage-1
jobs:
- job: greeting
displayName: Greeting
steps:
- script: |
echo "Hello world!"
exit 1
- stage: stage_2
displayName: Stage-2
condition: always()
jobs:
- job: thanking
displayName: Thanking
steps:
- script: |
echo "Thank you!"
Define the pipeline completion trigger with stage filters for the triggered pipeline.
Triggered pipeline YAML definition:
trigger: none
pr: none
resources:
pipelines:
- pipeline: Pipeline_1
source: pipeline1
trigger:
stages:
- stage_2
pool:
vmImage: 'ubuntu-latest'
jobs:
- job: greeting
steps:
- script: |
echo "Hello world!"
Then the triggered pipeline will be triggered irrespective to the stage_1 in the triggering pipeline since stage_2 will be kept successful in each run.
I'm aware that is possible to trigger another pipeline from another project by adding the below commands in a gitlab-ci file:
bridge:
stage: stage_name_here
trigger:
project: path_to_another_project
branch: branch_name
strategy: depend
The problem is that the config above will trigger all jobs, and I want to trigger only 2 jobs within the pipeline.
Any idea on how to trigger only those 2 specific jobs?
You can play with rules, only and except keywords in triggered ci file, for example add this to jobs that you want to trigger:
except:
variables:
- $CI_PROJECT_ID != {{ your main project id }}
And for jobs you don't want trigger this:
except:
variables:
- $CI_PROJECT_ID == {{ your main project id }}
Or if you want use rules, add this to jobs you want to run in main project:
rules:
- if: $CI_PROJECT_ID == {{ your main project id }}
when: never
- when: always
Instead of defining a variable that needs to be passed by the upstream pipeline that is triggering the downstream pipeline, I simply added the lines below in the jobs that I don't want to run in the downstream pipeline when triggered by another job:
except:
refs:
- pipelines
source:
https://docs.gitlab.com/ee/ci/triggers/index.html#configure-cicd-jobs-to-run-in-triggered-pipelines
I have the following content of the .gitlab-ci.yml job:
stages:
- stage1
- stage2
job1:
stage: stage1
script:
- echo "Running default stage1, pipeline_source=$CI_PIPELINE_SOURCE"
job2:
stage: stage2
rules:
- if: $CI_PIPELINE_SOURCE == "push"
- when: always
script:
- echo "Running STAGE2! pipeline_source=$CI_PIPELINE_SOURCE"
when I commit this change to a merge-request branch, it seems two pipelines are being started.
Is this a known issue in gitlab? Or do I understand something wrong here?
GitLab creates pipelines both for your branch and for the merge request. This is an "expected"[1] feature of GitLab as a consequence of using rules:. (oddly enough, when using only/except, merge request pipelines will only happen when using only: - merge_requests).
If you simply want to disable the 'pipelines for merge requests' and only run branch pipelines, you can include the default branch pipelines template, which provides a workflow: that prevents pipelines for merge requests.
include:
- template: 'Workflows/Branch-Pipelines.gitlab-ci.yml'
Additionally, you can see this answer for a workflow that will prevent duplicates between the pipelines for merge requests and branch pipelines only when a merge request is open.
[1]: I've always found this to be a quirk of GitLab and, as an administrator of GitLab for hundreds of users, I've gotten this question many many times. So, you're not alone in being surprised by this 'expected feature'
You didn't do anything wrong. This is actually intended, though it's a weird side-effect of the fact that Merge Requests have their own pipeline contexts. So when you commit to a branch that's associated with a merge request, two pipelines start:
A branch-based pipeline, with no context of the merge request
A merge request pipeline, with all the merge request variables populated (this is called a "detached" pipeline)
You can control this behavior by using a workflow keyword in your pipeline. We use the following workflow settings on our repositories:
workflow:
rules:
- if: $CI_MERGE_REQUEST_IID
- if: $CI_COMMIT_TAG
- if: $CI_PIPELINE_SOURCE == "schedule"
- if: $CI_COMMIT_REF_PROTECTED == "true"
The above rules will prevent the branch pipelines from running unless the branch is a protected branch (I.e., you're merging into the main branch), a tagged commit (I.e., you're releasing code), or the pipeline has been scheduled. This means that when you commit to a MR, the branch-based pipeline (#1 from the above numbers) doesn't run, and you are left with one pipeline running.
I have an Azure DevOps Pipeline for a Git repository. I currently have a script to validate the PR comments in the Azure Pipeline.
When the code is merged into the main branch I want to trigger a build. I am not sure how to achieve this with a Azure DevOps pipeline.
#Trigger for Development
trigger:
branches:
include:
- development
- master
#Trigger checks for PR
pr:
branches:
include:
- development
- master
- feature
- main
paths:
exclude:
- README/*
When the code is merged into the main branch I wanted to trigger build
If you want to verify the comments after the code is merged into the main branch, we need to trigger the build after the PR completed instead of when PR is created.
So, the PR triggers could not meet our requirement in this case.
To resolve this issue, we could enable CI triggers for the main branch with ** condition** eq(variables['Commitcomment'], 'Merge pull request') for the task of script to validate the PR comments.
With this condition, the pipeline will execute the job only when the Commitcomment is Merge pull request, this can filter out modifications not done by PR.
To get the value of the variable Commitcomment, we could to check the commits message on our github by the variable Build.SourceVersionMessage:
If the commit comes from PR, it will given a default comment, starting with: Merge pull request xxx, we could add a bash\powershell script to get the first few fields.
Then use Logging Command to set the variable Commitcomment to true if the first few fields is Merge pull request:
- task: CmdLine#2
displayName: get the first few fields
inputs:
script: >-
echo $(Build.SourceVersionMessage)
set TempVar=$(Build.SourceVersionMessage)
set Commitcomment=%TempVar:~0,18%
echo %Commitcomment%
echo ##vso[task.setvariable variable=Commitcomment]%Commitcomment%
Reference link: Is there a short 7-digit version of $(SourceVersion) in Azure Devops?
Then add this variable as condition condition: and(succeeded(), eq(variables['Commitcomment'], 'Merge pull request')) for your task to verify the PR comments:
- task: CmdLine#2
displayName: script to validate the PR comments
condition: and(succeeded(), eq(variables['Commitcomment'], 'Merge pull request'))
inputs:
script: >
echo To validate the PR comments
In this case, if the commit not comes from PR, it will skip the PR comments verify task:
If you just want to launch a build when the merge is done (pull request validated) in a specific branch, your code is good.
If you want to run a validation build currently it is not integrated into the Yaml pippeline configuration (https://learn.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema%2Cparameter-schema#pr-trigger)
To do this, it must be done via the graphical interface:
Project Settings -> Repositories -> Select your repo -> Policies -> Branch Policies -> Select your branch -> Build Validation -> + -> add build information
(https://learn.microsoft.com/en-us/azure/devops/repos/git/branch-policies?view=azure-devops#build-validation)