I am using YML file to configure my azure pipelines. My pipeline can either be triggered by new commits to development branch and/or scheduled cron. How can I track which triggered the pipeline? I wanted to do some task based on what triggered by pipeline, whether from a new commit or a cron. It's a small add-on task, so I am avoiding writing a new pipeline to separate the tasks. Thanks in advance!
Here is my code sample:
trigger:
- development
schedules:
- cron: '0 0 * * *' # will run midnight every day
displayName: 'Midnight tests' (UTC - 7:00)'
branches:
include:
- development
always: true
stages:
### do some work
We could add task bash and enter script printenv to print all pipeline env variable. Then we could see the variable BUILD_REASON, we can track this variable to check how the build pipeline is triggered.
Schedule is trigger from cron, IndividualCI is trigger from commit push. You could check the pic below.
Related
I want to send a slack notification as soon as trigger the pipeline build manually.
Steps I am following:
Go to your project’s CI/CD > Pipelines and select Run pipeline.
Choose the branch you want to run the pipeline for.
Input the variable and its value in the UI.
Click on the Run pipeline button
From the documentation I see:
How the pipeline was triggered. Can be push, web, schedule, api, external, chat, webide, merge_request_event, external_pull_request_event, parent_pipeline, trigger, or pipeline. For a description of each value, see Common if clauses for rules, which uses this variable to control when jobs run.
In this init stage I want to execute a script only when I run the pipeline using the above steps
init:
stage: init
environment: $DEPLOY_ENVIRONMENT
script:
- bundle exec fastlane slack_build_start_summary
rules:
- if: $CI_PIPELINE_SOURCE == "trigger" // also tried with "pipeline"
when: always
But I can't see this stage in the Gitlab pipeline stages.
I want to know which CI_PIPELINE_SOURCE ("trigger" or "pipeline" or something else?) I need to check when I trigger the pipeline manually from Gitlab using the above steps.
I got the answer here from the documentation
web -> For pipelines created by using Run pipeline button in the GitLab UI, from the project’s CI/CD > Pipelines section.
So I need to use:
- if: $CI_PIPELINE_SOURCE == "web"
i have a build pipeline i want to add mannual intervention in the pipeline and when that manual intervention is done the user can click on resume the pipeline but i want to do this in build pipeline.
i checked in this https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/manual-validation?view=azure-devops&tabs=yaml but it looks like we need to do using stage pipeline in release pipeline. but i want this to happen in build pipeline
We seem have not such task for build pipeline, especially classic build pipeline.
If you want to add manual intervention in a build pipeline, you can try to set up this pipeline with YAML and add this run the Manual Validation task in an agentless job.
However, it might not be possible if you want to pause a running agent job in the build pipeline. Maybe you can try run the pipeline on a self-hosted agent with interactive mode.
[UPDATE]
You should use the Manual Validation task in an agentless job in the YAML pipeline, not in the classic build pipeline.
To set up an agentless job in the YAML pipeline, you need to set the value of 'pool' key to ne 'server'.
jobs:
- job: agentless-job
pool: server
To view more details, you can see "Server jobs".
To add a manual validation task in pipeline without creating a agent-less job you will need to define pool: server on the top of your steps. as below.
I got this from by referring to canary deployments but have tested the same works for rolling deployment and probably other steps introduced in the pipeline.
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/deployment-jobs?view=azure-devops
pool: server
steps:
- task: ManualValidation#0
timeoutInMinutes: 1440 # task times out in 1 day
inputs:
notifyUsers: |
youremailadress#hotmail.com
instructions: 'Please validate the build configuration and resume'
onTimeout: 'reject'
In the situation where multiple PRs are approved within a short amount of time (minutes), the build pipeline will be started for every single PR causing a queue. On top of that, the release pipeline will start for every PR.
Is there a way for Azure devops to do this more efficient?
I'm thinking about a postpone of the execution of the pipeline for x minutes, but there might be other solutions to this.
Who can help me?
Is there a way for Azure devops to do this more efficient?
According to your description, these builds could be CI Builds triggered by approved PRs, right? (When the PRs are completed, the commits will push to the target branch and trigger the build )
If yes, you could try to use the Batch changes while a build is in progress option in Triggers.
Classic Editor:
Yaml Editor
trigger:
batch: true
branches:
include:
- features/*
exclude:
- features/experimental/*
Here is a Doc about Batching CI runs.
This option can combine all commits while the pipeline is running.
To wait for x minutes to collect PRS, you could add a Powershell task in your pipeline.
For example:
steps:
- powershell: 'Start-Sleep -s 300'
displayName: 'PowerShell Script'
The command Start-Sleep -s 300 could let pipeline wait for x minutes. And it could collect the PRs during x minutes.
A slightly different edit form showed in my case.
Navigate to Pipelines>...>Edit>...>Triggers
I'm having problems triggering a pipeline from another Pipeline in Azure DevOps. I have a CI pipeline and I want to trigger a Deploy Pipeline whenever CI passes on a master branch. This seems to be technically possible, but the documentation is unclear.
I see the following:
# this is being defined in app-ci pipeline
resources:
pipelines:
- pipeline: securitylib
source: security-lib-ci
trigger:
branches:
- releases/*
- master
But it's unclear as to a) whether this goes in the triggering pipeline (in my case the CI pipeline) or the triggered pipeline (in my case, the deploy pipeline).
It's also unclear as to what the pipeline and source refer to, and how I find out these variables? Are they both the name of the pipeline? I've tried various different permutations and nothing seems to be working.
EDIT 2
Finally Microsoft has improved their documentation with regards to the pipeline triggers in YAML! Here's the link.
EDIT
After having written my answer, Microsoft has come up with another solution to solve this problem, by using a build completion trigger via a classic pipeline. Their solution can be found here.
If you're not publishing an artifact from the triggering pipeline, it won't trigger the triggered pipeline.
Also, there is a very big restriction on the use of these types of triggers. It is necessary to change the defaultBranch for manual and scheduled builds in the depends pipeline, to the working branch. Otherwise it won't kick in at the end of the source pipeline execution. So, let's say you're working on feature branch, and defaultBranch is set to feature. You commit your code, and everything will run as expected: the source pipeline kicks in, and at its end, the depends pipeline will be triggered. All good! But when you will merge into master, if you do not change the defaultBranch, the depends pipeline won't be triggered at the end of the source pipeline. I explain how to change the defaultBranch at the end of the answer.
How to setup a pipeline trigger
I managed to get this up and running on a minimalistic project. Here you can have the code and here the project on Azure DevOps. I will try to guide you through how I did it, and answer the questions you've asked in your post.
I will be calling the triggered pipeline as depends pipeline and the triggering pipeline as source pipeline.
On the source pipeline, there's no need to do anything except publishing an artifact. If you don't publish an artifact from the source pipeline, it won't work. Below you can find the code I am using for my dummy source pipeline. I want it to be triggered for master branch, and at the end I want to be sure to publish an artifact.
trigger:
branches:
include: # branch names which will trigger a build
- master
pr: none
steps:
# required to cause pipeline triggering downstream
- task: CopyFiles#2
inputs:
contents: $(System.DefaultWorkingDirectory)/**/*.yml
targetFolder: $(Build.ArtifactStagingDirectory)
- task: PublishBuildArtifacts#1
inputs:
pathtoPublish: $(Build.ArtifactStagingDirectory)
artifactName: dummy-$(Build.BuildId)
On the depends pipeline (code shown below), I have to disable CI and PR triggers, otherwise when I commit to this repo, this pipeline will be triggered by the CI trigger, and then by the end of the execution of the source pipeline. This is done by the two first lines of my code. Then I want that the pipeline named source (this is the source property in the YAML below), within the project named Pipelining (project property in the YAML) will trigger the current (depends) pipeline when this updates master branch.
trigger: none
pr: none
resources:
pipelines:
- pipeline: source
project: Pipelining
source: source
trigger:
branches:
include:
- master
steps:
- checkout: none
- script: echo 'triggered depends'
Does it make sense? It is important for your project name on Azure DevOps to match the property in the YAML depends pipeline code.For me it is Pipelining
As well as the source property, again in the YAML depends pipeline code.
Change the default branch
In order to change the defaultBranch, because of the issue mentioned above, you should edit the pipeline (in this case, the depends pipeline), then on the three dots on the top right corner pick Triggers. Then choose the YAML tab, and you will get to the screen shown in the image below, where you can set the working branch.
Above yaml pipeline trigger should be defined in the triggered pipeline(deploy pipeline).
- pipeline: string the string here is identifier you give to this pipeline resource. It can any string.
source: string the string here is the definition name of the triggering pipeline(the name of your CI pipeline).
Below yaml is from the document pipeline resource.
resources:
pipelines:
- pipeline: string # identifier for the pipeline resource
project: string # project for the build pipeline; optional input for current project
source: string # source pipeline definition name
branch: string # branch to pick the artifact, optional; defaults to all branches
version: string # pipeline run number to pick artifact, optional; defaults to last successfully completed run
trigger: # optional; triggers are not enabled by default.
branches:
include: [string] # branches to consider the trigger events, optional; defaults to all branches.
exclude: [string] # branches to discard the trigger events, optional; defaults to none.
Option: You can also set the pipeline triggers from Ui page. Go the edit page of the triggered yaml pipeline(Deploy pipeline), Click the 3dots and choose Triggers
Go to Triggers--> Build completion and click add--> Select your triggering pipeline(CI pipeline)
Update:
I saw the pipeline resource in azure-deploy.yml is defined as below.
resources:
pipelines:
- pipeline: 'Deploy to Development'
source: 'DFE-Digital.dfe-teachers-payment-service'
trigger:
branches:
include:
- "master"
- "release-stuff"
please try changing the indentation of trigger element the same as source element. Check below example:
resources:
pipelines:
- pipeline: 'Deploy to Development'
source: 'DFE-Digital.dfe-teachers-payment-service'
trigger:
branches:
include:
- "master"
- "release-stuff"
I found the following:
In source pipeline I didn't need to create an artifact
In depends pipeline if I wanted to build after any commit to the source branch I could get it to work with this:
trigger: none
pr: none
resources:
pipelines:
- pipeline: 'depends'
source: 'common-gulp-trigger'
trigger: true
I may assume you are not working on the master branch, right? I have the same issue previously. But after I read the section Default branch for triggers of MS's doc. I understand why. The trigger only examine master's branch's yaml file by default. This means the pipeline will only be triggered by the definition of triggers in master branch's yaml file.
Therefore, whatever branches you add in the trigger section of yaml file in other branches(not master), tirgger is not active. You need to change the pipeline to look the yaml file in your current branch, not master. Just follow the doc's instruction, change the default trigger branch. You will get it working.
Once you merge your work into master, you probably need to change the dedault trigger branch back to master.
enter image description here
Hello - I'm using Gitlab 10.x Enterprise version.
This is in the context of a Production deployment to an environment having eight servers. The requirement is to deploy to one server first, and then after couple of days do a deployment to the remaining servers with the press of just one button.
Pipeline Stages:
Release-Tag-Creation -> Production-OneServer-Deployment -> OneButtonPush -> DeploytoAllServers
Question:
How can I tie the dependency between the stages "OneButtonPush" and "DeploytoAllServers"?. "DeploytoAllServers" stage should be kicked-off only when the Status of the job in the "OneButtonPush" stage is successful. "DeploytoAllServers" stage will have parallel jobs to deploy to each server.
I made few attempts based on the Gitlab CI documentation, but wasn't successful. Also, can the concept of Rolling deployment used in the context of GitLab-CI.
Thanks!
In Gitlab CI, the stages are run one after another. Each stage can have multiple jobs which run in parallel.
As per your use case, you'll need to have different stages each for Release-Tag-Creation, Production-OneServer-Deployment, OneButtonPush and DeploytoAllServers. You can have manual triggers for particular jobs (OneButtonPush in your case) by specifying when: manual in the job definition.
By default, if there is a job awaiting manual trigger, jobs from further stages will start executing considering the job with manual trigger as successful. To change this behavior one will need to use allow_failure: false. Mode details on allow_failure over here
If you want to run the Stage only after then specific one get successful you can use the needs
Example
image: maven:latest
stages:
- deploy_dev_lambda
- test_dev_lambda
deploy_dev_lambda:
stage: deploy_dev_lambda
image: maven:latest
script:
- ./deployLambda
environment: dev
variables:
TARGET_ENV: "dev"
TAG_NAME: main-$CI_COMMIT_SHORT_SHA
test_dev_lambda:
stage: test_dev_lambda
needs:
- deploy_dev_lambda
script:
- ./testLambda
So test_dev_lambda only run after the deploy_dev_lambda stage and if stage deploy_dev_lambda fails test_dev_lambda wont get executed.