Is there a way that we can trigger pipeline after another pipeline when triggering pipeline build on any existing/new tags?
like following:
resources:
pipelines:
- pipeline: name1
source: pipeline1
trigger:
branches:
include:
- master
tags:
- '*'
This is failing with
Tags set for trigger didn't match any pipeline.
I know we can add wild character with some character as prefix, something like
tags:
- 'tag*'
But in my repo tag naming convention is not consistent. which starts with numeric/any character.
I Looked here, but no luck yet.
https://github.com/microsoft/azure-pipelines-yaml/blob/master/design/pipeline-triggers.md
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/pipeline-triggers?view=azure-devops
Related
I would like to achieve this Gitlab CI/CD behaviour:
Two jobs defined in the pipeline:
job_beta triggered when a tag is created with a semver number containing a "-rc*" suffix. For example: v1.0.0-rc1.
job_production triggered when a tag is created with a semver number, without any suffix. For example: v1.0.0.
Is there a way to parse the $CI_COMMIT_TAG variable in the .gitlab-ci.yml file of the project? Or any other ideas about how to achieve this?
To achieve this kind of behaviour, you can rely on the only keyword to control which job will be created. Simply specify a regular expression that will either match a semver with the rc suffix or one without any suffix.
You could do it the following way if your semantic versions are prefixed with v:
build-beta:
image: your-build-image:latest
stage: build
only:
- /^v[0-9]+\.[0-9]+\.[0-9]+-rc[0-9]+$/
script:
- ./build-beta.sh # Do something...
build-prod:
image: your-build-image:latest
stage: build
only:
- /^v[0-9]+\.[0-9]+\.[0-9]+$/
script:
- ./build-prod.sh # Do something...
You can also achieve something similar with the rules keyword by matching $CI_COMMIT_REF_NAME with the appropriate regex in an if condition.
I have a yaml pipeline that builds and deploys a project. it deploys to dev & tst. But i dont always want to deploy to tst. at the moment i have solved this with approvals. when the dev stage finishes the pipeline waits for approval.
this introduces some problems:
all developers get an email every time, which can be annoying.
the pipeline waits for deployment to tst, even when deploymen to tst isnt needed. it keeps
waiting
new ci builds dont start when the pipeline is waiting
If there is a well defined way how to determine whether to run the tst stage or not, you can either specify a custom condition on the stage or conditionally include the stage using an expression
Specify a condition
From the docs:
You can specify the conditions under which each stage, job, or step
runs. By default, a job or stage runs if it does not depend on any
other job or stage, or if all of the jobs or stages that it depends on
have completed and succeeded.
There are a number of ways you can customize this behavior, either by built in or custom defined conditions, for example, to run tun the tst stage only if the branch is main:
stages:
- stage: dev
jobs:
- job: dev1
steps:
- script: echo Hello from dev!
- stage: tst
condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/main'))
jobs:
- job: tst1
steps:
- script: echo Hello From tst!
The tst stage will still be visible when you open the pipeline run, it will be marked as skipped
The condition under the tst stage can be altered to fit your need (it can include both variables, parameters and even output from previous stages.
Conditional expression
From the docs:
You can use if, elseif, and else clauses to conditionally assign
variable values or set inputs for tasks. You can also conditionally
run a step when a condition is met.
Conditional insertion only works with template syntax, which is evaluated when the pipeline yaml is compiled (i.e before the pipeline starts). Thus it cannot evaluate output of previous steps. In the below example the tst stage is completely removed from the pipeline run when the pipeline is instantiated (and the template compiled) if the branch is not main
stages:
- stage: dev
jobs:
- job: dev1
steps:
- script: echo Hello from dev!
- ${{ if eq(variables['Build.SourceBranch'], 'refs/heads/main') }}:
- stage: tst
jobs:
- job: tst1
steps:
- script: echo Hello From tst!
I want to create a pipeline trigger in azure devops that triggers only on changes in folder named src/Subscription.*
My Folder structure:
src
Subscription.Data
Subscriptin.Api
Subscription.Shared
How my pipeline looks
trigger:
branches:
include:
- development
- master
- release
paths:
include:
- src/Subscription.**/*
How can i change the include to trigger on file changes in all the folders ?
How can i change the include to trigger on file changes in all the folders ?
Until September 8, 2021, this is still a known request on our main product forum：
Support wildcards (*) in Trigger > Path Filters
Now, it is possible now as it is written here, but the function needs to be improved:
Wild cards can be used when specifying inclusion and exclusion branches for CI or PR triggers in a pipeline YAML file. However, they cannot be used when specifying path filters. For instance, you cannot include all paths that match src/app//myapp*. This has been pointed out as an inconvenience by several customers. This update fills this gap. Now, you can use wild card characters (, *, or ?) when specifying path filters.
Now, we could use * but not ** & *.
As workaround for this issue:
trigger:
branches:
include:
- development
- master
- release
paths:
include:
- src/Subscriptin.Api/*
- src/Subscription.Data/*
- src/Subscription.Shared/*
While there are many questions and answers around conditional job execution of GitLab CI pipelines I could not find a solution for my problem: To trigger a particular job from pipeline when there are changes in some files OR a env variable is set to a specific value. Something like this:
job_build:
tags:
- executor-shell
stage:
build
script:
- ./gitlab_ci_helper.sh build
only:
changes:
- /path/to/files/with/changes/*
variables:
- $BUILD_IS_A_MUST == "TRUE"
As per the documentation from GitLab itself, we have the following:
Now the question I have is how to make the condition NOT(any of variables) AND (any of changes)) which is same as (any of variables) OR (any of changes) but applied for the only parameter?
Well, in the end, looks like found myself the answer to the question. It turns out that starting from GitLab version 12.3 a new configuration parameter was introduced called rules:
Looks like this parameter is exactly addressing the problem I was looking for an answer. While I don't have yet GitLab 12.3 version to test it, the above ci job would be changed into something like this:
job_build:
tags:
- executor-shell
stage:
build
script:
- ./gitlab_ci_helper.sh build
rules:
- changes:
- /path/to/files/with/changes/*
when: on_success
- if: $BUILD_IS_A_MUST == "TRUE"
when: on_success
In this case, the default rule will get resolved to when: never which is what I'm after thus no need to specify it.
I have two AzureDevOps Git branches:
master
feature/mybranch
I have a multi-stage build pipeline defined in yaml, where some of the steps are templated into separate .yml files.
In my outer azure-pipelines.yml I reference a repository where my template .yml's live:
resources:
repositories:
- repository: templates
type: git
name: MyProject/MyRepo
when I'm building in the 'master' branch everything is good as by default the repository will look in refs/heads/master.
when I'm working in the feature branch and I want to test experimental changes to my template .yml files, I don't want it to fetch them from the master branch, I want it to use the files from the branch I am working in.
The following works and allows me to do this:
resources:
repositories:
- repository: templates
type: git
name: MyProject/MyRepo
ref: refs/heads/feature/mybranch
However, when I merge this back to master, I obviously don't want 'ref:' still pointing at the feature branch, so I'd like to generate the value of 'ref:' dynamically with a variable.
I've tried using ref: $(Build.SourceBranch) where $(Build.SourceBranch) should expand to 'refs/heads/feature/mybranch'
But it doesn't work. Error:
62638: "/azure-pipelines.yml: Could not get the latest source version for repository MySolution hosted on Azure Repos using ref refs/heads/$(Build.SourceBranch)."
Instead of referencing the repo in resources, use inline checkout as described here
https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/multi-repo-checkout?view=azure-devops#checking-out-a-specific-ref
- checkout: git://MyProject/MyRepo#features/tools
And this yaml element allows use of template expressions using variables, parameters e.g.
- checkout: git://${{ variables.repoName}}#${{ variables.branchRef }}
OR
- checkout: git://${{ parameters.repoName}}#${{ parameters.branchRef }}
And you can change that dynamically
Or the other alternative is use script task as below
- script: |
# note checkout: git://$(System.TeamProject)/${{ parameters.repoName }}#${{ parameters.repoRef }} this does not work if this task is run multiple times in same pipeline
# see here for more details :https://developercommunity.visualstudio.com/t/checkout-fails-with-an-error-occurred-while-loadin/1005991#T-N1270459
repoDir=$(Agent.BuildDirectory)/${{ parameters.repoName }}
/bin/rm -rf $repoDir
url_with_token=$(echo $(System.CollectionUri) | sed -e "s/https\:\/\//https\:\/\/$(System.AccessToken)\#/g")
git clone $url_with_token/$(System.TeamProject)/_git/${{ parameters.repoName }} $repoDir
cd $repoDir
git fetch origin '${{ parameters.repoRef }}':'localBranch'
git checkout localBranch
name: clone_script
displayName: Checkout using script ${{ parameters.repoName }}#${{ parameters.repoRef }}
I have added above as a template and its usage in a gist, to make it re-usable easily.
Hope that helps
https://gist.github.com/scorpionlion/1773d08b62ca5875cc2fd6dcdd0394d2
Is it possible to use a variable in the ref property of
resources:repository for Azure DevOps YAML?
For this question, the answer is Yes, it's possible.
About why you receive that error message, just is the variable($(Build.SourceBranch)) you used is incorrect. You should use $(Build.SourceBranchName).
As normal, for ref, we should input master or any other feature branches. Such as
ref: refs/heads/master
This may make you thought that this is same with the value of $(Build.SourceBranch). It looks same, I know, but different. In fact, for server, it will read the exactly branch name not the branch path, which we can clearly figure out with the classic editor type:
According with classic editor type, we can know here we should input the exactly branch name.
So, as the Predefined variables defined, the value of $(Build.SourceBranch) is the branch path, but for $(Build.SourceBranchName), it's represent a exactly branch name.
So, if you want to execute successfully, you need to use : $(Build.SourceBranchName). And it's worked on my side.
Hope this also can help you stay away from the error message.
Edit:
The complete script which is worked for me is:
resources:
repositories:
- repository: templates
type: git
name: MyApp/MyconApp
ref: $(Build.SourceBranchName)
The azure docs state
Variables can't be used to define a repository in a YAML statement.
So that seems to place some limitations on what you can do here. Perhaps there is a workaround that still allows you to do what you want.