I'm using Azure DevOps's multiple repository functionality, documented here:
​https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/multi-repo-checkout?view=azure-devops
I have my YAML file in one repo, and the pipeline points to that YAML. The YAML has a trigger set up for another repository resource, so that when that repo gets updated, the pipeline will be triggered:
resources:
repositories:
- repository: MyRepo
endpoint: 'MyRepos'
type: git
name: RepoName
trigger:
- '*'
The documentation claims that the 'Build.SourceBranch' variable will allow me to determine which branch in MyRepo triggered the pipeline build: "When an update to one of the repositories triggers a pipeline, then the following variables are set based on triggering repository"
However, this doesn't seem to be the case. No matter what branch triggers the build, 'Build.SourceBranch' is always 'refs/heads/master', presumably because the repo that holds the YAML has 'master' as its default branch.
I can't find any environment variable that gets set to the name of the branch that triggered the build, either. So how can I get the name of the branch that triggered the build? If there's no possible way, I think this needs to be added!
The issue is:
According to the document, Build.SourceBranch is set based on triggering repository. However, its value is determined by repo in which the YAML file resides in practice.
I have done following tests. There are two repos, 'RepoA' and 'RepoB'. Both repos have two branches, 'master' and 'bran'. And the YAML file is in 'master' of 'RepoA'
Commit a change in 'bran' of 'RepoB'. The value of Build.SourceBranch is refs/heads/master. It is not consistent with the documentation.
Commit a change in 'bran' of 'RepoA'. The value of Build.SourceBranch is refs/heads/bran. It is consistent with the documentation.
Commit a change in 'master' of 'RepoB'. The value of Build.SourceBranch is refs/heads/master. It is consistent with the documentation.
Commit a change in 'master' of 'RepoA'. The value of Build.SourceBranch is refs/heads/master. It is consistent with the documentation.
Thus, if the build is triggered by 'RepoA', Build.SourceBranch can successfully represent the true branch. However, if the build is triggered by 'RepoB', the value of Build.SourceBranch are always refs/heads/master.
We have reported this issue to the product group.
Related
I need to integrate a Gitlab CICD pipeline with a custom Delivery Manager Tool. My pipeline will need to invoke the delivery manager API with some info.
In Jenkins we developed a plugin that provided a pipeline step - void deployEnv (String app, String environment ) - that can be used in the different stages, e.g.:
deployEnv("applicationx", "production")
Is there a way to develop a similar plugin in Gitlab CICD?
Is it possible to invoke a remote URL from a Gitlab CICD pipeline passing some credentials?
The closest analog for this kind of "plugin" in GitLab CI is probably a CI templated job definition. There's maybe a few ways to formulate this. There are a number of methods for abstracting and including job definitions provided by others. Some of the basic tools are: include:, extends:, !reference, "hidden job" keys, and YAML anchors.
Providing reusable templates
If you just need to provide an abstraction for a series of steps, a "hidden key" definition would be the closest to what you want.
Consider the following template YAML. This might be embedded directly in your .gitlab-ci.yml file(s) or you might choose to include it any number of configurations from a remote location using the include: keyword.
In this fictional example, we provide a script step that expects two environment variables to be present: APPLICATION_NAME and DEPLOYMENT_ENV. These variables are used (in this fictional example) to call a remote API passing those values as path parameters. Here, the definitions are provided in a "hidden job" key
.deploy_to_env:
image: curlimages/curl # or otherwise have `curl` in your environment
script:
- |
if [[ -z "$APPLICATION_NAME" || -z "$DEPLOYMENT_ENV" ]]; then
echo "FATAL: you must set APPLICATION_NAME and DEPLOYMENT_ENV variables"
exit 1
fi
- curl -XPOST "https://my-deployment-api.example.com/${APPLICAITON_NAME}/${DEPLOYMENT_ENV}"
Let's assume this yaml file exists in a file named deploy.yml in a project whose path is my-org/templates.
Using templates
Now let's say a pipeline configuration wants to leverage the above definition to deploy an application named applicationx to production.
First, in any case, the project should include: the remote definition (unless you choose to embed it directly -- e.g., copy/paste).
include:
- project: my-org/templates
file: deploy.yml
ref: main # or any git ref, or omit to use default branch
Then you can use the extends: keyword to form a concrete job from the hidden key.
deploy_production:
stage: deploy
extends: .deploy_to_env
variables:
APPLICATION_NAME: "applicationx"
DEPLOYMENT_ENV: "production"
Or, if you want to embed the deployment steps in the middle of other script steps using !reference is useful here.
deploy_production:
stage: deploy
script:
- export APPLICATION_NAME="applicationx"
- export DEPLOY_ENV="production"
# these could also be set in `variables:`
- echo "calling deployment API to deploy ${APPLICATION_NAME} to ${DEPLOY_ENV}"
- !reference [.deploy_to_env, script]
- echo "done"
There's a lot of ways to handle this, these are just two examples.
In a nutshell, I am looking to get a list of work items linked to a git branch.
In more detail
I am working with 4 repositories that I add as resources to my pipeline
- repository: Cms # code repository
name: <ProjectName>/Cms
type: git
ref: develop2022
- repository: QA-Automation # Automated Testing Repo
name: <ProjectName>/QA-Automation
type: git
ref: main
- repository: TdsWDPExplorer # Generate Reports Repo
name: <ProjectName>/TdsWDPExplorer
type: git
ref: master
The Pipeline yaml files them self's are in the 4th Repo and checked out as self
- checkout: Self
path: s/DE-DevOps
I am trying to update the work items associated with the Cms Repository.
I tried using the Workitem Updater task https://marketplace.visualstudio.com/items?itemName=BlueBasher.bluebasher-workitemupdater
But it only sees the workitems associated with the Repository holding the yaml Files (Self).
I also looked at the API to get a list of the work items.
_apis/git/repositories//refs?filter=heads%2fBRANCHNAME&includeLinks=true
Gives me details to a branch but I didn't find the linked work items
Also looking at the workitem I dint see that info
_apis/wit/workitems?ids=ITEM-ID's&$expand=all&api-version=6.0
I am thinking it might be somewhere in _apis/wit/reporting/workitemlinks but haven been able to get the info.
I found a answer that works for me, in a response to Obtain all work items from Azure DevOps that have been merged into a branch via JavaScript
we link the work items to pull requests, I can use the API to query the pull requests in to the given branch and get the linked work items like:
https://dev.azure.com/Organisation/Project/_apis/git/repositories/Cms/pullrequests?searchCriteria.status=completed&searchCriteria.targetRefName=refs/heads/BRANCHNAME&api-version=6.0
I should now be able to extract the ID's and pass them in a PS loop to the Item Updater Task https://marketplace.visualstudio.com/items?itemName=BlueBasher.bluebasher-workitemupdater or using the API to update the workitem
There is also the option to call the API for the pipeline run info https://learn.microsoft.com/en-us/rest/api/azure/devops/pipelines/runs/get?view=azure-devops-rest-7.1#runresources
But I didn't see a way to extract the work items but the information must be somewhere there as well because I can see the work items listed in the pipeline view by resource
I'm quite new to CI/CD and basically I'm trying to add this job to Gitlab CI/CD that will run through the repo looking for secret leaks. It requires some API key to be passed there. I was able to directly insert this key into .gitlab-ci.yml and it worked as it was supposed to - failing the job and showing that this happened due to this key being in that file.
But I would like to have this API key to be stored in .env file that won't be pushed to a remote repo and to pull it somehow into .gitlab-ci.yml file from there.
Here's mine
stages:
- scanning
gitguardian scan:
variables:
GITGUARDIAN_API_KEY: ${process.env.GITGUARDIAN_API_KEY}
image: gitguardian/ggshield:latest
stage: scanning
script: ggshield scan ci
The pipeline fails with this message: Error: Invalid API key. so I assume that the way I'm passing it into variables is wrong.
CI variables should be available in gitlab-runner(machine or container) as environment variables, they are either predefined and populated by Gitlab like the list of predefined variables here, or added by you in the settings of the repository or the gitlab group Settings > CI/CD > Add Variable.
After adding variables you can use the following syntax, you can test if the variable has the correct value by echoing it.
variables:
GITGUARDIAN_API_KEY: "$GITGUARDIAN_API_KEY"
script:
- echo "$GITGUARDIAN_API_KEY"
- ggshield scan ci
I would like to use Parent\Child pipilens https://docs.gitlab.com/ee/ci/parent_child_pipelines.html in this way.
I have this source structure:
- backend
--- .gitlab-ci.yml
--- src
- frontend
--- .gitlab-ci.yml
--- src
-.gitlab-ci.yml
I want to trigger backend or frontend .gitlab-ci.yml based on the path where new commit happens: if it happend on frontend, only frontend.gitlab-ci.yml should be used for build\publish.
Is it possible?
You can specify to execute different pipelines based on where the changes in the code occurred using the only: changes configuration documented here.
You can therefor specify to execute a pipeline frontend only if changes happen within the frontend folder (analog for `backend).
You can use the include: local feature (documented here) to include the frontend/.gitlab-ci.yml-file within the pipeline for the frontend that is defined in the root .gitlab-ci.yml.
For examples on how to exactly configure the pipeline so that it triggers a configuration provided in a local file, please see here.
Parent-child pipelines also support the only: changes configuration as documented here.
I can invoke a simple Jenkins pipeline from gitlab merge requests using a webhook. Now I'd like to know what is the source branch to make the checkout against it. Example: if I push code to develop branch, in my pipeline script i'd checkout develop branch. Thanks.
node {
stage('Build') {
def mybranch = '?' // get branch name from gitlab webhook
git branch: mybranch,
credentialsId: 'mycredential',
url: 'myurl'
// ...
}
}
You can get the current branch name via env.gitlabBranch.
Reference: gitlab-plugin
You can parameterize your pipeline and use the webhook payload data to fill in the branch value as described here.
GitLab plugin creates a lot of useful environment variable. You can see them in here. I think the one you need is CI_COMMIT_REF_NAME