Tracking Azure DevOps build activities - azure

I have set up multiple Azure DevOps build pipelines that are triggered by submitting pull requests to a GitHub repo. Whenever a build happens, I can navigate to that build's page and see its build steps and whether the build was successful. How do I pull that information out programmatically?

You can get a specific build status by restful api
https://dev.azure.com/{organization}/{project}/_apis/build/builds/{buildId}?api-version=5.1
But I guess you probably want a build status badge to be added to your git repo.
First go to the Pipelines page in your Azure Devops, select the pipeline of your git repo.
On the right top corner, click on the eclipse and choose status badge
Copy the sample markdown link.
Then go to your github repo, and paste the status markdown link in the read.me file and commit the change to your repo. The build status will be shown as below in your github repo
And aslo you need go to the Project Settings of your azure devops. Go to the Settings under Pipelines. Make sure Allow anonymous access to badges checkbox under General

To pull information out of Azure DevOps programmatically, I can use the Azure DevOps Python API. This SDK provides multiple clients to pull various data from Azure DevOps. As #Levi Lu-MSFT's answer hinted, the data I wanted can be obtained from the build client.
I installed the SDK in a conda environment with this YAML:
name: DevOpsData
channels:
- conda-forge
dependencies:
- python=3.6
- pip==19.2.3
- nb_conda_kernels==2.2.1
- papermill==1.0.1
- pandas==0.23.4
- scikit-learn==0.20.0
- lightgbm==2.2.1
- pip:
- prompt_toolkit==2.0.9
- azure-cli==2.0.69
- azure-devops
I pulled what I needed using this script:
# Copyright (C) Microsoft Corporation. All rights reserved.
from azure.devops.connection import Connection
from msrest.authentication import BasicAuthentication
import pprint
import datetime
if __name__ == "__main__":
# Fill in with your personal access token and org URL
personal_access_token = "MY_ACDESS_TOKEN"
organization_url = "https://dev.azure.com/MY_ORGANIZATION"
# Create a connection to the org
credentials = BasicAuthentication("", personal_access_token)
connection = Connection(base_url=organization_url, creds=credentials)
# Get a client (the "core" client provides access to projects, teams, etc)
build_client = connection.clients.get_build_client()
# Get the first page of builds
project = "MY_PROJECT_NAME"
get_builds_response = build_client.get_builds(project=project)
index = 0
while get_builds_response is not None:
for build in get_builds_response.value:
duration = build.finish_time - build.start_time
seconds = duration.days*(24*60*60)+duration.seconds
print("[{}]\t{}\t{}\t{}\t{:,} seconds".format(
index, build.build_number, build.source_branch, build.result,
seconds
))
index += 1
if (get_builds_response.continuation_token is not None
and get_builds_response.continuation_token != ""):
# Get the next page of builds
get_builds_response = build_client.get_builds(
continuation_token=get_builds_response.continuation_token)
else:
# All builds have been retrieved
get_builds_response = None

Related

Azure Yaml Pipeline - Update Workitems from multiple Repositories

In a nutshell, I am looking to get a list of work items linked to a git branch.
In more detail
I am working with 4 repositories that I add as resources to my pipeline
- repository: Cms # code repository
name: <ProjectName>/Cms
type: git
ref: develop2022
- repository: QA-Automation # Automated Testing Repo
name: <ProjectName>/QA-Automation
type: git
ref: main
- repository: TdsWDPExplorer # Generate Reports Repo
name: <ProjectName>/TdsWDPExplorer
type: git
ref: master
The Pipeline yaml files them self's are in the 4th Repo and checked out as self
- checkout: Self
path: s/DE-DevOps
I am trying to update the work items associated with the Cms Repository.
I tried using the Workitem Updater task https://marketplace.visualstudio.com/items?itemName=BlueBasher.bluebasher-workitemupdater
But it only sees the workitems associated with the Repository holding the yaml Files (Self).
I also looked at the API to get a list of the work items.
_apis/git/repositories//refs?filter=heads%2fBRANCHNAME&includeLinks=true
Gives me details to a branch but I didn't find the linked work items
Also looking at the workitem I dint see that info
_apis/wit/workitems?ids=ITEM-ID's&$expand=all&api-version=6.0
I am thinking it might be somewhere in _apis/wit/reporting/workitemlinks but haven been able to get the info.
I found a answer that works for me, in a response to Obtain all work items from Azure DevOps that have been merged into a branch via JavaScript
we link the work items to pull requests, I can use the API to query the pull requests in to the given branch and get the linked work items like:
https://dev.azure.com/Organisation/Project/_apis/git/repositories/Cms/pullrequests?searchCriteria.status=completed&searchCriteria.targetRefName=refs/heads/BRANCHNAME&api-version=6.0
I should now be able to extract the ID's and pass them in a PS loop to the Item Updater Task https://marketplace.visualstudio.com/items?itemName=BlueBasher.bluebasher-workitemupdater or using the API to update the workitem
There is also the option to call the API for the pipeline run info https://learn.microsoft.com/en-us/rest/api/azure/devops/pipelines/runs/get?view=azure-devops-rest-7.1#runresources
But I didn't see a way to extract the work items but the information must be somewhere there as well because I can see the work items listed in the pipeline view by resource

Python : Implementing Feature Flags based on Environment (dev, prod)

I would like the features to be based on the environment. For example a feature is being worked or tested so I could have it on in DEV , but it's not ready for the public, so it's turned off in PROD.
Do I need to implement a custom strategy or can I use one of the existing strategies in a creative way?
If there any concise example that would be most helpful.
The easiest way I’ve found to implement feature flags at the environment level is to use a third-party hosted management system. Many feature flag services allow you to control which environment a flag will be enabled in. In the example below, I used DevCycle’s Python SDK, referencing feature flags and environments I had created in the DevCycle dashboard.
First, I installed the SDK from the command line:
$ pip install devcycle-python-server-sdk
Then I initialized the SDK it with the SDK key that corresponded to my desired environment. In this case, I used the key for my Dev environment. DevCycle provides SDK keys for each environment you set up.
from __future__ import print_function
from devcycle_python_sdk import Configuration, DVCClient
from devcycle_python_sdk.rest import ApiException
configuration = Configuration()
# Set up authorization
configuration.api_key['Authorization'] = 'SDK_KEY_FOR_DEV_ENV'
# Create an instance of the API class
dvc = DVCClient(configuration)
# Create user object. All functions require user data to be an instance of the UserData class
user = UserData(
user_id='test'
)
key = 'enable-feature-flag' # feature flag key created in the DevCycle Dashboard
try:
# Fetch variable values using the identifier key, with a default value and user object
# The default value can be of type string, boolean, number, or JSON
flag = dvc.variable(user, key, False)
# Use received value of feature flag.
if flag.value:
# Put feature code here, or launch feature from here
else:
# Put existing functionality here
except ApiException as e:
print("Exception when calling DVCClient->variable: %s" %e)
By passing in the SDK key for my Dev environment, 'SDK_KEY_FOR_DEV_ENV', it gives my program access to only the features enabled in Dev. You can choose which environment(s) a feature is enabled in directly from the DevCycle dashboard. So if 'enable-feature-flag' was set to true for your Dev environment, you would see your feature. Likewise, you could set 'enable-feature-flag' to false in your Prod environment, and replace 'SDK_KEY_FOR_DEV_ENV' with the key for your Prod environment. This would disable the new functionality from Prod.
Full disclosure: My name is Sandrine and I am a Developer Advocate for DevCycle. I hope this answer helps you get started on environment-specific feature flags.

How to get branch name from commit id using azure Devops REST API?

Scenario: I need to get when was the latest commit done in the repo and by whom, and to which branch did that user do the commit on?
Solution:
I'm using python azure.devops module. and here is my code:
cm_search_criteria = models.GitQueryCommitsCriteria(history_mode='firstParent', top=10)
commits = git_client.get_commits(repo.id, search_criteria=cm_search_criteria, project=project.name)
for i in commits:
datetimeobj = datetime.strptime(i.committer.date.strftime("%x"), '%m/%d/%y')
last_commit_on = datetimeobj.date()
last_commit_by = i.committer.email
break
Now how do I get the branch name to which the user had committed the code? In the UI we can see the branch name... how can i get the same data using Azure Devops REST API ?
enter image description here
you may need to use Stats - List to retrieve statistics about all branches within a repository, then evaluate the latest commit, you also need to consider argument of baseVersionDescriptor.versionOptions of firstParent
I'm not sure if Python wrapper module support this seems like the github project is achieved now.

How to give repository access to installed GitHub app

I am using google cloud build for CI/CD purpose, in which I need to give access for specific repositories (can't use All repositories option). Is there any way to provide access by repository wise through python code. If not possible through python is there any alternative to this requirement.
Thanks,
Raghunath.
When I checked with GitHub support, they have shared me below links.
To get repository id -- https://developer.github.com/v3/repos/#get-a-repository
To get installations details -- https://developer.github.com/v3/orgs/#list-installations-for-an-organization
To add repository to an installation -- https://developer.github.com/v3/apps/installations/#add-repository-to-installation
I used these links to create below mentioned code which has helped me to implement the desired requirement.
# Header to use in request
header = dict()
header['Authorization'] = 'token %s' % personal_token
header['Accept'] = 'application/vnd.github.machine-man-preview+json'
# Fetching repository id
url = f'https://api.github.com/repos/{organization_name}/{repo_name}'
output = requests.get(url, headers=header)
repo_id = output.json()['id']
# Adding repository to Google Cloud build installation
url = f'https://api.github.com/user/installations/{gcb_installation_id}/repositories/{repo_id}'
output = requests.put(url, headers=header)

How to establish a continuous deployment of non-.NET project/solution to Azure?

I have connected Visual Studio Online to my Azure website. This is not a .NET ASP.NET MVC project, just several static HTML files.
Now I want to get my files uploaded to Azure and available 'online' after my commits/pushes to the TFS.
When a build definition (based on GitContinuousDeploymentTemplate.12.xaml) is executed it fails with an obvious message:
Exception Message: The process parameter ProjectsToBuild is required but no value was set.
My question: how do I setup a build definition so that it automatically copies my static files to Azure on commits?
Or do I need to use a different tooling for this task (like WebMatrix).
update
I ended up with creating an empty website and deploying it manually from Visual Studio using webdeploy. Other possible options to consider to create local Git at Azure.
Alright, let me try to give you an answer:
I was having quite a similar issue. I had a static HTML, JS and CSS site which I needed to have in TFS due to the project and wanted to make my life easier using the continuous deployment. So what I did was following:
When you have a Git in TFS, you get an URL for the repository - something like:
https://yoursite.visualstudio.com/COLLECTION/PROJECT/_git/REPOSITORY
, however in order to access the repository itself, you need to authenticate, which is not currently possible, if you try to put the URL with authentication into Azure:
https://username:password#TFS_URL
It will not accept it. So what you do, in order to bind the deployment is that you just put the URL for repository there (the deployment will fail, however it will prepare the environment for us to proceed).
However, when you link it there, you can get DEPLOYMENT TRIGGER URL on the Configure tab of the Website. What it is for is that when you push a change to your repository (say to GitHub) what happens is that GitHub makes a HTTP POST request to that link and it tells Azure to deploy new code onto the site.
Now I went to Kudu which is the underlaying system of Azure Websites which handles the deployments. I figured that if you send correct contents in the HTTP POST (JSON format) to the DEPLOYMENT TRIGGER URL, you can have it deploy code from any repository and it even authenticates!
So the thing left to do is to generate the alternative authentication credentials on the TFS site and put the whole request together. I wrapped this entire process into the following PowerShell script:
# Windows Azure Website Configuration
#
# WAWS_username: The user account which has access to the website, can be obtained from https://manage.windowsazure.com portal on the Configure tab under DEPLOYMENT TRIGGER URL
# WAWS_password: The password for the account specified above
# WAWS: The Azure site name
$WAWS_username = ''
$WAWS_password = ''
$WAWS = ''
# Visual Studio Online Repository Configuration
#
# VSO_username: The user account used for basic authentication in VSO (has to be manually enabled)
# VSO_password: The password for the account specified above
# VSO_URL: The URL to the Git repository (branch is specified on the https://manage.windowsazure.com Configuration tab BRANCH TO DEPLOY
$VSO_username = ''
$VSO_password = ''
$VSO_URL = ''
# DO NOT EDIT ANY OF THE CODE BELOW
$WAWS_URL = 'https://' + $WAWS + '.scm.azurewebsites.net/deploy'
$BODY = '
{
"format": "basic",
"url": "https://' + $VSO_username + ':' + $VSO_password + '#' + $VSO_URL + '"
}'
$authorization = "Basic "+[System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($WAWS_username+":"+$WAWS_password ))
$bytes = [System.Text.Encoding]::ASCII.GetBytes($BODY)
$webRequest = [System.Net.WebRequest]::Create($WAWS_URL)
$webRequest.Method = "POST"
$webRequest.Headers.Add("Authorization", $authorization)
$webRequest.ContentLength = $bytes.Length
$webRequestStream = $webRequest.GetRequestStream();
$webRequestStream.Write($bytes, 0, $bytes.Length);
$webRequest.GetResponse()
I hope that what I wrote here makes sense. The last thing you would need is to bind this script to a hook in Git, so when you perform a push the script gets automatically triggered after it and the site is deployed. I haven't figured this piece yet tho.
This should also work to deploy a PHP/Node.js and similar code.
The easiest way would be to add them to an empty ASP .NET project, set them to be copied to the output folder, and then "build" the project.
Failing that, you could modify the build process template, but that's a "last resort" option.

Resources