Auto abandon Pull Request after 2 weeks with no updates - azure

I am new in azure devops and scripting. I want to auto abandon azure pull request after 2 weeks with no updates. I am able to list pull request, and do the status update. But looking for a powershell or bash script which can read the list command output, find out the date, and abandon pull request if no update for two week.
az repos pr list --repository "myrepo" --status "active"
az repos pr update --id "16729" --status "abandoned"

The below code can achieve your requirements.
$org_name = "<Your Organization Name>"
$project_name = "<Your Project Name>"
$repo_name = "<Your Repo Name>"
$PAT = "<Your Personal Access Token>"
$headers_prs = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers_prs.Add("Authorization", "Basic "+$PAT)
$prs_url = "https://dev.azure.com/"+$org_name+"/"+$project_name+"/_apis/git/repositories/"+$repo_name+"/pullrequests?api-version=6.0"
$response_prs = Invoke-RestMethod $prs_url -Method 'GET' -Headers $headers_prs
$response_prs | ConvertTo-Json
$response_prs.value | foreach {
$pr_id = $_.pullRequestId
# Write-Host $pr_id
$create_date = $_.creationDate
# Write-Host $create_date # Get all of the PRs' create dates
###
$headers_pr = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers_pr.Add("Authorization", "Basic "+$PAT)
$headers_pr.Add("Content-Type", "application/json")
$body_pr = "{
`n `"contributionIds`": [
`n `"ms.vss-code-web.pullrequests-artifact-stats-data-provider`"
`n ],
`n `"dataProviderContext`": {
`n `"properties`": {
`n `"artifactIds`": [
`n {
`n `"artifactId`": `"vstfs:///Git/PullRequestId/c6358b04-e91a-4bd1-a894-1adb543134d6%2F044a82e2-9d12-43b2-801c-d33ac6812173%2F"+$pr_id+"`",
`n `"discussionArtifactId`": `"vstfs:///CodeReview/ReviewId/c6358b04-e91a-4bd1-a894-1adb543134d6%2F"+$pr_id+"`"
`n }
`n ],
`n `"sourcePage`": {
`n `"url`": `"https://dev.azure.com/<Organization Name>/<Project Name>/_git/<Repo Name>/pullrequests?_a=mine`",
`n `"routeId`": `"ms.vss-code-web.lwp-prs-route`",
`n `"routeValues`": {
`n `"project`": `"<Project Name>`",
`n `"GitRepositoryName`": `"<Repo Name>`",
`n `"vctype`": `"git`",
`n `"controller`": `"ContributedPage`",
`n `"action`": `"Execute`",
`n `"serviceHost`": `"b1cc953d-b564-4eec-a222-84393e4406b1 (<some personal information>)`"
`n }
`n }
`n }
`n }
`n}"
$pr_url = "https://dev.azure.com/"+$org_name+"/_apis/Contribution/HierarchyQuery/project/"+$project_name+"?api-version=5.0-preview.1"
$response_pr = Invoke-RestMethod $pr_url -Method 'POST' -Headers $headers_pr -Body $body_pr
$response_pr | ConvertTo-Json
$last_updated_time_stamp = $response_pr.dataProviders.'ms.vss-code-web.pullrequests-artifact-stats-data-provider'.'TFS.VersionControl.PullRequestListArtifactStatsProvider.artifactStats'.lastUpdatedDate
if($last_updated_time_stamp -ne $null){
Write-Host 'last update time stamp is not null.'
Write-Host $last_updated_time_stamp
$last_updated_time = Get-Date -Date $last_updated_time_stamp
$now_time = Get-Date
Write-Host $now_time
$diff_time = $now_time - $last_updated_time
Write-Host $diff_time.TotalSeconds
#if the diff is longer than two weeks
if ($diff_time.TotalSeconds -gt 60*60*24*14) {
Write-Host "The difference is greater than two weeks"
#Abandon the PR here.
#==========================================================
$headers_abpr = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers_abpr.Add("Authorization", "Basic "+$PAT)
$headers_abpr.Add("Content-Type", "application/json")
$body_abpr = "{
`n `"status`":`"abandoned`"
`n}"
$abpr_url = "https://dev.azure.com/"+$org_name+"/"+$project_name+"/_apis/git/repositories/"+$repo_name+"/pullrequests/"+$pr_id+"?api-version=6.0"
$response_abpr = Invoke-RestMethod $abpr_url -Method 'PATCH' -Headers $headers_abpr -Body $body_abpr
$response_abpr | ConvertTo-Json
Write-Host 'Abandoned successfully.'
#==========================================================
} else {
Write-Host "The difference is less than two weeks"
}
}elseif ($last_updated_time_stamp -eq $null) {
Write-Host 'last update time stamp is null.'
Write-Host $last_updated_time_stamp
$last_updated_time = Get-Date -Date $create_date
$now_time = Get-Date
$diff_time = $now_time - $last_updated_time
Write-Host $diff_time.TotalSeconds
if ($diff_time.TotalSeconds -gt 60*60*24*14) {
Write-Host "The difference is greater than two weeks"
#Abandon the PR here.
#==========================================================
$headers_abpr = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers_abpr.Add("Authorization", "Basic "+$PAT)
$headers_abpr.Add("Content-Type", "application/json")
$body_abpr = "{
`n `"status`":`"abandoned`"
`n}"
$abpr_url = "https://dev.azure.com/"+$org_name+"/"+$project_name+"/_apis/git/repositories/"+$repo_name+"/pullrequests/"+$pr_id+"?api-version=6.0"
$response_abpr = Invoke-RestMethod $abpr_url -Method 'PATCH' -Headers $headers_abpr -Body $body_abpr
$response_abpr | ConvertTo-Json
Write-Host 'Abandoned successfully.'
#==========================================================
} else {
Write-Host "The difference is less than two weeks"
}
}
###
}
The official REST API documents don't have such data 'last update time' of Pull requests. The above code is based on network traffic interception and analysis. The above code has been tested by me and works very well on my side.
There may be a lot of code, I will share with you the ideas of how I code those here. These will help you understand the above code.
Steps:
1, get all of the pull requests information, especially the 'id'.
https://learn.microsoft.com/en-us/rest/api/azure/devops/git/pull-requests/get-pull-requests?view=azure-devops-rest-6.0&tabs=HTTP
ID and Name in this REST API is equivalent.
2, Go to the pull request page and analyze.
Please notice that if you only create pull request but do nothing, last updated time in this place will be 'null', you need to use pull request 'creation time' in this situation.
3, calculate the difference between the last update time and the now time.
$ago = Get-Date -Date "<Time String>"
$now = Get-Date
$diff = $now - $ago
Write-Host $diff.TotalSeconds
#if the diff is longer than two weeks
if ($diff.TotalSeconds -gt 60*60*24*14) {
Write-Host "The difference is greater than two weeks"
} else {
Write-Host "The difference is less than two weeks"
}
The most important is the second step, because only this place has the relevant information of last updated time of pull request.
As long as you have ideas about how to deal with the problem, you can write code in any language you can to achieve your requirements.

#Bowman Zhu-MSFT Helped me to resolve this issue, Thanks for guidance, and as he said "As long as you have ideas about how to deal with the problem, you can write code in any language you can to achieve your requirements. "I only tweaked some of the code, Like for header I am using different code, using az cli for abandon PR.
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
$org_name = "XXX"
$project_name = "XXXX"
$repo_name = "XXX"
$PAT = $env:AZURE_DEVOPS_EXT_PAT
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f "", $PAT)))
$headers_prs = #{Authorization = ("Basic {0}" -f $base64AuthInfo) }
$prs_url = "https://dev.azure.com/"+$org_name+"/"+$project_name+"/_apis/git/repositories/"+$repo_name+"/pullrequests?api-version=6.0"
$response_prs = Invoke-RestMethod $prs_url -Method 'GET' -Headers $headers_prs
$response_prs.value | foreach {
Write-Host 'repository name' $repo_name
$pr_id = $_.pullRequestId
Write-Host 'pr id =' $pr_id
$create_date = $_.creationDate
Write-Host 'pr create date = ' $create_date # Get all of the PRs' create dates
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f "", $PAT)))
$headers_pr = #{Authorization = ("Basic {0}" -f $base64AuthInfo) }
$pr1_url = "https://dev.azure.com/$org_name/$project_name/_apis/git/repositories/$repo_name/pullRequests/$pr_id/threads?api-version=6.0"
$response_pr1 = Invoke-RestMethod $pr1_url -Method 'GET' -Headers $headers_pr
$count = $response_pr1.value.count
$last_updated_time_stamp = $response_pr1.value[$count-1].lastUpdatedDate
if($last_updated_time_stamp -ne $null){
Write-Host 'last update time stamp is not null.'
#Write-Host $last_updated_time_stamp
$last_updated_time = Get-Date -Date $last_updated_time_stamp
Write-Host 'last_updated_time = ' $last_updated_time
$now_time = Get-Date
Write-Host 'current time = ' $now_time
$diff_time = $now_time - $last_updated_time
Write-Host 'time difference in seconds = ' $diff_time.TotalSeconds
#if the diff is longer than four weeks
if ($diff_time.TotalSeconds -gt 60*60*24*30) {
Write-Host "The difference is greater than four weeks"
#Abandon the PR here.
#==========================================================
az devops configure --defaults organization=$org_name
az repos pr update --id $pr_id --status "abandoned"
Write-Host 'Abandoned successfully.'
#==========================================================
} else {
Write-Host "The difference is less than four weeks"
}
}elseif ($last_updated_time_stamp -eq $null) {
#Write-Host $last_updated_time_stamp
$last_updated_time = Get-Date -Date $create_date
Write-Host 'last_updated_time = ' $last_updated_time
$now_time = Get-Date
Write-Host 'current time = ' $now_time
$diff_time = $now_time - $last_updated_time
Write-Host 'time difference in seconds = ' $diff_time.TotalSeconds
#if the diff is longer than four weeks
if ($diff_time.TotalSeconds -gt 60*60*24*30) {
Write-Host "The difference is greater than four weeks"
#Abandon the PR here.
#==========================================================
az devops configure --defaults organization=$org_name
az repos pr update --id $pr_id --status "abandoned"
Write-Host 'Abandoned successfully.'
#==========================================================
} else {
Write-Host "The difference is less than four weeks"
}
}
###
}
env:
AZURE_DEVOPS_EXT_PAT: $(PAT_svc_azuresync)

Related

Azure devops build pipeline depends on other build pipeline

I have four projects. One is a common project for other three projects.
Other three project build pipeline are depend on common build pipeline. When common build pipeline is in progress, other three build pipeline should be wait until common build complete. How to achive this in on premise azure devops?
How to achive this in on premise azure devops?
You could add a PowerShell task at the beginning of the other three pipelines.
Here is Powershell script sample:
$token = "PAT"
$url="https://{instance}/{collection}/{project}/_apis/build/definitions/{definitionId}?includeLatestBuilds=true&api-version=5.1"
$token = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($token)"))
$response = Invoke-RestMethod -Uri $url -Headers #{Authorization = "Basic $token"} -Method Get -ContentType application/json
$buildid = $response.latestBuild.id
$success = $false
do{
try{
$Buildurl2 = "https://{instance}/{collection}/{project}/_apis/build/builds/$($buildid)?api-version=5.0"
$Buildinfo2 = Invoke-RestMethod -Method Get -ContentType application/json -Uri $Buildurl2 -Headers #{Authorization=("Basic {0}" -f $token)}
$BuildStatus= $Buildinfo2.status
$result = $Buildinfo2.result
echo $result
echo $BuildStatus
if($BuildStatus -eq "completed") {
write-output "No Running Pipeline, starting Next Pipeline"
$success = $true
} else {
Write-output "Pipeline Build In Progress, Waiting for it to finish!"
Write-output "Next attempt in 30 seconds"
Start-sleep -Seconds 30
}
}
catch{
Write-output "catch - Next attempt in 30 seconds"
write-output "1"
Start-sleep -Seconds 30
# Put the start-sleep in the catch statemtnt so we
# don't sleep if the condition is true and waste time
}
$count++
}until($count -eq 2000 -or $success -eq $true )
if ($result -ne "succeeded")
{
echo "##vso[task.logissue type=error]Something went very wrong."
}
if(-not($success)){exit}
Explanation:
This powershell script runs the following two Rest APIs:
Definitions - Get
Builds - Get
The script checks the status of the pipeline(by polling) that is in process. If the pipeline is completed and the result is successful, it will run the other three pipelines. Or it will wait for the pipeline finishing the build.
Result Sample:

Passing a script from runspacepool

I am trying to work on an existing script that I had some assistance from in another thread. With some member assistance I was able to get my script to run using the "ThreadJob" module, however I was hoping I can also make use of runspacepools in conjunction with the ThreadJob to make it run faster.
In my code I post below, I am printing out a line to notify me that the function get's called. And I can see it is getting called. So it makes me think that line 56 and line 59 are incorrectly being called and I can't figure out how to call them.
if I run the "$rootPath\UpdateContacts\UpdateContacts.ps1" file manually through powershell ISE, it runs fine (obviously outside of the runspace), but I'd like to try and get it to work within the runspacepool.
Here is what I'm working with. I think I am pretty close.
begin
{
CLS
[switch]$MultiThread=$true
$rootPath = $(Split-path $MyInvocation.MyCommand.path -Parent)
$userEmail = "user#domain.com"
$SessionState = [system.management.automation.runspaces.initialsessionstate]::CreateDefault()
$RunspacePool = [RunspaceFactory]::CreateRunspacePool(1, 10,$Sessionstate, $Host)
$RunspacePool.Open()
$Jobs = #()
}
process
{
function ContactUpdater()
{
##################### Start Comparing Data #####################
#Our Array of values we will be comparing
[array]$CompareValues = "FirstName","MiddleName","LastName","DisplayName","Email","Mobile","TelephoneNumber","Title","Dept","Company"
for($i=0; $i -lt $CompareValues.Count; $i++)
{
#First let's create 2 variables that will hold the info we want
$A = ($Users).($CompareValues[$i])
$B = ($Contacts).($CompareValues[$i])
##################### Update Contacts #####################
#Only Run if there are contacts; otherwise there is nothing for us to compare
if(($NULL -ne $B))
{
#Displays all differences
#$Differences = [string[]]([Linq.Enumerable]::Except([object[]]$a, [object[]]$b) + [Linq.Enumerable]::Except([object[]]$b, [object[]]$a))
#Displays what accounts we need to import
$NeedsToBeAdded = [string[]]([Linq.Enumerable]::Except([object[]]$a, [object[]]$b))
#Displays what accounts we need to delete because they no longer exist
$NeedsToBeDeleted = [string[]]([Linq.Enumerable]::Except([object[]]$b, [object[]]$a))
}
}
##################### Import All Contacts #####################
if($NULL -eq $Contacts)
{
Write-Host "I am in the import"
# Load UpdateContacts function in memory
. "$rootPath\UpdateContacts\UpdateContacts\UpdateContacts.ps1"
#Write-host "Importing Contacts. This could take several minutes."
& "$rootPath\UpdateContacts\UpdateContacts.ps1"
}
}
if($MultiThread)
{
foreach($userEmail in $EmailAddress)
{
try
{
##################### Create Contact Folder #####################
if($NULL -eq $folderId)
{
$start = [datetime]::UtcNow
Write-Host "Creating Contacts Folder"
Try
{
while($NULL = (Invoke-RestMethod -Uri "https://graph.microsoft.com/v1.0/users/$UPN/contactFolders/$folderId" -Headers $headers -Method get))
{
$NewContactFolder = Invoke-RestMethod -Uri "https://graph.microsoft.com/v1.0/users/$UPN/contactFolders" -Body $ContactsFolderBody -Headers $headers -Method post -ContentType 'application/json'
sleep -Milliseconds 1
$folderId = $($NewContactFolder.id)
}
}
Catch
{
Out-Null
}
Add-Content "$rootPath\progress.txt" "`t`tCreated Contacts Folder in: $('{0:N2}' -f ([datetime]::UtcNow - $start).TotalSeconds) seconds"
Add-Content "$rootPath\progress.txt" ""
}
##################### Getting All User Info #####################
$start = [datetime]::UtcNow
$Users = & $rootPath\GetUserInfo\GetUserInfo.ps1
Add-Content "$rootPath\progress.txt" "`t`tFinished Getting all User Info in: $('{0:N2}' -f ([datetime]::UtcNow - $start).TotalSeconds) seconds"
Add-Content "$rootPath\progress.txt" ""
##################### Getting Contact Info #####################
if($NULL -ne $folderId)
{
$start = [datetime]::UtcNow
$Contacts = & $rootPath\GetContactInfo\GetContactInfo.ps1
Add-Content "$rootPath\progress.txt" "`t`tFinished Getting all Contact Info in: $('{0:N2}' -f ([datetime]::UtcNow - $start).TotalSeconds) seconds"
Add-Content "$rootPath\progress.txt" ""
}
##################### Import Contacts #####################
$start = [datetime]::UtcNow
CLS
if($NULL -eq $ImportMsg)
{
Write-host "Importing Contacts. This could take several minutes."
$ImportMsg = "Ran"
}
$ContactImporter = ContactUpdater
Add-Content "$rootPath\progress.txt" "`t`tFinished Importing Contact Info in: $('{0:N2}' -f ([datetime]::UtcNow - $start).TotalSeconds) seconds"
}
catch
{
$LogFile = "$rootPath\log.txt"
$errcond = $_.Exception.Message
$timestamp = (get-date).DateTime
"Time of exception: $timestamp" | Out-File $LogFile -Append
"User: $userEmail" | out-file $LogFile -Append
$errcond | out-file -FilePath $LogFile -append
}
1..10 | Foreach-Object {
$PowershellThread = [powershell]::Create()
$PowershellThread.RunspacePool = $RunspacePool
$PowershellThread.AddScript($ContactImporter).AddArgument($userEmail)
$Jobs += $PowershellThread.BeginInvoke()
}
}
}
}
end
{
if($MultiThread)
{
while ($Jobs.IsCompleted -contains $false)
{
Start-Sleep -Milliseconds 100
}
$RunspacePool.Close() | Out-Null
$RunspacePool.Dispose() | Out-Null
}
}
The part in the "Import all Contacts" section within the ContactUpdater() Function, should call the script:
& "$rootPath\UpdateContacts\UpdateContacts.ps1"
That script looks like this:
# Save the function in a scriptBlock, we need this
# so we can pass this function in the scope of the ThreadJobs
$updateContacts = "function UpdateContacts { $function:updateContacts }"
# Define the Number of Threads we are going to use
# (Get-CimInstance win32_processor).NumberOfLogicalProcessors
# Can give you a good perspective as to how many Threads is safe to use.
$numberOfThreads = 10
# $users is the array we want to process with
# the UpdateContacts function.
# Grouping the users in chunks so each running Job can process
# a chunk of users. Each chunk will contain around 50 users to process.
$groupSize = [math]::Ceiling($users.Count / $numberOfThreads)
$counter = [pscustomobject]#{ Value = 0 }
$chunks = $users | Group-Object -Property {
[math]::Floor($counter.Value++ / $groupSize)
}
foreach($chunk in $chunks)
{
# Capture this chunk of users in a variable
$thisGroup = $chunk.Group
# This is what we are running inside the scope
# of our threadJob
$scriptBlock = {
# Pass our variables to this scope
$UPN = $using:UPN
$folderID = $using:folderId
$headers = $using:headers
$contactsBody = $using:contactsBody
$ImportMsg = $using:ImportMsg
# First we need to define the function inside this scope
. ([scriptBlock]::Create($using:updateContacts))
# Loop through each user
foreach($user in $using:thisGroup)
{
UpdateContacts -User $user
}
}
# ThrottleLimit is the number of Jobs that can run at the same time.
# Be aware, a higher number of Jobs running does NOT mean that the
# task will perform faster. This always depends on your CPU & Memory.
# And, this case in particular, the number of requests your URI is able to handle
Start-ThreadJob -ScriptBlock $scriptBlock -ThrottleLimit $numberOfThreads
}
# Now we should have 10 Jobs running at the same time, each Job
# is processing a chunk of 50 users aprox. (500 users / 10)
# the output of all Jobs:
$result = Get-Job | Receive-Job -Wait
# Free up memory:
Get-Job | Remove-Job
That code above, starts a threadjob and launches another function in "$rootPath\UpdateContacts\UpdateContacts\UpdateContacts.ps1"
And that script looks like this:
Function UpdateContacts($User)
{
#FirstName, MiddleName, LastName, DisplayName, SamAccountName, Email, Mobile, TelephoneNumber, Title, Dept, Company, Photo, ExtensionAttribute2
$ContactsBody = #"
{
"givenName" : "$($User.FirstName)",
"middleName" : "$($User.MiddleName)",
"surname" : "$($User.LastName)",
"fileAs" : "$($User.LastName)",
"displayName" : "$($User.DisplayName)",
"jobTitle" : "$($User.Title)",
"companyName" : "$($User.Company)",
"department" : "$($User.Dept)",
"mobilePhone" : "$($User.Mobile)",
"homePhones" : ["$($User.TelephoneNumber)"],
"emailAddresses":
[
{
"address": "$($User.Email)",
"name": "$($User.DisplayName)"
}
]
}
"#
Try
{
Invoke-RestMethod -Uri "https://graph.microsoft.com/v1.0/users/$UPN/contactFolders/$folderId/contacts" -Headers $headers -Body $ContactsBody -Method Post -ContentType 'application/json' | Out-Null
#After each user clear the info
$User = $NULL
}
Catch
{
if($error)
{
$User
$error
pause
}
$_.Exception.Message
Write-Host "--------------------------------------------------------------------------------------"
$_.Exception.ItemName
}
}

Contacts import using runspacepools

I'm trying to import contacts using RunspacePools, but I'm having trouble getting it to work. If I take it out of the runspace logic, it works fine, just takes a long time. I'd really like to use runspacepools to speed up the import process and make it run multithreaded so it imports faster. On avg each import takes about 5-6 mins per user, and I have about 500 users, so it can take up to 3000 mins to run.
Here is what I currently have:
#---------------------------------------------
. $rootPath\UpdateContacts\UpdateContacts.ps1
# Set up runspace pool
$RunspacePool = [runspacefactory]::CreateRunspacePool(1,10)
$RunspacePool.Open()
# Assign new jobs/runspaces to a variable
$Runspaces = foreach ($User in $Users)
{
# Create new PowerShell instance to hold the code to execute, add arguments
$PSInstance = [powershell]::Create().AddScript({
$Users | ForEach{ UpdateContacts($_) }
}).AddParameter('$_')
# Assing PowerShell instance to RunspacePool
$PSInstance.RunspacePool = $RunspacePool
# Start executing asynchronously, keep instance + IAsyncResult objects
New-Object psobject -Property #{
Instance = $PSInstance
IAResult = $PSInstance.BeginInvoke()
Argument = $User
}
}
# Wait for the the runspace jobs to complete
while($Runspaces |Where-Object{-not $_.IAResult.IsCompleted})
{
Start-Sleep -Milliseconds 500
}
# Collect the results
$Results = $Runspaces |ForEach-Object {
$Output = $_.Instance.EndInvoke($_.IAResult)
New-Object psobject -Property #{
User = $User
}
}
And my "UpdateContacts.ps1" file looks like this:
Function UpdateContacts($User)
{
Write-host "Importing Contacts. This could take several minutes."
#FirstName, MiddleName, LastName, DisplayName, SamAccountName, Email, Mobile, TelephoneNumber, Title, Dept, Company, Photo, ExtensionAttribute2
$ContactsBody = #"
{
"givenName" : "$($User.FirstName)",
"middleName" : "$($User.MiddleName)",
"surname" : "$($User.LastName)",
"displayName" : "$($User.DisplayName)",
"jobTitle" : "$($User.Title)",
"companyName" : "$($User.Company)",
"department" : "$($User.Dept)",
"mobilePhone" : "$($User.Mobile)",
"homePhones" : ["$($User.TelephoneNumber)"],
"emailAddresses":
[
{
"address": "$($User.Email)",
"name": "$($User.DisplayName)"
}
]
}
"#
Try
{
Invoke-RestMethod -Uri "https://graph.microsoft.com/v1.0/users/$UPN/contactFolders/$folderId/contacts" -Headers $headers -Body $ContactsBody -Method Post -ContentType 'application/json' | Out-Null
#After each user clear the info
$User = $NULL
}
Catch
{
if($error)
{
$User
$error
pause
}
$_.Exception.Message
Write-Host "--------------------------------------------------------------------------------------"
$_.Exception.ItemName
}
}
Any help is appreciated.
EDIT:
Here is the full script (with the exception of the ContactUploader.ps1 script. That function is in a separate script but the whole code (Function) is posted above).
CLS
##################### Import Thread/Job Module to perform multithreading #####################
if(!(Get-Module -ListAvailable -Name ThreadJob))
{
$NULL = Install-Module -Name ThreadJob -Scope CurrentUser -Force -Confirm:$False
}
##################### ------------------------------------------------------------- #####################
#Root Path
$rootPath = $(Split-path $MyInvocation.MyCommand.path -Parent)
#Prevent connection from closing on us when we use "Invoke-RestMethod"
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls -bor [Net.SecurityProtocolType]::Tls11 -bor [Net.SecurityProtocolType]::Tls12
Add-Content "$rootPath\progress.txt" ""
Add-Content "$rootPath\progress.txt" "********** Starting Script $(Get-Date -Format "HH:mm ss")**********"
Add-Content "$rootPath\progress.txt" ""
##################### Connect to Microsoft Graph API and configure all of our Variables #####################
Add-Content "$rootPath\progress.txt" "********** Connecting to Graph API $(Get-Date -Format "HH:mm ss")**********"
Add-Content "$rootPath\progress.txt" ""
$ApplicationID = "ApplicationID"
$TenatDomainName = "domain.com"
$AccessSecret = "ItsASecret"
$global:Body = #{
Grant_Type = "client_credentials"
Scope = "https://graph.microsoft.com/.default"
client_Id = $ApplicationID
Client_Secret = $AccessSecret
}
$ConnectGraph = Invoke-RestMethod -Uri "https://login.microsoftonline.com/$TenatDomainName/oauth2/v2.0/token" -Method POST -Body $Body
Add-Content "$rootPath\progress.txt" "********** Finished Connecting to Graph API $(Get-Date -Format "HH:mm ss")**********"
Add-Content "$rootPath\progress.txt" ""
$global:token = $ConnectGraph.access_token
$global:UPN = "user#domain.com"
$global:AccessToken = $token
$global:User = $NULL
$global:Contact = $NULL
$global:NeedsToBeAdded = $NULL
$global:NeedsToBeDeleted = $NULL
$global:folderId = $NULL
$global:NewContactFolder = $NULL
$global:FolderName = "Test Contacts"
$global:headers = #{
"Authorization" = "Bearer $AccessToken"
"Accept" = "application/json;odata.metadata=none"
"Content-Type" = "application/json; charset=utf-8"
"ConsistencyLevel" = "eventual"
}
#Create Contact Folder if it doesn't exist
$global:ContactsFolderBody = #"
{
"parentFolderId": "$ParentFolderID",
"displayName": "Test Contacts"
}
"#
Add-Content "$rootPath\progress.txt" "********** Grabbing Contact Folder Info $(Get-Date -Format "HH:mm ss")**********"
Add-Content "$rootPath\progress.txt" ""
$global:folders = Invoke-RestMethod -Uri "https://graph.microsoft.com/v1.0/users/$UPN/contactFolders" -Headers $headers
$global:ParentFolderID = $folders[0].value.parentFolderId
#Get Folder ID we are working with
foreach($folder in $folders.value)
{
#Reset the Value
$folderId = $NULL
if($FolderName -eq $folder.displayName)
{
$folderId = $folder.id
break
}
}
Add-Content "$rootPath\progress.txt" "********** Finished Grabbing Contact Folder Info $(Get-Date -Format "HH:mm ss")**********"
Add-Content "$rootPath\progress.txt" ""
##################### Check if our Contacts Folder exists. If it doesn't, create it. #####################
if($NULL -eq $folderId)
{
Add-Content "$rootPath\progress.txt" "********** Creating Contact Folder $(Get-Date -Format "HH:mm ss")**********"
Add-Content "$rootPath\progress.txt" ""
$Start = Get-Date
Write-Host "Creating Contacts Folder"
Try
{
while($NULL = (Invoke-RestMethod -Uri "https://graph.microsoft.com/v1.0/users/$UPN/contactFolders/$folderId" -Headers $headers -Method get))
{
$NewContactFolder = Invoke-RestMethod -Uri "https://graph.microsoft.com/v1.0/users/$UPN/contactFolders" -Body $ContactsFolderBody -Headers $headers -Method post -ContentType 'application/json'
sleep -Milliseconds 1
$folderId = $($NewContactFolder.id)
}
}
Catch
{
Out-Null
}
$End = Get-Date
Write-Host "Contacts Folder created in $($Start - $End) seconds"
Write-Host ""
Add-Content "$rootPath\progress.txt" "********** Finished Creating Contact Folder $(Get-Date -Format "HH:mm ss")**********"
Add-Content "$rootPath\progress.txt" ""
}
##################### Grab all of our User Information from AD #####################
Add-Content "$rootPath\progress.txt" "********** Grabbing AD User Info $(Get-Date -Format "HH:mm ss")**********"
Add-Content "$rootPath\progress.txt" ""
$Start = Get-Date
$searcher=[adsisearcher]""
$searcher.Sort.PropertyName = "sn"
$searcher.Filter = "(&(objectcategory=person)(objectclass=user)(extensionAttribute2=custom)(|(mobile=*)(telephonenumber=*)))"
$colProplist = #(
'givenname', 'extensionattribute2'
'initials', 'mobile', 'telephonenumber'
'sn', 'displayname', 'company'
'title', 'mail', 'department'
'thumbnailphoto', 'samaccountname'
)
$colPropList | & { process {
$NULL = $searcher.PropertiesToLoad.Add($_)
}}
$End = Get-Date
Write-Host "User info took $($Start - $End) seconds"
Write-Host ""
Add-Content "$rootPath\progress.txt" "********** Finished Grabbing AD User Info $(Get-Date -Format "HH:mm ss")**********"
Add-Content "$rootPath\progress.txt" ""
##################### Create our User Hashtable #####################
Add-Content "$rootPath\progress.txt" "********** Creating User Hashtable $(Get-Date -Format "HH:mm ss")**********"
Add-Content "$rootPath\progress.txt" ""
Write-Host "Creating User Hashtable"
$Start = Get-Date
$users = $searcher.FindAll() | & { process {
[pscustomobject]#{
FirstName = [string]$_.properties.givenname
MiddleName = [string]$_.properties.initials
LastName = [string]$_.properties.sn
DisplayName = [string]$_.properties.displayname
SamAccountName = [string]$_.properties.samaccountname
Email = [string]$_.properties.mail
Mobile = [string]$_.properties.mobile
TelephoneNumber = [string]$_.properties.telephonenumber
Title = [string]$_.properties.title
Dept = [string]$_.properties.department
Company = [string]$_.properties.company
Photo = [string]$_.properties.thumbnailphoto
ExtensionAttribute2 = [string]$_.properties.extensionattribute2
}
}}
Write-Host "User Hashtable took $($Start - $End) seconds"
Write-Host ""
Add-Content "$rootPath\progress.txt" "********** Finished Creating User Hashtable $(Get-Date -Format "HH:mm ss")**********"
Add-Content "$rootPath\progress.txt" ""
##################### Get Existing Contacts (Only if the Contacts Folder wasn't newly created )#####################
if($NULL -ne $folderId)
{
Add-Content "$rootPath\progress.txt" "********** Grabbing Contact Info $(Get-Date -Format "HH:mm ss")**********"
Add-Content "$rootPath\progress.txt" ""
$Start = Get-Date
$AllContacts = Invoke-RestMethod -Uri "https://graph.microsoft.com/v1.0/users/$UPN/contactFolders/$folderId/contacts?`$top=999&`$Orderby=Surname" -Headers $headers -Method Get
$End = Get-Date
Write-Host "Contact info took $($Start - $End) seconds"
Write-Host ""
Add-Content "$rootPath\progress.txt" "********** Finished Grabbing Contact Info $(Get-Date -Format "HH:mm ss")**********"
Add-Content "$rootPath\progress.txt" ""
##################### Create our Contact Hashtable #####################
Add-Content "$rootPath\progress.txt" "********** Creating Contact Hashtable $(Get-Date -Format "HH:mm ss")**********"
Add-Content "$rootPath\progress.txt" ""
Write-Host "Creating Contact Hashtable"
$Start = Get-Date
$Contacts = $AllContacts.value | & { process {
[PSCustomObject]#{
'FirstName' = [string]$_.givenName
'MiddleName' = [string]$_.initials
'LastName' = [string]$_.surname
'DisplayName' = [string]$_.displayName
'Email' = [string](($_.emailAddresses) | %{$_.Address})
'Mobile' = [string]$_.mobilePhone
'TelephoneNumber' = [string]$_.homePhones
'Title' = [string]$_.jobTitle
'Dept' = [string]$_.department
'Company' = [string]$_.companyName
}
}}
$End = Get-Date
Write-Host "Contact HashTable took $($Start - $End) seconds"
Write-Host ""
Add-Content "$rootPath\progress.txt" "********** Finished Creating Contact Hashtable $(Get-Date -Format "HH:mm ss")**********"
Add-Content "$rootPath\progress.txt" ""
}
##################### Start Comparing Data #####################
#Our Array of values we will be comparing
[array]$CompareValues = "FirstName","MiddleName","LastName","DisplayName","Email","Mobile","TelephoneNumber","Title","Dept","Company"
for($i=0; $i -lt $CompareValues.Count; $i++)
{
#First let's create 2 variables that will hold the info we want
$A = ($Users).($CompareValues[$i])
$B = ($Contacts).($CompareValues[$i])
##################### Update Contacts #####################
#Only Run if there are contacts; otherwise there is nothing for us to compare
if(($NULL -ne $B))
{
#Displays all differences
#$Differences = [string[]]([Linq.Enumerable]::Except([object[]]$a, [object[]]$b) + [Linq.Enumerable]::Except([object[]]$b, [object[]]$a))
#Displays what accounts we need to import
$NeedsToBeAdded = [string[]]([Linq.Enumerable]::Except([object[]]$a, [object[]]$b))
#Displays what accounts we need to delete because they no longer exist
$NeedsToBeDeleted = [string[]]([Linq.Enumerable]::Except([object[]]$b, [object[]]$a))
}
##################### Import All Contacts #####################
Else
{
Add-Content "$rootPath\progress.txt" "********** Importing All Contacts $(Get-Date -Format "HH:mm ss")**********"
Add-Content "$rootPath\progress.txt" ""
$Start = Get-Date
<#
#---------------------------------------------
. $rootPath\UpdateContacts\UpdateContacts.ps1
# Set up runspace pool
$RunspacePool = [runspacefactory]::CreateRunspacePool(1,10)
$RunspacePool.Open()
# Assign new jobs/runspaces to a variable
$Runspaces = foreach ($User in $Users)
{
# Create new PowerShell instance to hold the code to execute, add arguments
$PSInstance = [powershell]::Create().AddScript({
$Users | ForEach{ UpdateContact($_) }
}).AddParameter('$_')
# Assing PowerShell instance to RunspacePool
$PSInstance.RunspacePool = $RunspacePool
# Start executing asynchronously, keep instance + IAsyncResult objects
New-Object psobject -Property #{
Instance = $PSInstance
IAResult = $PSInstance.BeginInvoke()
Argument = $User
}
}
# Wait for the the runspace jobs to complete
while($Runspaces |Where-Object{-not $_.IAResult.IsCompleted})
{
Start-Sleep -Milliseconds 500
}
# Collect the results
$Results = $Runspaces |ForEach-Object {
$Output = $_.Instance.EndInvoke($_.IAResult)
New-Object psobject -Property #{
User = $User
}
}
#---------------------------------------------
#>
Write-host "Importing Contacts. This could take several minutes."
#There are no contacts, so let's import them
#Path to our script that imports Contacts
. $rootPath\UpdateContacts\UpdateContacts.ps1
#$Users | & { process { UpdateContacts($_) } }
#Start-ThreadJob -ScriptBlock { $Users | & { process { UpdateContacts($_) } } }
Start-ThreadJob -ScriptBlock { $Users | ForEach{ UpdateContacts($_) } }
Get-Job
$End = Get-Date
Write-Host "Contact Import took $($Start - $End) seconds"
Write-Host ""
Add-Content "$rootPath\progress.txt" "********** Finished Importing All Contacts $(Get-Date -Format "HH:mm ss")**********"
Add-Content "$rootPath\progress.txt" ""
break
}
}
Add-Content "$rootPath\progress.txt" "********** Finished Script $(Get-Date -Format "HH:mm ss")**********"
Add-Content "$rootPath\progress.txt" ""
There is a bunch of code to go through so I'm gonna give you a blueprint of how you can achieve processing all users in $users using ThreadJob.
So, step by step, I'll try to add as much comments as I consider appropriate to guide you through the thought process.
I'm not sure what is the output of your function since I see an | Out-Null at the end of the Invoke-RestMethod. You would need to clarify on this.
# requires -Modules ThreadJob
# Load UpdateContacts function in memory
. "$rootPath\UpdateContacts\UpdateContacts.ps1"
# Save the function in a scriptBlock, we need this
# so we can pass this function in the scope of the ThreadJobs
$updateContacts = "function UpdateContacts { $function:updateContacts }"
# Define the Number of Threads we are going to use
# (Get-CimInstance win32_processor).NumberOfLogicalProcessors
# Can give you a good perspective as to how many Threads is safe to use.
$numberOfThreads = 10
# I'm assuming that $users is the array we want to process with
# the UpdateContacts function. Around 500 as you said in your question.
# Here I'm grouping the users in chunks so each running Job can process
# a chunk of users. Each chunk will contain around 50 users to process.
$groupSize = [math]::Ceiling($users.Count / $numberOfThreads)
$counter = [pscustomobject]#{ Value = 0 }
$chunks = $users | Group-Object -Property {
[math]::Floor($counter.Value++ / $groupSize)
}
# Here is the magic
foreach($chunk in $chunks)
{
# Capture this chunk of users in a variable
$thisGroup = $chunk.Group
# This is what we are running inside the scope
# of our threadJob
$scriptBlock = {
# As in my comments, these variables don't exist inside here,
# you need to pass them to these scope
$UPN = $using:UPN
$folderID = $using:folderId
$headers = $using:headers
$contactsBody = $using:contactsBody
# First we need to define the function inside
# this scope
. ([scriptBlock]::Create($using:updateContacts))
# Loop through each user
foreach($user in $using:thisGroup)
{
UpdateContacts -User $user
}
} # EOF Job's ScriptBlock
# ThrottleLimit is the number of Jobs that can run at the same time.
# Be aware, a higher number of Jobs running does NOT mean that the
# task will perform faster. This always depends on your CPU & Memory.
# And, this case in particular, the number of requests your URI is able to handle
Start-ThreadJob -ScriptBlock $scriptBlock -ThrottleLimit $numberOfThreads
}
# Now we should have 10 Jobs running at the same time, each Job
# is processing a chunk of 50 users aprox. (500 users / 10)
# Note: As in my previous comments, I see an Out-Null in your function
# not sure what is meant to return but in case, this is how you capture
# the output of all Jobs:
$result = Get-Job | Receive-Job -Wait
# Free up memory:
Get-Job | Remove-Job

Looping through Azure DevOps Work List Result values with Powershell prints nothing to console

Using the following docs, https://learn.microsoft.com/en-us/rest/api/azure/devops/wit/Work%2520Items/List?view=azure-devops-rest-5.0&viewFallbackFrom=vsts-rest-5.0, I have an api call that gets a list of workitems.
I convert that to Json in my devops build yml via a powershell script but looping through the "value" array doesn't seem to print anything to my devops console. Any help on syntax?
$jsonResp = $($workItemResponse | ConvertTo-Json -Depth 100)
Write-Host "json formatted response"
Write-Host $jsonResp.value
foreach($item in $($jsonResp.value))
{
$releaseNotesHtml += "<div>" + $item.fields.System.Id + "</div>"
Write-Host $item.fields.System.Id
Write-Host $item.fields.System.Title
Write-Host $item.fields.System.State
Write-Host $item.fields.System.Description
}
I've also tried this, with no success.
foreach($item in $jsonResp.value)
{
$releaseNotesHtml += "<div>" + $item.fields.System.Id + "</div>"
Write-Host $item.fields.System.Id
Write-Host $item.fields.System.Title
Write-Host $item.fields.System.State
Write-Host $item.fields.System.Description
}
Use the following format:
$token = "<pat>"
$orgurl = "https://dev.azure.com/<org_name>/_apis/wit/workitems?ids=200,250,300&api-version=5.0"
$token = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($token)"))
$wiResponse = Invoke-RestMethod -Uri $orgurl -Headers #{Authorization = "Basic $token"} -Method Get -ContentType application/json
foreach($item in $wiResponse.value)
{
$releaseNotesHtml += "<div>" + $item.id + "</div>"
Write-Host $item.id
Write-Host $item.fields.'System.Title'
Write-Host $item.fields.'System.State'
Write-Host $item.fields.'System.Description'
}
The result:

Azure Repos API Commits - GetChanges returning empty list

I have an Azure build pipeline with the following powershell task running on the agent to get the commits of my PR:
$id = $(System.PullRequest.PullRequestId)
$uri = "$(System.TeamFoundationCollectionUri)NPI/_apis/git/repositories/$(Build.Repository.ID)/pullRequests/$id/commits?api-version=5.1"
Write-Host "Getting all commits for PR" $id ":" $uri
$J = Invoke-WebRequest -URI $uri -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
} | ConvertFrom-Json
And then loop through the reply like this:
foreach ($c in $J.value)
{
$id = $c.commitId
$uri = "$(System.TeamFoundationCollectionUri)NPI/_apis/git/repositories/$(Build.Repository.ID)/commits/$id/changes?api-version=5.1"
Write-Host "Getting details for commit" $id ":" $uri
$J = Invoke-WebRequest -URI $uri -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
} | ConvertFrom-Json
Write-Host $J
}
The log file of the Azure task contains this:
Getting all commits for PR 1026 : https://dev.azure.com/xxx/xxx/_apis/git/repositories/xxx/pullRequests/1026/commits?api-version=5.1
Getting details for commit xxx : https://dev.azure.com/xxx/xxx/_apis/git/repositories/xxx/commits/xxx/changes?api-version=5.1
#{changeCounts=; changes=System.Object[]}
As you can see, there are no changes listed. However, if I click on the logged url for getting the commit details, I get a non-empty list. What's going on here?
Oops, turns out it does actually work but the Write-Host output is broken.
This runs fine:
foreach ($cc in $J.changes)
{
Write-Host $cc.item.path
}
You need a variable to store it before convert to json. This is work correctly to me.
"$($env:SYSTEM_TEAMFOUNDATIONCOLLECTIONURI)$env:SYSTEM_TEAMPROJECTID/_apis/git/repositories/$($pipeline.repository.id)/commits/$(Build.BuildId)/changes?api-version=5.0"
Write-Host $commitUrl
$rsCommit = Invoke-RestMethod -Uri $commitUrl -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"}
Write-Host "CommitInfo = $($rsCommit | ConvertTo-Json -Depth 100)"

Resources