Azure table storage REST Query doesn't show new entries - azure

I'm building a query for a Azure Storage Account(tables in this case). I try to receive the last log entries of the last day.
The filter looks like this in the storage query builder:
{
"filter": [
{
"id": "{ID}",
"andOr": "And",
"field": "TIMESTAMP",
"type": "DateTime",
"operator": ">=",
"value": "datetime'2019-11-18T10:08:41.296Z'",
"timeValue": "Last 24 hours",
"timestampType": "time",
"customTimestamp": {
"lastNumber": 7,
"lastTimeUnit": "Days"
},
"isLocal": true
}
],
"select": null,
"group": {
"id": "{ID}",
"children": [
"{CHILD}"
]
},
"top": null,
"filterType": "Query Builder",
"fileVersion": 0.2
}
This query returnes all the entries from the last day.
I tried to use this filter in the Azure REST API call. in a powershell function like this:
function GetTableEntityAll($TableName) {
$version = "2017-04-17"
$resource = "$tableName()"
$DST = [DateTime]::Today.AddDays(-1)
$table_url = "https://$storageAccount.table.core.windows.net/$resource"+ '?$filter=Timestamp gt datetime' + "'"+ $DST.ToString("s") + "'"
write-host $table_url
$GMTTime = (Get-Date).ToUniversalTime().toString('R')
$stringToSign = "$GMTTime`n/$storageAccount/$resource"
$hmacsha = New-Object System.Security.Cryptography.HMACSHA256
$hmacsha.key = [Convert]::FromBase64String($accesskey)
$signature = $hmacsha.ComputeHash([Text.Encoding]::UTF8.GetBytes($stringToSign))
$signature = [Convert]::ToBase64String($signature)
$headers = #{
'x-ms-date' = $GMTTime
Authorization = "SharedKeyLite " + $storageAccount + ":" + $signature
"x-ms-version" = $version
Accept = "application/json;odata=fullmetadata"
}
$item = Invoke-RestMethod -Method GET -Uri $table_url -Headers $headers -ContentType application/json
return $item.value
}
The URL I generate with this is:
https://{Accountname}.table.core.windows.net/{Tablename}()?$filter=Timestamp gt datetime'2019-11-18T00:00:00'
This doesn't return any records. I also tried to encode the url but that doesns't work either. If I change the datetime to 2016 I receive the first few records.
Azure documentation: https://learn.microsoft.com/en-us/rest/api/storageservices/querying-tables-and-entities
Any ideas?

Related

How to rename and edit an item in a single change request - Azure devops pushes api

Using the Devops pushes endpoint like _apis/git/repositories/<Project>/pushes?api-version=6.0 we can rename or edit a file.
This is working no problem. However, I want to rename and edit a file in a single commit. I've tried passing two changes in a single request, like:
{
"changes": [
{
"changeType": "rename",
"item": {
"path": "/path/new-filename.txt"
},
"sourceServerItem": "/path/old-filename.txt"
},
{
"changeType": "edit",
"item": {
"path": "/path/new-filename.txt"
},
"newContent": {
"content": "...new content...",
"contentType": "rawtext"
}
}
]
}
This gave the error "Multiple operations were attempted on file 'path/new-filename.txt' within the same request. Only a single operation may be applied to each file within the same commit. Parameter name: newPush"
So I tried combining them with the change type of 'all'
{
"changeType": "all",
"item": {
"path": "/path/new-filename.txt",
},
"sourceServerItem": "/path/old-filename.txt",
"newContent": {
"content": "...new content...",
"contentType": "rawText"
}
}
Still no joy: "The parameters supplied are not valid. Parameter name: newPush"
Is this possible, or do I have to separate the changes in two commits?
Edit:
Can't even do this with multiple commits in one request 😭. I mean what's the point of having commits as an array when you must have exactly one commit anyway?
The parameters are incorrect. A posted push must contain exactly one commit and one refUpdate.
Parameter name: newPush
Test with the Rest API and I can reproduce the same situation.
The issue comes from the limitations of the Rest API itself. When we use the Rest API to do the push action, it limits that you can only make commit once and changes cannot be made to the same file at the same time.
You need to separate the changes in two commits.
To meet your requirement, you can use PowerShell Script to run the Rest API twice to commit the changes.
For example:
$token = "PAT"
$url="https://dev.azure.com/{Org}/{Project}/_apis/git/repositories/{Repo}/pushes?api-version=6.0"
$token = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($token)"))
$JSON = #'
{
"refUpdates": [
{
"name": "refs/heads/master",
"oldObjectId": "commitid"
}
],
"commits": [
{
"comment": "Renaming tasks.md to xxx",
"changes": [
{
"changeType": "rename",
"sourceServerItem": "/path/old-filename.txt",
"item": {
"path": "/path/new-filename.txt"
}
}
]
}
]
}
'#
$response = Invoke-RestMethod -Uri $url -Headers #{Authorization = "Basic $token"} -Method Post -Body $JSON -ContentType application/json
$newobjectid = $response.commits.commitid
echo $newobjectid
$JSON1 = "
{
`"refUpdates`": [
{
`"name`": `"refs/heads/master`",
`"oldObjectId`": `"$newobjectid`"
}
],
`"commits`": [
{
`"comment`": `"Renaming tasks.md to xxx`",
`"changes`": [
{
`"changeType`": `"edit`",
`"item`": {
`"path`": `"/path/new-filename.txt`"
},
`"newContent`": {
`"content`": `"...new content...`",
`"contentType`": `"rawtext`"
}
}
]
}
]
}
"
$response1 = Invoke-RestMethod -Uri $url -Headers #{Authorization = "Basic $token"} -Method Post -Body $JSON1 -ContentType application/json

Azure Graph API - Approve PIM requests

I'm trying to approve/deny requests for role activations using the new Azure AD Privileged Identity Management REST API.
I am already able to read all pending role activation requests using the following request:
GET https://graph.microsoft.com/beta/roleManagement/directory/roleAssignmentScheduleRequests
The response looks like this:
{
"#odata.context": "https://graph.microsoft.com/beta/$metadata#roleManagement/directory/roleAssignmentScheduleRequests",
"value": [
{
"id": "40b1dff9-9703-4da8-bf8f-275141347b6e",
"status": "PendingApproval",
"createdDateTime": "2021-06-04T10:47:40.34Z",
"completedDateTime": "2021-06-04T10:47:40.15Z",
"approvalId": "40b1dff9-9703-4da8-bf8f-275141347b6e",
"customData": null,
"action": "SelfActivate",
"principalId": "049bad91-8812-4daa-870e-1edf05f5ced1",
"roleDefinitionId": "9b895d92-2cd3-44c7-9d02-a6ac2d5ea5c3",
"directoryScopeId": "/",
"appScopeId": null,
"isValidationOnly": false,
"targetScheduleId": "40b1dff9-9703-4da8-bf8f-275141347b6e",
"justification": "My custom reason",
"createdBy": {
"application": null,
"device": null,
"user": {
"displayName": null,
"id": "049bad91-8812-4daa-870e-1edf05f5ced1"
}
},
"scheduleInfo": {
"startDateTime": null,
"recurrence": null,
"expiration": {
"type": "afterDuration",
"endDateTime": null,
"duration": "PT8H"
}
},
"ticketInfo": {
"ticketNumber": "",
"ticketSystem": ""
}
}
]
}
Which matches the request I can see in the Azure Portal:
Now I am trying to approve the above request using the PATCH operation by providing the returned id:
PATCH https://graph.microsoft.com/beta/roleManagement/directory/roleAssignmentScheduleRequests/40b1dff9-9703-4da8-bf8f-275141347b6e
For the payload, I tried adding Provisioned and Denied:
{
"status": "Denied"
}
But for whatever reason, I keep getting the following Error (Code 404):
{
"error": {
"code": "UnknownError",
"message": "{\"message\":\"No HTTP resource was found that matches the request URI 'https://api.azrbac.mspim.azure.com/api/v3/roleManagement/directory/roleAssignmentScheduleRequests('40b1dff9-9703-4da8-bf8f-275141347b6e')?'.\"}",
"innerError": {
"date": "2021-06-04T11:06:18",
"request-id": "ec668ea0-cf33-4e41-bfb4-19ca4ac683ad",
"client-request-id": "ca765884-79b1-7695-5c72-c5783dd9968c"
}
}
}
Any ideas?
Finally, I was able to find a solution. Here is a complete sample using the PowerShell Graph SDK:
$scopes = #(
"PrivilegedAccess.Read.AzureAD",
"RoleAssignmentSchedule.ReadWrite.Directory",
"PrivilegedAccess.ReadWrite.AzureAD"
)
Connect-MgGraph -Scopes $scopes
[array]$pendingApprovals = Invoke-GraphRequest `
-Method GET `
-Uri '/beta/roleManagement/directory/roleAssignmentScheduleRequests?$filter=(status eq ''PendingApproval'')' |
Select-Object -ExpandProperty value
$approvalSteps = Invoke-GraphRequest `
-Method GET `
-Uri ('/beta/roleManagement/directory/roleAssignmentApprovals/{0}' -f $pendingApprovals[0].approvalId) |
Select-Object -ExpandProperty steps | Where-Object status -eq InProgress
$body = #{
reviewResult = 'Approve'
justification = 'Seems legit'
}
Invoke-GraphRequest `
-Method PATCH `
-Uri ('https://graph.microsoft.com/beta/roleManagement/directory/roleAssignmentApprovals/{0}/steps/{1}' -f $pendingApprovals[0].approvalId, $approvalSteps.id) `
-Body $body
I also wrote a blog article about it:
Approve requests for Azure AD roles in PIM
You are on the right track using roleAssignmentScheduleRequests to get the pending request. Check out the response and in the value array there is an ID. Use that ID in the Update privilegedapproval call. (https://learn.microsoft.com/en-us/graph/api/privilegedapproval-update?view=graph-rest-beta&tabs=http)
PATCH /privilegedApproval/{id}
Example using the IDs you shared:
PATCH https://graph.microsoft.com/beta/privilegedApproval/40b1dff9-9703-4da8-bf8f-275141347b6e
Content-type: application/json
Content-length: 180
{
"approvalState": "approved",
"approverReason": "Martin Brandl approves you!"
}
In the request body for approval state your possible values are: pending, approved, denied, aborted, canceled.

Clone Azure Test Plan to new area path using powershell

I have a powershell script that clones my test plans - I am using Azure 2019 so the function is not built in. My script works, but only clones plans to the same area path. I would like to be able to clone to a new area path. For example, in the image below, under Test Plans, I want to clone 'TestPlan1.7' in MyTests\Version1.7 to 'NewTestPlan' in MyTests\Version1.8. The code I use currently creates the new test plan as it clones, but either that or manually creating a new one is an option. Also, I cannot download/ install any addons or mods to azure, so those aren't a solution for me. Thanks for any help!
Here is my current code snippet to clone. I have the variables listed in my script, but don't think they're needed for this.
EDIT: I tried your answer Sunny (and also just realized I can edit my own question to add stuff) Here is my entire code, along with the error. The variables have been changed for obvious reasons, but the structure is the same. The line number the error gave is the one that starts with 'Invoke-WebRequest'
$organization = "myOrg"
$PAT = "MyAzurePAT"
$myBaseUrl = "myAzureUrl"
$project = "myTests"
$planId = "123456"
$suiteId = "123457"
$cloneTo = "NewtestPlan"
$authorization = [Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes(":$PAT"))
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Accept-Charset", 'UTF-8')
$headers.Add('Content-Type','Application/Json')
$headers.Add('Authorization',"Basic $authorization")
$postParams=#"
{
"destinationTestPlan": {
"name": "$cloneTo",
"Project": {
"Name": "$project"
}
},
"options": {
"copyAncestorHierarchy": true,
"copyAllSuites": true,
"overrideParameters": {
"System.AreaPath": "myTests\Version1.8",
"System.IterationPath": "NewtestPlan"
}
},
"suiteIds": [
$suiteId
]
}
"#
$apiUrl = "$myBaseUrl/$organization/$project/_apis/test/Plans/$planId/cloneoperation?api-version=5.0-preview.2"
Invoke-WebRequest -Uri $apiUrl -Method POST -Headers $headers -Body $postParams | Select-Object -Expand StatusDescription
<#Error
Invoke-WebRequest : {"$id":"1","innerException":null,"message":"Value cannot be
null.\r\nParameter name: cloneRequestBody","typeName":"System.ArgumentNullException,
mscorlib","typeKey":"ArgumentNullException","errorCode":0,"eventId":0}
At [locationRemoved]\testPlanCloneTool.ps1:39 char:1
+ Invoke-WebRequest -Uri $apiUrl -Method POST -Headers $headers -Body $ ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand
#>
have you tried the "overrideParameters" under "options"?
$postParams=#"
{
"destinationTestPlan": {
"name": "$cloneTo",
"Project": {
"Name": "$project"
}
},
"options": {
"copyAncestorHierarchy": true,
"copyAllSuites": true,
"overrideParameters": {
"System.AreaPath": "MyTests\Version1.8",
"System.IterationPath": "NewTestPlan"
}
},
"suiteIds": [
$sourceSuiteId,
]
}
"#
MS Docs Ref: https://learn.microsoft.com/en-us/rest/api/azure/devops/test/clone%20operation/clone%20test%20plan?view=azure-devops-rest-5.0#cloneoptions

Interacting with AzureDevops using python

I am trying to achieve following things while interacting with AzureDevops using python:
To get the stories under Boards section
To get the branches under Repos section
I am able to establish connection to AzureDevops URL using requests API, but not sure how to read the data using it.
Is there any way by which I can do the same or if there is some other API using which I can do this?
Thanks in advance!!
Agree with Krzysztof Madej,
We could use write a WIQL query to list all User Story, you could refer this link for more details.
from vsts.vss_connection import VssConnection
from msrest.authentication import BasicAuthentication
import json
from vsts.work_item_tracking.v4_1.models.wiql import Wiql
def emit(msg, *args):
print(msg % args)
def print_work_item(work_item):
emit(
"{0} {1}: {2}".format(
work_item.fields["System.WorkItemType"],
work_item.id,
work_item.fields["System.Title"],
)
)
personal_access_token = 'YourPATToken'
organization_url = 'https://dev.azure.com/YourorgName'
# Create a connection to the org
credentials = BasicAuthentication('', personal_access_token)
connection = VssConnection(base_url=organization_url, creds=credentials)
wiql = Wiql(
query="""select [System.Id] From WorkItems """
)
wit_client = connection.get_client('vsts.work_item_tracking.v4_1.work_item_tracking_client.WorkItemTrackingClient')
wiql_results = wit_client.query_by_wiql(wiql).work_items
if wiql_results:
# WIQL query gives a WorkItemReference with ID only
# => we get the corresponding WorkItem from id
work_items = (
wit_client.get_work_item(int(res.id)) for res in wiql_results
)
for work_item in work_items:
print_work_item(work_item)
List all branches, we could use Refs - List and add variable filter to list all branches, If we do not add it, it will also list pull request.
Sample:
List all repo and get repo ID.
GET https://dev.azure.com/{Org name}/{Project name}/_apis/git/repositories?api-version=4.1
List repo branches via repo ID
GET https://dev.azure.com/{Org name}/{Project name}/_apis/git/repositories/{Repo ID}/refs?filter=heads&api-version=4.1
Result:
Update1
Power shell script
#List all branches name
$connectionToken="PAT"
$base64AuthInfo= [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($connectionToken)"))
$BranchDetailURL = "https://dev.azure.com/{Org name}/{Project name}/_apis/git/repositories/{Repo ID}/refs?filter=heads&?api-version=6.0"
$BranchInfo = Invoke-RestMethod -Uri $BranchDetailURL -Headers #{authorization = "Basic $base64AuthInfo"} -Method Get
Write-Host $BranchInfo.value.name
#List all User Story ID
$WorkItemWiqlQuery = "https://dev.azure.com/v-viliu/test_Agile/test/_apis/wit/wiql?api-version=5.1"
$query = "Select [System.Id], [System.Title], [System.State] From WorkItems Where [System.WorkItemType] = 'User Story'"
$body = #{query=$query} | ConvertTo-Json
$WorkItemDetailInfo = Invoke-RestMethod -Uri $WorkItemWiqlQuery -Method Post -ContentType "application/json" -Headers #{Authorization=("Basic $base64AuthInfo")} -Body $body
Write-Host $WorkItemDetailInfo.workItems.id
Result:
To get stories please check this answer but keep in mind to set proper filter [System.WorkItemType] = 'Task'".
To get branches you need to use Refs - List endpoint
GET https://dev.azure.com/{organization}/{project}/_apis/git/repositories/{repositoryId}/refs?api-version=6.0
Sample response
{
"value": [
{
"name": "refs/heads/feature/calcApp",
"objectId": "ffe9cba521f00d7f60e322845072238635edb451",
"creator": {
"displayName": "Normal Paulk",
"url": "https://vssps.dev.azure.com/fabrikam/_apis/Identities/ac5aaba6-a66a-4e1d-b508-b060ec624fa9",
"_links": {
"avatar": {
"href": "https://dev.azure.com/fabrikam/_apis/GraphProfile/MemberAvatars/aad.YmFjMGYyZDctNDA3ZC03OGRhLTlhMjUtNmJhZjUwMWFjY2U5"
}
},
"id": "ac5aaba6-a66a-4e1d-b508-b060ec624fa9",
"uniqueName": "dev#mailserver.com",
"imageUrl": "https://dev.azure.com/fabrikam/_api/_common/identityImage?id=ac5aaba6-a66a-4e1d-b508-b060ec624fa9",
"descriptor": "aad.YmFjMGYyZDctNDA3ZC03OGRhLTlhMjUtNmJhZjUwMWFjY2U5"
},
"url": "https://dev.azure.com/fabrikam/7484f783-66a3-4f27-b7cd-6b08b0b077ed/_apis/git/repositories/d3d1760b-311c-4175-a726-20dfc6a7f885/refs?filter=heads%2Ffeature%2FcalcApp"
},
{
"name": "refs/heads/feature/replacer",
"objectId": "917131a709996c5cfe188c3b57e9a6ad90e8b85c",
"creator": {
"displayName": "Normal Paulk",
"url": "https://vssps.dev.azure.com/fabrikam/_apis/Identities/ac5aaba6-a66a-4e1d-b508-b060ec624fa9",
"_links": {
"avatar": {
"href": "https://dev.azure.com/fabrikam/_apis/GraphProfile/MemberAvatars/aad.YmFjMGYyZDctNDA3ZC03OGRhLTlhMjUtNmJhZjUwMWFjY2U5"
}
},
"id": "ac5aaba6-a66a-4e1d-b508-b060ec624fa9",
"uniqueName": "dev#mailserver.com",
"imageUrl": "https://dev.azure.com/fabrikam/_api/_common/identityImage?id=ac5aaba6-a66a-4e1d-b508-b060ec624fa9",
"descriptor": "aad.YmFjMGYyZDctNDA3ZC03OGRhLTlhMjUtNmJhZjUwMWFjY2U5"
},
"url": "https://dev.azure.com/fabrikam/7484f783-66a3-4f27-b7cd-6b08b0b077ed/_apis/git/repositories/d3d1760b-311c-4175-a726-20dfc6a7f885/refs?filter=heads%2Ffeature%2Freplacer"
},

Convert name and Value to column and rows during Export to CSV in Powershell

I am trying to convert name and values returned from my API call into column and rows during export but so far I have no luck.
$searchResponse = Invoke-RestMethod -Method POST -Uri $searchUri -ContentType application/json -Header $requestHeader -Body $searchResultsRequest
$lists = $searchResponse.businessO.fields | Select name, value
The $list returns:
name value
---- ----
Name YY
Address ABC street
City Seattle
Zip 01256
Name XYZ
Address XYZ street
City XYZ
Zip 45456
Name Mike
Address 1256 Street
City New York
Zip 78965
I want to output this result as following in CSV:
Name Address City Zip
YY ABC street Seattle 01256
.
.
.
I tried looping through list and applied condition to check name and populate value based on it, but i end up getting either duplicates or all my output data are out of sysnc meaning Mike gets Address of YY and so on....
If($lists.count -ge 0){
ForEach($list in $lists){
if($list.name -eq "Name") {$name= $list.value }
}
I would really appreciate any help on this one. Thank You.
$searchResponse results
$searchResponse |ConvertTo-Json
{
"businessObjects": [
{
"busObId": "xxxxxxxxxxxxxx",
"busObPublicId": "abc345",
"busObRecId": "xxxxxxxxxxxxxxxxxxx",
"fields": " ",
"links": "",
"errorCode": null,
"errorMessage": null,
"hasError": false
},
{
"busObId": "xxxxxxxxxxxxx",
"busObPublicId": "rty567",
"busObRecId": "xxxxxxxxxxxxxxxx",
"fields": " ",
"links": "",
"errorCode": null,
"errorMessage": null,
"hasError": false
}
],
"hasPrompts": false,
"links": [
],
"prompts": [
],
"searchResultsFields": [
],
"totalRows": 2500
}
Fields has the name and value which I want to output.
$searchResponse.businessObjects.fields |ConvertTo-Json
[
{
"dirty": false,
"displayName": "Name",
"fieldId": "xxxxxxxxxxx",
"html": null,
"name": "Name",
"value": "Mike"
},
{
"dirty": false,
"displayName": "Address",
"fieldId": "456451212113132",
"html": null,
"name": "Address",
"value": "Seattle"
},
and so on.
This answer as a teaching exercise with the expectation, one will walk through the code in the VSCode or ISE debugger and learn more about the objects, members, and more.
There are a few ways to solve this. In this answer, I try to break down the steps in a way newer PowerShell user may learn more about the language.
# mock your data
$list1 = #(
[pscustomobject]#{Name='YY'},
[pscustomobject]#{Address='ABC street'},
[pscustomobject]#{City='Seattle'},
[pscustomobject]#{Zip='01256'},
[pscustomobject]#{Name='XYZ'},
[pscustomobject]#{Address='XYZ street'},
[pscustomobject]#{City='XYZ'},
[pscustomobject]#{Zip='45456'},
[pscustomobject]#{Name='Mike'},
[pscustomobject]#{Address='1256 Street'},
[pscustomobject]#{City='New York'},
[pscustomobject]#{Zip='78965'}
)
# mock your data
$list2 = #(
#{Name='YY'},
#{Address='ABC street'},
#{City='Seattle'},
#{Zip='01256'},
#{Name='XYZ'},
#{Address='XYZ street'},
#{City='XYZ'},
#{Zip='45456'},
#{Name='Mike'},
#{Address='1256 Street'},
#{City='New York'},
#{Zip='78965'}
)
# debuggng...
#$list1
#$list1.GetType()
#$list1[0].GetType()
#$list2
#$list2.GetType()
#$list2[0].GetType()
# seems your data looks like list1
$data = #()
$entry = [ordered]#{}
# transform the data into separate objects based on the field you want
foreach ($item in $list)
{
if ($item.Name) {$entry.Name = $item.Name}
elseif ($item.Address) {$entry.Address = $item.Address}
elseif ($item.City) {$entry.City = $item.City}
elseif ($item.Zip) {$entry.Zip = $item.Zip; $data += $entry; $entry = [ordered]#{}}
}
# data transformed now into an array of hashtables
# there are a few says to save as csv, one is to just roll your own rather
# than GetEnumerator(), etc.
$out = #()
$out += $data[0].Keys -join ', '
$out += $data | % {
#$_['Name']+','+$_['Address']+','+$_['City']+','+$_['Zip']
# or
#$_.Name+','+$_.Address+','+$_.City+','+$_.Zip
# or
$_.Name,$_.Address,$_.City,$_.Zip -join ', '
}
$out
# save $out to a file, uncomment to use after editing path
#$out | Out-File -FilePath 'c:\mycsvfile.csv'

Resources