Azure Devops : Replace variable in list with unknown order - azure

We are running a build pipeline in azure DevOps. After the build we deploy the stuff and substitute variables in JSON files.
I have a problem substituting a variable in a list of equal objects. My son looks like this:
"Friends": [
{ "Name": "John",
"Phone": "12345678"
},
{ "Name": "Frank",
"Phone": "12235578"
},
{ "Name": "Bill",
"Phone": "13790678"
}
]
I can substitute Franks phone using the path: Friends.1.Phone
What should I do, if the order of "Friends" is not determined and I can not use the index?
I searched for the azure plugin "variable substitution condition" but I couldn't find any useful task.

You can use a powershell task and use Convertfrom-Json command
$Json = #'
{
"Friends": [
{ "Name": "John",
"Phone": "12345678"
},
{ "Name": "Frank",
"Phone": "12235578"
},
{ "Name": "Bill",
"Phone": "13790678"
}
]
}
'#
$result = $Json | ConvertFrom-Json
$result.Friends[0].Phone='12345'
$result | ConvertTo-Json -Depth 4
This will change 'John' phone number.
Answer 2:
$result = $Json | ConvertFrom-Json
$result.Friends | % {if($_.name -eq 'john'){$_.phone=98765}}
$result | ConvertTo-Json -depth 10

Related

How to rename and edit an item in a single change request - Azure devops pushes api

Using the Devops pushes endpoint like _apis/git/repositories/<Project>/pushes?api-version=6.0 we can rename or edit a file.
This is working no problem. However, I want to rename and edit a file in a single commit. I've tried passing two changes in a single request, like:
{
"changes": [
{
"changeType": "rename",
"item": {
"path": "/path/new-filename.txt"
},
"sourceServerItem": "/path/old-filename.txt"
},
{
"changeType": "edit",
"item": {
"path": "/path/new-filename.txt"
},
"newContent": {
"content": "...new content...",
"contentType": "rawtext"
}
}
]
}
This gave the error "Multiple operations were attempted on file 'path/new-filename.txt' within the same request. Only a single operation may be applied to each file within the same commit. Parameter name: newPush"
So I tried combining them with the change type of 'all'
{
"changeType": "all",
"item": {
"path": "/path/new-filename.txt",
},
"sourceServerItem": "/path/old-filename.txt",
"newContent": {
"content": "...new content...",
"contentType": "rawText"
}
}
Still no joy: "The parameters supplied are not valid. Parameter name: newPush"
Is this possible, or do I have to separate the changes in two commits?
Edit:
Can't even do this with multiple commits in one request 😭. I mean what's the point of having commits as an array when you must have exactly one commit anyway?
The parameters are incorrect. A posted push must contain exactly one commit and one refUpdate.
Parameter name: newPush
Test with the Rest API and I can reproduce the same situation.
The issue comes from the limitations of the Rest API itself. When we use the Rest API to do the push action, it limits that you can only make commit once and changes cannot be made to the same file at the same time.
You need to separate the changes in two commits.
To meet your requirement, you can use PowerShell Script to run the Rest API twice to commit the changes.
For example:
$token = "PAT"
$url="https://dev.azure.com/{Org}/{Project}/_apis/git/repositories/{Repo}/pushes?api-version=6.0"
$token = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($token)"))
$JSON = #'
{
"refUpdates": [
{
"name": "refs/heads/master",
"oldObjectId": "commitid"
}
],
"commits": [
{
"comment": "Renaming tasks.md to xxx",
"changes": [
{
"changeType": "rename",
"sourceServerItem": "/path/old-filename.txt",
"item": {
"path": "/path/new-filename.txt"
}
}
]
}
]
}
'#
$response = Invoke-RestMethod -Uri $url -Headers #{Authorization = "Basic $token"} -Method Post -Body $JSON -ContentType application/json
$newobjectid = $response.commits.commitid
echo $newobjectid
$JSON1 = "
{
`"refUpdates`": [
{
`"name`": `"refs/heads/master`",
`"oldObjectId`": `"$newobjectid`"
}
],
`"commits`": [
{
`"comment`": `"Renaming tasks.md to xxx`",
`"changes`": [
{
`"changeType`": `"edit`",
`"item`": {
`"path`": `"/path/new-filename.txt`"
},
`"newContent`": {
`"content`": `"...new content...`",
`"contentType`": `"rawtext`"
}
}
]
}
]
}
"
$response1 = Invoke-RestMethod -Uri $url -Headers #{Authorization = "Basic $token"} -Method Post -Body $JSON1 -ContentType application/json

jq update substring matched object

Json file:
{
"A": "jack",
"B": [
{
"name": "jack/jil",
"version": "0.1"
},
{
"name": "went/well",
"version": "1.2"
}
]
}
now I need to update every objects version: "$version-dev" where name starts from jack while retaining rest of the json intact.
the closest I can get.
jq '.B[] | select(.name|test("jack.")) | .version += "-dev"' old.json > new.json
in the above command I'm only getting the that particular object with updated value in the new.json but I need the whole json too.
any suggestions
You need to put parantheses around the whole selection to be updated: (… | .version) += …
jq '(.B[] | select(.name|test("jack.")) | .version) += "-dev"' old.json > new.json
{
"A": "jack",
"B": [
{
"name": "jack/jil",
"version": "0.1-dev"
},
{
"name": "went/well",
"version": "1.2"
}
]
}
Demo

jq parsing and linux formatting to desired output

I am trying to format json output and exclude an element when a condition is met.
1) In this case I'd like to exclude any element that contains "valueFrom" using jq
[{
"name": "var1",
"value": "var1value"
},
{
"name": "var2",
"value": "var2value"
},
{
"name": "var3",
"value": "var3value"
},
{
"name": "var4",
"value": "var4value"
},
{ # <<< exclude this element as valueFrom exists
"name": "var5",
"valueFrom": {
"secretKeyRef": {
"key": "var5",
"name": "var5value"
}
}
}
]
After excluding the element mentioned above I am trying to return a result set that looks like this.
var1: var1value
var2: var2value
var3: var3value
var4: var4value
Any feedback is appreciated. Thanks.
Select array items that doesn't have the valueFrom key using a combination of select/1, has/1, and not/0. Then format the objects as you please.
$ jq -r '.[] | select(has("valueFrom") | not) | "\(.name): \(.value)"' input.json

Convert name and Value to column and rows during Export to CSV in Powershell

I am trying to convert name and values returned from my API call into column and rows during export but so far I have no luck.
$searchResponse = Invoke-RestMethod -Method POST -Uri $searchUri -ContentType application/json -Header $requestHeader -Body $searchResultsRequest
$lists = $searchResponse.businessO.fields | Select name, value
The $list returns:
name value
---- ----
Name YY
Address ABC street
City Seattle
Zip 01256
Name XYZ
Address XYZ street
City XYZ
Zip 45456
Name Mike
Address 1256 Street
City New York
Zip 78965
I want to output this result as following in CSV:
Name Address City Zip
YY ABC street Seattle 01256
.
.
.
I tried looping through list and applied condition to check name and populate value based on it, but i end up getting either duplicates or all my output data are out of sysnc meaning Mike gets Address of YY and so on....
If($lists.count -ge 0){
ForEach($list in $lists){
if($list.name -eq "Name") {$name= $list.value }
}
I would really appreciate any help on this one. Thank You.
$searchResponse results
$searchResponse |ConvertTo-Json
{
"businessObjects": [
{
"busObId": "xxxxxxxxxxxxxx",
"busObPublicId": "abc345",
"busObRecId": "xxxxxxxxxxxxxxxxxxx",
"fields": " ",
"links": "",
"errorCode": null,
"errorMessage": null,
"hasError": false
},
{
"busObId": "xxxxxxxxxxxxx",
"busObPublicId": "rty567",
"busObRecId": "xxxxxxxxxxxxxxxx",
"fields": " ",
"links": "",
"errorCode": null,
"errorMessage": null,
"hasError": false
}
],
"hasPrompts": false,
"links": [
],
"prompts": [
],
"searchResultsFields": [
],
"totalRows": 2500
}
Fields has the name and value which I want to output.
$searchResponse.businessObjects.fields |ConvertTo-Json
[
{
"dirty": false,
"displayName": "Name",
"fieldId": "xxxxxxxxxxx",
"html": null,
"name": "Name",
"value": "Mike"
},
{
"dirty": false,
"displayName": "Address",
"fieldId": "456451212113132",
"html": null,
"name": "Address",
"value": "Seattle"
},
and so on.
This answer as a teaching exercise with the expectation, one will walk through the code in the VSCode or ISE debugger and learn more about the objects, members, and more.
There are a few ways to solve this. In this answer, I try to break down the steps in a way newer PowerShell user may learn more about the language.
# mock your data
$list1 = #(
[pscustomobject]#{Name='YY'},
[pscustomobject]#{Address='ABC street'},
[pscustomobject]#{City='Seattle'},
[pscustomobject]#{Zip='01256'},
[pscustomobject]#{Name='XYZ'},
[pscustomobject]#{Address='XYZ street'},
[pscustomobject]#{City='XYZ'},
[pscustomobject]#{Zip='45456'},
[pscustomobject]#{Name='Mike'},
[pscustomobject]#{Address='1256 Street'},
[pscustomobject]#{City='New York'},
[pscustomobject]#{Zip='78965'}
)
# mock your data
$list2 = #(
#{Name='YY'},
#{Address='ABC street'},
#{City='Seattle'},
#{Zip='01256'},
#{Name='XYZ'},
#{Address='XYZ street'},
#{City='XYZ'},
#{Zip='45456'},
#{Name='Mike'},
#{Address='1256 Street'},
#{City='New York'},
#{Zip='78965'}
)
# debuggng...
#$list1
#$list1.GetType()
#$list1[0].GetType()
#$list2
#$list2.GetType()
#$list2[0].GetType()
# seems your data looks like list1
$data = #()
$entry = [ordered]#{}
# transform the data into separate objects based on the field you want
foreach ($item in $list)
{
if ($item.Name) {$entry.Name = $item.Name}
elseif ($item.Address) {$entry.Address = $item.Address}
elseif ($item.City) {$entry.City = $item.City}
elseif ($item.Zip) {$entry.Zip = $item.Zip; $data += $entry; $entry = [ordered]#{}}
}
# data transformed now into an array of hashtables
# there are a few says to save as csv, one is to just roll your own rather
# than GetEnumerator(), etc.
$out = #()
$out += $data[0].Keys -join ', '
$out += $data | % {
#$_['Name']+','+$_['Address']+','+$_['City']+','+$_['Zip']
# or
#$_.Name+','+$_.Address+','+$_.City+','+$_.Zip
# or
$_.Name,$_.Address,$_.City,$_.Zip -join ', '
}
$out
# save $out to a file, uncomment to use after editing path
#$out | Out-File -FilePath 'c:\mycsvfile.csv'

how to find the string and it value using grep or awk [duplicate]

This question already has answers here:
Extract a specific field from JSON output using jq
(3 answers)
Closed 4 years ago.
I'm new to shell scripting and started baby steps in it.
I've written shell script recently to call rest API and I was able to execute it without any issues.
I've stored the output in a variable like below
{
"id": 3184136,
"name": "XXX TEST API",
"url": "http://xxxxxxxxxxx/_apis/test/Runs/3184136",
"isAutomated": true,
"owner": {
"displayName": "XXXX",
"url": "http://xxxxxxxxxxx/_apis/Identities/dbf722a9-73b0-46d6-a2bd-9835c1f0c221",
"_links": {
"avatar": {
"href": "http://xxxxxxxxxxx/_api/_common/identityImage?id=dbf722a9-73b0-46d6-a2bd-9835c1f0c221"
}
},
"id": "dbf722a9-73b0-46d6-a2bd-9835c1f0c221",
"uniqueName": "xxxxxxxxxxx\\ServiceLaunchpadDev",
"imageUrl": "http://xxxxxxxxxxx/_api/_common/identityImage?id=dbf722a9-73b0-46d6-a2bd-9835c1f0c221"
},
"project": {
"id": "6d5e21e7-c75e-464a-9708-90fbff086902",
"name": "eDellPrograms"
},
"startedDate": "2018-10-11T06:36:50.627Z",
"completedDate": "2018-10-11T07:04:45.153Z",
"state": "Completed",
"plan": {
"id": "5299555",
"name": "Smoke Dashboard Peso - DIT",
"url": "http://xxxxxxxxxxx/_apis/test/Plans/5299555"
},
"postProcessState": "Complete",
"totalTests": 5,
"incompleteTests": 0,
"notApplicableTests": 0,
"passedTests": 0,
"unanalyzedTests": 5,
"createdDate": "2018-10-11T06:36:50.533Z",
"lastUpdatedDate": "2018-10-11T07:04:45.153Z",
"lastUpdatedBy": {
"displayName": "xxxxxxxxxxx",
"url": "http://xxxxxxxxxxx/_apis/Identities/8de2a654-063b-48bd-8101-87e4ec2f05e3",
"_links": {
"avatar": {
"href": "http://xxxxxxxxxxx/_api/_common/identityImage?id=8de2a654-063b-48bd-8101-87e4ec2f05e3"
}
},
"id": "8de2a654-063b-48bd-8101-87e4ec2f05e3",
"uniqueName": "xxxxxxxxxxx\\xxxxxxxxxxx",
"imageUrl": "http://xxxxxxxxxxx/_api/_common/identityImage?id=8de2a654-063b-48bd-8101-87e4ec2f05e3"
},
"controller": "xxxxxxxxxxx",
"revision": 5,
"comment": "Build Definition : xxxxxxxxxxx \nBuild Version : xxxxxxxxxxx_20180925.1\nConfiguration : DIT\nBatch type : Suite\nTest type : Parallel\nTest Controller Name : xxxxxxxxxxx\nPreferred Agents : ADPTAW10A618|ADPTAW10A619|ADPTAW10A621 \nRequested by : xxxxxxxxxxx\nEmail Request : Y\nEmail To : xxxxxxxxxxx\nEmailCc : xxxxxxxxxxx\nEnvironment : DIT\nTest Setting : DIT\nContinue On Failure : false\nDNS Setting : false",
"dropLocation": "\\\\xxxxxxxxxxx\\DropFolder\\xxxxxxxxxxx_20180925.1",
"runStatistics": [
{
"state": "Completed",
"outcome": "Failed",
"count": 5
}
],
"webAccessUrl": "http://xxxxxxxxxxx/_TestManagement/Runs#runId=3184136&_a=runCharts"
}
from the above output, I'm trying to find the "state" and its value. But I couldn't make that happen. kindly anyone helps me.
echo $result | grep -o 'state*'
with above command, I was able to print state. but I'm expecting both state and its value.
Appreciate your help. Thanks in Advance.
I tried storing your json in a file called n2.json.
cat n2.json
{
"id":3184136,
"name":"XXX TEST API",
"url":"http://xxxxxxxxxxx/_apis/test/Runs/3184136",
"isAutomated":true,
"owner":{
"displayName":"XXXX",
"url":"http://xxxxxxxxxxx/_apis/Identities/dbf722a9-73b0-46d6-a2bd-9835c1f0c221",
"_links":{
"avatar":{
"href":"http://xxxxxxxxxxx/_api/_common/identityImage?id=dbf722a9-73b0-46d6-a2bd-9835c1f0c221"
}
},
"id":"dbf722a9-73b0-46d6-a2bd-9835c1f0c221",
"uniqueName":"xxxxxxxxxxx\\ServiceLaunchpadDev",
"imageUrl":"http://xxxxxxxxxxx/_api/_common/identityImage?id=dbf722a9-73b0-46d6-a2bd-9835c1f0c221"
},
"project":{
"id":"6d5e21e7-c75e-464a-9708-90fbff086902",
"name":"eDellPrograms"
},
"startedDate":"2018-10-11T06:36:50.627Z",
"completedDate":"2018-10-11T07:04:45.153Z",
"state":"Completed",
"plan":{
"id":"5299555",
"name":"Smoke Dashboard Peso - DIT",
"url":"http://xxxxxxxxxxx/_apis/test/Plans/5299555"
},
"postProcessState":"Complete",
"totalTests":5,
"incompleteTests":0,
"notApplicableTests":0,
"passedTests":0,
"unanalyzedTests":5,
"createdDate":"2018-10-11T06:36:50.533Z",
"lastUpdatedDate":"2018-10-11T07:04:45.153Z",
"lastUpdatedBy":{
"displayName":"xxxxxxxxxxx",
"url":"http://xxxxxxxxxxx/_apis/Identities/8de2a654-063b-48bd-8101-87e4ec2f05e3",
"_links":{
"avatar":{
"href":"http://xxxxxxxxxxx/_api/_common/identityImage?id=8de2a654-063b-48bd-8101-87e4ec2f05e3"
}
},
"id":"8de2a654-063b-48bd-8101-87e4ec2f05e3",
"uniqueName":"xxxxxxxxxxx\\xxxxxxxxxxx",
"imageUrl":"http://xxxxxxxxxxx/_api/_common/identityImage?id=8de2a654-063b-48bd-8101-87e4ec2f05e3"
},
"controller":"xxxxxxxxxxx",
"revision":5,
"comment":"Build Definition : xxxxxxxxxxx \nBuild Version : xxxxxxxxxxx_20180925.1\nConfiguration : DIT\nBatch type : Suite\nTest type : Parallel\nTest Controller Name : xxxxxxxxxxx\nPreferred Agents : ADPTAW10A618|ADPTAW10A619|ADPTAW10A621 \nRequested by : xxxxxxxxxxx\nEmail Request : Y\nEmail To : xxxxxxxxxxx\nEmailCc : xxxxxxxxxxx\nEnvironment : DIT\nTest Setting : DIT\nContinue On Failure : false\nDNS Setting : false",
"dropLocation":"\\\\xxxxxxxxxxx\\DropFolder\\xxxxxxxxxxx_20180925.1",
"runStatistics":[
{
"state":"Completed",
"outcome":"Failed",
"count":5
}
],
"webAccessUrl":"http://xxxxxxxxxxx/_TestManagement/Runs#runId=3184136&_a=runCharts"
}
Then use jq on top of this:
jq -r '.state' n2.json
Completed
You are looking for 'state', 'statee', 'stateee', 'stateeee', etc.
The wildcard applies to the preceeding character.
Try this:
echo $result | grep -o '"state":[^,]*'
It looks for everything up to, but excluding, the next comma.

Resources