Cannot POST Cosmos DB Stored Procedures using PowerShell - azure

I'm trying to deploy stored procedures to a collection within an Azure Cosmos DB account as part of my deployment pipeline in Azure DevOps. Due to security reasons, I have to use the REST API (cannot use or import PowerShell modules to do this).
The build agents that I'm using are on-premise agents. Again, security reasons.
In order to generate an authorization token to make requests, I have the following PowerShell function:
Add-Type -AssemblyName System.Web
# Generate Authorization Key for REST calls
Function Generate-MasterKeyAuthorizationSignature
{
[CmdletBinding()]
Param
(
[Parameter(Mandatory=$true)][String]$verb,
[Parameter(Mandatory=$true)][String]$resourceLink,
[Parameter(Mandatory=$true)][String]$resourceType,
[Parameter(Mandatory=$true)][String]$dateTime,
[Parameter(Mandatory=$true)][String]$key,
[Parameter(Mandatory=$true)][String]$keyType,
[Parameter(Mandatory=$true)][String]$tokenVersion
)
$hmacSha256 = New-Object System.Security.Cryptography.HMACSHA256
$hmacSha256.Key = [System.Convert]::FromBase64String($key)
$payLoad = "$($verb.ToLowerInvariant())`n$($resourceType.ToLowerInvariant())`n$resourceLink`n$($dateTime.ToLowerInvariant())`n`n"
$hashPayLoad = $hmacSha256.ComputeHash([System.Text.Encoding]::UTF8.GetBytes($payLoad))
$signature = [System.Convert]::ToBase64String($hashPayLoad);
[System.Web.HttpUtility]::UrlEncode("type=$keyType&ver=$tokenVersion&sig=$signature")
}
I then call this function within the POST function that I'm using to POST the stored procedure to Azure Cosmos DB:
Function Post-StoredProcToCosmosDb
{
[CmdletBinding()]
Param
(
[Parameter(Mandatory=$true)][String]$EndPoint,
[Parameter(Mandatory=$true)][String]$DataBaseId,
[Parameter(Mandatory=$true)][String]$CollectionId,
[Parameter(Mandatory=$true)][String]$MasterKey,
[Parameter(Mandatory=$true)][String]$JSON
)
$Verb = 'Post'
$ResourceType = "sprocs"
$ResourceLink = "dbs/$DataBaseId/colls/$CollectionId"
$dateTime = [DateTime]::UtcNow.ToString("r")
$authHeader = Generate-MasterKeyAuthorizationSignature -verb $Verb -resourceLink $ResourceLink -resourceType $ResourceType -key $MasterKey -keyType "master" -tokenVersion "1.0" -dateTime $dateTime
$header = #{authorization=$authHeader;"x-ms-version"="2017-02-22";"x-ms-date"=$dateTime;"xs-ms-session-token"="28"}
$contentType = "application/json"
$queryUri = "$EndPoint$ResourceLink/sprocs"
$result = Invoke-RestMethod -Headers $header -Method $Verb -ContentType $contentType -Uri $queryUri -Body $JSON
return $result.statuscode
}
I see from the documentation that I need to pass my stored procedure in the body as a string, so I set the path of my stored procedure to a variable like so:
$HelloWorld = Get-Content -Path '.\Databases\MyCosmosDB\MyCosmosCollection\Stored Procedures\HelloWorld.js' | Out-String
I then call my POST function like so:
Post-StoredProcToCosmosDb -EndPoint $env:COSMOSENDPOINT -DataBaseId $env:MYCOSMOSDB -CollectionId $MyCollection -MasterKey $env:MASTERKEY -JSON $HelloWorld
However, when I run the task, I get the following error:
Invoke-RestMethod : The remote server returned an error: (400) Bad Request
At D:_workAzure\r12\a_Cosmos\Deployment\scripts\postStoredProcedures.ps1:61 char:15
$result = Invoke-RestMethod -Headers $header -Method $Verb -Content ...
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand
I have followed the example request body as outlined in the documentation for my stored procedure.
Here is what it looks like for your info:
"function () {\r\n var context = getContext();\r\n var response = context.getResponse();\r\n\r\n response.setBody(\"Hello, World\");\r\n}"
I'm fairly new to PowerShell, so I'm wondering where I am going wrong. I've tried setting the contentType to both application/json and application/query+json but otherwise, I'm not sure where I am going wrong?
If anyone can provide any guidance on this, I'd be most grateful.

So turns out the request body was wrong. It should be like this:
{
"body":"function HelloWorld() {\r\n var context = getContext();\r\n var response = context.getResponse();\r\nresponse.setBody(\"Hello, World\");\r\n}",
"id": "HelloWorld"
}
That's the acceptable request body for Stored Procedures. So what I should have done is set my $HelloWorld variable to:
$HelloWorld = Get-Content -Path '.\Databases\MyCosmosDB\MyCosmosCollection\Stored Procedures\HelloWorld.json' | Out-String
Hope my stupidity helps someone someday :/

Related

Export all API names Azure APIM

I am trying to export all the url endpoints of my APIs which are stored in Azure APIM.
I need them listed in the format [POST]: https://apim-resource-name.azure-api.net/api-name/api-function, or something similar, so I am trying to enumerate all the endpoints first and the VERB that can be used much like what SwaggerUI or one of the OpenAPI docs might show.
I have this code so far but cannot get the output I would like, am I on the right track or is there any easier way to do this even via UI? I have also tried exporting the template and parsing the script, but it seemed overly complicated for this task.
#az login
$token = az account get-access-token | ConvertFrom-Json
$token = $token.accessToken -replace "`n","" -replace "`r",""
$headers = #{Authorization="Bearer $token.accessToken"}
$subscriptionId = 'subscriptionid'
$resourceGroupName = 'resourcegroupname'
$resourceName = 'resourcename'
$apiName = 'apiname'
$url_getapiexport = "https://management.azure.com/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.ApiManagement/service/$resourceName/apis/$apiName`?format=swagger-link&export=true&api-version=2021-08-01" #GET
$url_getapiexport
#Invoke-RestMethod -Uri $url_getapiexport -Method GET -Headers #{Authorization="Bearer $token"} -ContentType "application/json"
$url_getapibytags = "https://management.azure.com/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.ApiManagement/service/$resourceName/apisByTags?&includeNotTaggedApis=true&api-version=2021-08-01" #GET
$url_getapibytags
$api_tags = Invoke-RestMethod -Uri $url_getapibytags -Method GET -Headers #{Authorization="Bearer $token"} -ContentType "application/json"
$api_tags
foreach ($api in $api_tags) {
write-host API: $api.value.api.name
}
So if you want to export all APIM APIs and their service URLs you can do like this (replace xxx with correct values). Example in PHP
<?php
echo "[INFO] API Names and Service Urls \n\n";
$apiList = shell_exec('az apim api list --resource-group "xxx" --service-name "xxx" ');
$apis = json_decode($apiList, true);
$cli = "az rest --method get --url";
foreach ($apis as $api)
{
$name = $api["name"];
$apiRequest = "$cli " . "\"https://management.azure.com/subscriptions/xxx/resourceGroups/xxx/providers/Microsoft.ApiManagement/service/xxx/apis/$name?api-version=2021-08-01\"";
$json = shell_exec($apiRequest);
$apiInfo = json_decode($json, true);
$displayName = $apiInfo["properties"]["displayName"];
$serviceUrl = $apiInfo["properties"]["serviceUrl"];
printf("%s: %s \n", $displayName, $serviceUrl);
}
?>

How to make REST calls to Azure Pipelines from c#

From c# code, I want to call Azure Devops Pipeline Url and get the job/build ids for that url. And then get the result if the job/build succeds/fail.
How to make REST call from c# code my Azure pipelines (I have the pipeline url)?
You can use this sample through HttpClient. Additionally, you can use Microsoft.TeamFoundationServer.Client and BuildHttpClient.GetBuildsAsync: the example is here.
There are 2 ways to do authentication for Azure DevOps API.
Private PAT
Environment Variable: System.AccessToken
For PAT you can review Shamrai Aleksander's code, if you want to call Azure DevOps API in your pipeline, you can conside use System.AccessToken.
Here is Powershell Script for your reference, the field of Authorization is different.
function Http-Request {
param (
[Parameter(Mandatory=$true)]
[string] $Url,
[Parameter(Mandatory=$false)]
[string] $SystemToken
)
if([string]::IsNullOrEmpty($SystemToken)) {
$pat = 'YOUR_PAT'
$username = 'YOUR_USERNAME'
$auth = '{0}:{1}' -f $username, $pat
$bytes = [System.Text.Encoding]::UTF8.GetBytes($auth)
$encoded = [Convert]::ToBase64String($bytes)
$auth = 'Basic ' + $encoded
} else {
$auth = 'Bearer ' + $SystemToken
}
$headers = #{
'Content-Type' = 'application/json'
'Authorization' = $auth
}
Write-Host -ForegroundColor Green 'Http Request:'$Url
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls -bor [Net.SecurityProtocolType]::Tls11 -bor [Net.SecurityProtocolType]::Tls12
$response = Invoke-RestMethod -Uri $Url -Method Get -Headers $headers
Write-Host ($reponse | Format-List | Out-String)
return $response
}

Storage REST API Returns Remote Name Could Not be Resolve Often

I am calling the storage REST API to get container names using
Invoke-WebRequest -Method GET -Uri $storage_url -Headers $headers
This command often returns 'remote name could not be resolved error', even when the storage account exists and is reachable. Just running the command again gives correct result.
Invoke-WebRequest : The remote name could not be resolved: '<storageAccountName>.blob.core.windows.net'
At line:1 char:1
+ Invoke-WebRequest -Method GET -Uri $storage_url -Headers $headers #In ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand
From the information you provided, you use the client credential flow to get the access token, then use the token to call the Storage Rest API - List Containers.
You could use the script below, it works for me.
Make sure the service principal you used has a RBAC role e.g. Contributor/Owner in your storage account -> Access Control, if not, click the Add to add it.
$ClientID = "xxxxxxx"
$ClientSecret = "xxxxxxx"
$tennantid = "xxxxxxx"
$storageaccountname = "joystoragev2"
$TokenEndpoint = {https://login.microsoftonline.com/{0}/oauth2/token} -f $tennantid
$Resource = "https://storage.azure.com/"
$Body = #{
'resource'= $Resource
'client_id' = $ClientID
'grant_type' = 'client_credentials'
'client_secret' = $ClientSecret
}
$params = #{
ContentType = 'application/x-www-form-urlencoded'
Headers = #{'accept'='application/json'}
Body = $Body
Method = 'Post'
URI = $TokenEndpoint
}
$token = Invoke-RestMethod #params
$accesstoken = $token.access_token
$url = {https://{0}.blob.core.windows.net/?comp=list} -f $storageaccountname
$header = #{
'Authorization' = 'Bearer ' + $accesstoken
'x-ms-version' = '2019-02-02'
}
$response = Invoke-WebRequest –Uri $url –Headers $header –Method GET
$response.RawContent

Erorr exporting Azure sql database to blob storage with powershell

I'm trying to use Azure management API to export a SQL database to a bacpac file in a blob storage. The process seems fairly simple: obtain a token, and then:
$sqlAzureBackupHeaders = #{
Authorization = "Bearer $accessToken"
}
$sqlAzureBackupParams = #{
storageKey = "<key-goes-here>",
storageUri = "http://mystorageacct.blob.core.windows.net/my-blob-container/export-name.bacpac",
storageKeyType = "StorageAccessKey",
administratorLogin = "<sql-user>",
administratorLoginPassword = "<sql-password>",
authenticationType = "SQL"
}
$sqlAzureApiUri = "https://management.azure.com/subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.Sql/servers/<server-name>/databases/<database-name>/export?api-version=2014-04-01"
Invoke-RestMethod -Uri $sqlAzureApiUri -Method Post -Headers $sqlAzureBackupHeaders -Body $sqlAzureBackupParams
This results in an error:
Invoke-RestMethod : Receivera:InternalServiceFaultThe server was unable to process the request due to an internal error. For more
information about the error, either turn on IncludeExceptionDetailInFaults (either from ServiceBehaviorAttribute or from the
<serviceDebug> configuration behavior) on the server in order to send the exception information back to the client, or turn on tracing
as per the Microsoft .NET Framework SDK documentation and inspect the server trace logs.
At D:\Users\protec-admin\Desktop\run-backups.ps1:140 char:1
+ Invoke-RestMethod -uri $sqlAzureApiUri -Method Post -Headers $sqlAzur ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand
I tried using Invoke-WebRequest and converting body to json string - with the same result.
When I try the same call using Postman, it works fine, so there's something with making the call from powershell that's not working correctly.
How can I get this working from powershell?
According to my test, the rest API just accept the json body. Please use ConvertTo-Json to convert the body to json.
For example
$headers=#{"Authorization" = "Bearer "+$token}
$body=#{
"storageKeyType"= "StorageAccessKey";
"storageKey"= "<your storage account access key>";
"storageUri"= "https://blobstorage0516.blob.core.windows.net/sample/testbacpac2.bacpac";
"administratorLogin"= "<SQL admin>";
"administratorLoginPassword"= "<SQL admin passsword>";
"authenticationType"= "SQL"
}|ConvertTo-Json
$sqlAzureApiUri = "https://management.azure.com/subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.Sql/servers/<server-name>/databases/<database-name>/export?api-version=2014-04-01"
Invoke-RestMethod -Method Post -Uri $uri -Headers $headers -Body $body -UseBasicParsing -ContentType "application/json"

How to fix "401 Unauthorized" error in PUT Invoke-RestMethod (trying to replace stored procedure)

I'm trying to manage CosmosDB resources using REST API and PowerShell.
My code works fine when I use GET and POST, but when I try to replace an existing object using PUT, I get 401 error.
My code registers stored procedure when it doesn't exist and should update it when the SP already exists.
So when I create a new SP, I use the following variables:
$Verb = "POST"
$ResourceType = "sprocs"
$ResourceLink = "dbs/$DBName/colls/$CollName"
$queryUri = "$CosmosDBEndPoint$ResourceLink/$ResourceType"
get auth header:
...
$authHeader = Generate-MasterKeyAuthorizationSignature -verb $Verb -resourceLink $ResourceLink -resourceType $ResourceType -key $MasterKey -keyType "master" -tokenVersion "1.0" -dateTime $dateTime
and then call REST method:
$header = #{authorization=$authHeader;"x-ms-version"="2017-02-22";"x-ms-date"=$dateTime}
$contentType= "application/json"
Invoke-RestMethod -Method $Verb -ContentType $contentType -Uri $queryUri -Headers $header -Body $body
It works great and creates what it should.
And, when I need to replace existing object (SP in my case), I change variables like this:
$Verb = "PUT"
$ResourceType = "sprocs"
$ResourceLink = "dbs/$DBName/colls/$CollName"
$ItemName = "SP_Name"
$queryUri = "$CosmosDBEndPoint$ResourceLink/$ResourceType/$ItemName"
generate auth header just like in case with POST (only verb is different):
...
$authHeader = Generate-MasterKeyAuthorizationSignature -verb $Verb -resourceLink $ResourceLink -resourceType $ResourceType -key $MasterKey -keyType "master" -tokenVersion "1.0" -dateTime $dateTime
...
and the invoke REST method with new URI (which includes now name of SP to be changed):
$header = #{authorization=$authHeader;"x-ms-version"="2017-02-22";"x-ms-date"=$dateTime}
$contentType= "application/json"
Invoke-RestMethod -Method $Verb -ContentType $contentType -Uri $queryUri -Headers $header -Body $body
Which is throws me 401 Unauthorized, so it seems that auth header isn't right. Can't figure out what I should change there.
From here it looks like the resource or the value in the resource you are trying to change does not support a PUT action. You can confirm manually;
https://learn.microsoft.com/en-us/rest/api/cosmos-db/
or by going to;
https://resources.azure.com/
So the key was to change variables this way:
$ResourceType = "sprocs"
$ItemName = "SP_Name"
$ResourceLink = "dbs/$DBName/colls/$CollName/$ResourceType/$ItemName"
and then generate auth header.
I.e. resource link variable must contain the full path to the replaced object.

Resources