Trying to figure out how to use Azure AppConfiguration REST API (mostly to retrieve and create key-values). So far I found two sources of information: Configuration Stores REST API docs and this GitHub repo Azure App Configuration.
How these two sources are corresponding with each other? They apparently describe some different AppConfig REST API.
I managed to retrieve values from my AppConfig store using this type of URI and AAD authorization
https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.AppConfiguration/configurationStores/{configStoreName}/listKeyValue?api-version=2019-10-01 But it allows to get only one value of one particular key.
The other approach uses URI based on AppConfig endpoint {StoreName}.azconfig.io/kv/... and must have more flexible ways to retrieve data. But I can't make it work. I tried to follow instructions. And I tried to make a request to this URI using AAD token as I did for the first type of API. In both cases I get 401 auth error.
Could anyone share some detailed working examples (Powershell, Postman)? Any help would be appreciated.
https://management.azure.com/ is the Azure Resource Management API, while the azconfig.io one is App Configuration's own API.
I think you should use App Configuration's own API. The same Azure AD token will not work for this API however. You need to request another access token with resource=https://yourstorename.azconfig.io or scope=https://yourstorename.azconfig.io/.default, depending if you use v1 or v2 token endpoint of Azure AD.
Use the $headers in the script to authenticate your api calls:
function Sign-Request(
[string] $hostname,
[string] $method, # GET, PUT, POST, DELETE
[string] $url, # path+query
[string] $body, # request body
[string] $credential, # access key id
[string] $secret # access key value (base64 encoded)
)
{
$verb = $method.ToUpperInvariant()
$utcNow = (Get-Date).ToUniversalTime().ToString("R", [Globalization.DateTimeFormatInfo]::InvariantInfo)
$contentHash = Compute-SHA256Hash $body
$signedHeaders = "x-ms-date;host;x-ms-content-sha256"; # Semicolon separated header names
$stringToSign = $verb + "`n" +
$url + "`n" +
$utcNow + ";" + $hostname + ";" + $contentHash # Semicolon separated signedHeaders values
$signature = Compute-HMACSHA256Hash $secret $stringToSign
# Return request headers
return #{
"x-ms-date" = $utcNow;
"x-ms-content-sha256" = $contentHash;
"Authorization" = "HMAC-SHA256 Credential=" + $credential + "&SignedHeaders=" + $signedHeaders + "&Signature=" + $signature
}
}
function Compute-SHA256Hash(
[string] $content
)
{
$sha256 = [System.Security.Cryptography.SHA256]::Create()
try {
return [Convert]::ToBase64String($sha256.ComputeHash([Text.Encoding]::ASCII.GetBytes($content)))
}
finally {
$sha256.Dispose()
}
}
function Compute-HMACSHA256Hash(
[string] $secret, # base64 encoded
[string] $content
)
{
$hmac = [System.Security.Cryptography.HMACSHA256]::new([Convert]::FromBase64String($secret))
try {
return [Convert]::ToBase64String($hmac.ComputeHash([Text.Encoding]::ASCII.GetBytes($content)))
}
finally {
$hmac.Dispose()
}
}
# Stop if any error occurs
$ErrorActionPreference = "Stop"
$uri = [System.Uri]::new("https://{myconfig}.azconfig.io/kv?api-version=1.0")
$method = "GET"
$body = $null
$credential = "<Credential>"
$secret = "<Secret>"
$headers = Sign-Request $uri.Authority $method $uri.PathAndQuery $body $credential $secret
Sauce: https://github.com/Azure/AppConfiguration/blob/master/docs/REST/authentication/hmac.md#JavaScript
Related
I have an Azure Function which has authentication enabled and set to require an identity provider:
App Service authentication: Enabled
Restrict access: Require authentication
Unauthenticated requests: Return HTTP 401 Unauthorized
Token store: Enabled
Identity provider - App (client) ID
(Name of App Removed): Client ID Removed
Client Secret Setting Name: Microsoft_Provider_Authentication_Secret
When I use Power Automate to POST to the HTTP function it works. This tells me the security is set up and working as expected. When I try and POST to the function directly from PowerShell using my desktop, I get a 401 unauthorized. This is a GCCH environment.
This is the PowerShell code where I get an oath token and am trying to use that to POST to the HTTP function. I tried using the HTTP URL with and without the 'code=' and neither worked.
#These URLs are used to access get the token; scope has not been required is uses the app ID
$loginURL = "https://login.microsoftonline.us"
$resource = "https://graph.microsoft.us"
$Tenant = "mytenant.onmicrosoft.us"
$ClientID = "removed"
$Secret="removed"
$fcnKey = "removed"
$fcnURL = "https://removed?" #Azure function url without the code at the end
$AuthBody = #{
grant_type="client_credentials";
resource=$resource;
client_id=$ClientID;
client_secret=$Secret}
$Oauth = Invoke-RestMethod -Method POST -Uri $loginURL/$Tenant/oauth2/token?api-version=1.0 -Body
$AuthBody -ContentType "application/x-www-form-urlencoded"
$AuthToken = #{
'Authorization'="$($Oauth.token_type) $($Oauth.access_token)";
'Content-Type' = "application/json";
'x-functions-key' = $fcnkey;}
#This returns a 401 unauthorized
Invoke-RestMethod -Headers $AuthToken -Uri $fcnURL -Method POST
#This also returns a 401 unauthorized
$AuthToken = #{
'Authorization'="$($Oauth.token_type) $($Oauth.access_token)";
'Content-Type' = "application/json";}
$FullURL = "https://removed?code=removed"
Invoke-RestMethod -Headers $AuthToken -Uri $fullURL -Method POST
As #jdweng suggested to check the type of authentication and the process of authorization in this MS Doc of Azure AD Authentication Flows.
Check if the below steps help to fix the issue:
Make Sure you entered the "App ID URI" in the "Allowed Token Audiences" Box.
Same App ID URI should be used for acquiring the token.
Refer to the Similar issue-solutions where 401 Unauthorized error is registered on enabling the App service Authentication on Azure Function App such as MSQ&A337577, SO Questions 67957353 and 55226143.
I have two PowerShell Azure functions. The first one collects data and the second one pushes data to an Azure Storage Table. I can see my first function is calling the second function but the second function isn't getting the data. Here is my first function:
$body = #{"partitionKey"="01";"rowKey"="02";"userId"="00001"}
$dataJson = $body | ConvertTo-Json
$functionUri = 'https://functionNumber2'
Invoke-WebRequest -Uri $functionUri -Method POST -Body $dataJson
And my second function looks like this:
using namespace System.Net
# Input bindings are passed in via param block.
param($Request, $TriggerMetadata)
$tableData = $Request.Body.Address
# Write to the Azure Functions log stream.
Write-Host "PowerShell HTTP trigger function processed a request."
# Interact with query parameters or the body of the request.
$name = $Request.Query.Name
if (-not $name) {
$name = $Request.Body.Name
}
Write-Host "Name $name"
$body = "This HTTP triggered function executed successfully. Pass a name in the query string or in the request body for a personalized response."
if ($name) {
$body = "Hello, $name. This HTTP triggered function executed successfully."
}
# Associate values to output bindings by calling 'Push-OutputBinding'.
Push-OutputBinding -Name Response -Value ([HttpResponseContext]#{
StatusCode = [HttpStatusCode]::OK
Body = $body
})
Push-OutputBinding -Name tableName -Value $tableData -Clobber
The error I'm getting is ERROR: Cannot bind argument to parameter 'Value' because it is null. Exception : Type : System.Management.Automation.ParameterBindingValidationException Message : Cannot bind argument to parameter 'Value' because it is null. And if I print out $tableData there is nothing there. I am doing something wrong when passing the data.
Can anyone help me with this?
After reproducing from my end I observed that you are receiving this error because of the below line.
$tableData = $Request.Body.Address
I could able to make this work by using the below line.
$tableData = $Request.Body | ConvertFrom-Json
RESULTS:
I've created a release pipeline which generates release notes, and later publishes it on the Wiki.
Only issue i am having is that the Wiki cannot be updated, but rather a new Wiki pages/sub-page needs to be created manually each time. Due to this, i was wondering if there's anyway i can send my release notes to an email address instead?
my PowerShell
$content = [IO.File]::ReadAllText("$(System.DefaultWorkingDirectory)\releasenotes.md")
$data = #{content=$content;} | ConvertTo-Json;
$connectionToken= '[token]'
$base64AuthInfo= [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($connectionToken)"))
$params = #{uri = '$(WikiPath)';
Method = 'PUT';
Headers = #{Authorization = "Basic $base64AuthInfo" };
ContentType = "application/json";
Body = $data;
}
Invoke-WebRequest #params
Any guide will be appreciated, quite new to all this.
This might depend on whether you're using a on-premises agent or a hosted agent as you will require access to an SMTP server from the agent your job is running. You can then simply use PowerShell to send your email:
Send-MailMessage -From $From -To $To -Subject $Subject -Body $Body -BodyAsHTML -SmtpServer $SmtpServer -Priority High -Encoding UTF8
Wes
I'm trying to use the Azure Cognitive Services API with Powershell to 'read' the contents of a .jpg in a blob store. Everything I am trying to do works perfectly using the Azure API demo/test page, so I am fairly sure that this, to a degree, proves that some elements I'm using in my code are valid. Well, at least they are when using the API testing tool.
Here is my Powershell:
Clear-Host
$myUri = "<ENDPOINT value from "keys and endpoint blade">/vision/v3.0/read/analyze?language=en"
$imagePath = "<path to image in blob. accessible online and anonymously>"
$subKey = "<KEY #1 from "keys and endpoint" blade>"
$headersHash = #{}
$headersHash.Add( "Host", "westeurope.api.cognitive.microsoft.com" )
$headersHash.Add( "Ocp-Apim-Subscription-Key", $subKey )
$headersHash.Add( "Content-Type","application/json" )
$bodyHash = #{ "url" = $imagePath }
out-host -InputObject "Sending request:"
$response = Invoke-WebRequest -uri $myUri `
-Method Post `
-Headers $headersHash `
-Body $bodyHash `
-verbose
"Response: $response"
When I send that, all I ever get is a:
Invoke-WebRequest : The remote server returned an error: (400) Bad Request.
At C:\scratch\testy.ps1:15 char:13
+ $response = Invoke-WebRequest -uri $myUri `
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-W
ebRequest], WebException
+ FullyQualifiedErrorId :
WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand
I must be missing something basic but I cannot see what. There are no published examples of using Powershell to access the CS APIs that I can find but there is an example with Python (requests) and I'm pretty sure I'm emulating and representing what goes into the Python example correctly. But then again, its not working so something isn't right.
Strangely, when I try to recreate this in Postman, I get a 202 but no response body, so its not possible for me to view or extract the apim-request-id in order to manufacture the next request to retrieve the results.
Found the issue. The problem was made clear when I wrapped the call in a try/catch block and put this in the catch block:
$streamReader =
[System.IO.StreamReader]::new($_.Exception.Response.GetResponseStream())
$ErrResp = $streamReader.ReadToEnd() | ConvertFrom-Json
$streamReader.Close()
I was then able to look at the contents of the $ErrResp variable and there was a fragment of a string which said "unable to download target image.." or something to that effect. Odd because I could use the URL I was supplying to instantly connect to and get the image.. so it had to be the way the URL was being injecting into the body.
It was.
When using a hashtable as the body, where your Content-Type is 'application/json' all you need to do, it seems, is use convertto-json with your hash first. This worked and I instantly got my 202 and the pointer to where to collect my results.
Hopefully this will help someone, somewhere, sometime.
I am working on running runbooks (powershell and graphical) from ADF. One of the ways I found to accomplish this task is to use webhooks. I will have runbooks running in parallel and in series (if dependency exists on previous runbook).
Overall,
If a flat file is dropped in Azure Blob storage then it triggers the pipeline that contains respective runbook(s). This part is working.
The webhook of runbook(s) are used in ADF webhook activity. This is where I am facing the problem. I am unsure about what should be in the body of webhook activity?
After some research I was able to find something about Callback uri that needs to be added (or somehow generated) in the body of the webhook. How can I get this Callback uri? If I don't add proper callback uri then the activity runs till timeout. I believe the functioning should be webhook activity completes when the runbook it's running is executed successfully so we can move on to next webhook activity in a pipeline. I have tried web activity as well but it's the same issue.
The body I am using right now is just below json.
{"body":{"myMessage":"Sample"}}
I have referenced:
https://vanishedgradient.com/2019/04/25/webhooks-with-azure-data-factory/
https://mrpaulandrew.com/2019/06/18/azure-data-factory-web-hook-vs-web-activity/
https://social.msdn.microsoft.com/Forums/en-US/2effcefb-e65b-4d5c-8b01-138c95126b79/in-azure-data-factory-v2-how-to-process-azure-analysis-service-cube?forum=AzureDataFactory
Thanks for the links, they are useful sources. I've managed to get this working for a pipeline that calls a runbook to resize azure analysis services. Having the runbook return failure and success information was not well documented.
Here's some code to assist a little, which i've taken from several places but a lot from the open issue (https://github.com/MicrosoftDocs/azure-docs/issues/43897) on this Microsoft page: https://learn.microsoft.com/en-us/azure/data-factory/control-flow-webhook-activity
The datafactory Webhook activity passes in some "Headers", SourceHost which is #pipeline().DataFactory and SourceProcess which is #pipeline().Pipeline. This was so we can do some checking to confirm that the runbook is being run by acceptable processes.
The Body of the call is then other variables we required:
#json(concat('{"AnalysisServer":"', pipeline().parameters.AASName, '", "MinimumSKU":"', pipeline().parameters.SKU,'"}') )
Your runbook needs the WebhookData parameter
param
(
[Parameter (Mandatory=$false)]
[object] $WebhookData
)
You can then grab all the bits you need, including checking if a callbackuri was provided:
if ($WebhookData)
{
# Split apart the WebhookData
$WebhookName = $WebhookData.WebhookName
$WebhookHeaders = $WebhookData.RequestHeader
$WebhookBody = $WebhookData.RequestBody | Convertfrom-Json
$WebhookADF = $WebhookHeaders.SourceHost
$WebhookPipeline = $WebhookHeaders.SourceProcess
Write-Output -InputObject ('Runbook started through webhook {0} called by {1} on {2}.' -f $WebhookName, $WebhookPipeline, $WebhookADF)
# if there's a callBackURI then we've been called by something that is waiting for a response
If ($WebhookBody.callBackUri)
{
$WebhookCallbackURI = $WebhookBody.callBackUri
}
...
}
The variable $WebHookHeaders: #{Connection=Keep-Alive; Expect=100-continue; Host=sXXevents.azure-automation.net; SourceHost=**MYDATAFACTORYNAME**; SourceProcess=**MYPIPELINENAME**; x-ms-request-id=**UNIQUEIDENTIFIER**}
You can then grab information out of your json body: $AzureAnalysisServerName = $WebHookBody.AnalysisServer
Passing an error/failure back to your runbook is relatively easy, note that I put my success/update message in to $Message and only have content in $ErrorMessage if there's been an error:
$ErrorMessage = "Failed to do stuff I wanted"
if ($ErrorMessage)
{
$Output = [ordered]#{ output= #{
AzureAnalysisServerResize = "Failed" }
error = #{
ErrorCode = "ResizeError"
Message = $ErrorMessage
}
statusCode = "500"
}
} else {
$Output = [ordered]#{
output= #{
"AzureAnalysisServerResize" = "Success"
"message" = $Outputmessage
}
statusCode = "200"
}
}
$OutputJson = $Output | ConvertTo-Json -Depth 10
# if we have a callbackuri let the ADF Webhook activity know that the script is complete
# Otherwise it waits until its timeout
If ($WebhookCallBackURI)
{
$WebhookCallbackHeaders = #{
"Content-Type"="application/json"
}
Invoke-WebRequest -UseBasicParsing -Uri $WebhookCallBackURI -Method Post -Body $OutputJson -Header $WebhookCallbackHeaders
}
I then end the if ($WebhookData) { call with an else to say the runbook shouldn't be running if not called from webhook:
} else {
Write-Error -Message 'Runbook was not started from Webhook' -ErrorAction stop
}
Passing back an error message was quiet easy, passing back a success message has been traumatic, but the above seems to work, and in my datafactory pipeline i can access the results.
Output
{
"message": "Analysis Server MYSERVERNAME which is SKU XX is already at or above required SKU XX.",
"AzureAnalysisServerResize": "Success"
}
Note that with the Invoke-WebRequest, some examples online don't specify -UseBasicParsing but we had to as the runbook complained: Invoke-WebRequest : The response content cannot be parsed because the Internet Explorer engine is not available, or Internet Explorer's first-launch configuration is not complete.
I'm not sure if this is best practice but I have something that is working in a Powershell Workflow Runbook.
If the runbook has a webhook defined then you use the webhookdata parameter. Your request body needs to be in JSON format and the $WebhookData param picks it up. For example supposed the Body in your webhook activity looks like this:
{"MyParam":1, "MyOtherParam":"Hello"}
In your runbook you pick up the parameters this way:
Param([object]$WebhookData)
if($WebhookData){
$parameters=(ConvertFrom-Json -InputObject $WebhookData.RequestBody)
if($parameters.MyParam) {$ParamOne = $parameters.MyParam}
if($parameters.MyOtherParam) {$ParamTwo = $parameters.MyOtherParam}
}
The variables in your runbook $ParamOne and $ParamTwo are populated from the parsed JSON Body string. The data factory automatically appends the callBackUri to the Body string. You don't need to create it.
You have to use the $WebhookData name. It's a defined property.
I hope this helps.
Apologies for delay. I had found the complete solution few months back. Thanks to Nick and Sara for adding the pieces. I used similar code as return code. We were using graphical runbooks with limited changes allowed so I just added return code (Powershell) at the end of the runbook to have little to no impact. I plugged in below code:
if ($WebhookData)
{
Write-Output $WebhookData
$parameters = (ConvertFrom-Json -InputObject $WebhookData.RequestBody)
if ($parameters.callBackUri)
{
$callbackuri = $parameters.callBackUri
}
}
if ($callbackuri)
{
Invoke-WebRequest -Uri $callbackuri -UseBasicParsing -Method POST
}
Write-Output $callbackuri
After this I added an input parameter using "Input and Output" button available in the runbook. I named the input parameter as "WebhookData" and type as "Object". The name of input parameter is case-sensitive and should match the parameter used in Powershell code.
This resolved my issue. The runbook started when called from ADF pipeline and moved to next pipeline only when the underlying runbook called by the webhook was completed.