Azure Functions - New-AzContainerGroup : Object reference not set to an instance of an object - azure

I've been trying to create Azure Container instance using http triggered powershell Azure Function.
I've wrote some code and it works... kind of
using namespace System.Net
param($Request, $TriggerMetadata)
$appId = "appID"
$appPassword = "appPassword"
$secpasswd = ConvertTo-SecureString $appPassword -AsPlainText -Force
$mycred = New-Object System.Management.Automation.PSCredential ($appId, $secpasswd)
$status = [HttpStatusCode]::OK
New-AzContainerGroup -ResourceGroupName "test" -Name "testName" `
-Image "imageName" `
-RegistryCredential $mycred `
-RestartPolicy Never
Push-OutputBinding -Name Response -Value ([HttpResponseContext]#{
StatusCode = $status
Body = $body
})
When i hit endpoint, i always get 500 response code, but for some reason container instance is being created anyway and it works. Here's error from terminal:
Executed 'Functions.testFuntion' (Failed, Id=240084c5-6aa2-4207-ae84-66f85a03504b)
System.Private.CoreLib: Exception while executing function: Functions.testFuntion.
System.Private.CoreLib: Result: Failure
Exception: Object reference not set to an instance of an object.
Stack:
at
Microsoft.Azure.Functions.PowerShellWorker.Utility.TypeExtensions.
DeriveContentType(HttpResponseContext
Microsoft.Azure.Functions.PowerShellWorker.Utility.TypeExtensions.DeriveContentType
(HttpResponseContext httpResponseContext, RpcHttp rpcHttp) in C:\projects\azure-functions-powershell-
worker\src\Utility\TypeExtensions.cs:line 196
at
Microsoft.Azure.Functions.PowerShellWorker.Utility.TypeExtensions.ToRpcHttp(HttpResponseContext
httpResponseContext) in C:\projects\azure-functions-powershell-
worker\src\Utility\TypeExtensions.cs:line 188
at
Microsoft.Azure.Functions.PowerShellWorker.Utility.TypeExtensions.ToTypedData(Object value) in
C:\projects\azure-functions-powershell-worker\src\Utility\TypeExtensions.cs:line 242
at
Microsoft.Azure.Functions.PowerShellWorker.RequestProcessor.BindOutputFromResult(InvocationResponse
response, AzFunctionInfo functionInfo, Hashtable results) in C:\projects\azure-functions-powershell-
worker\src\RequestProcessor.cs:line 465
at
Microsoft.Azure.Functions.PowerShellWorker.RequestProcessor.ProcessInvocationRequestImpl
(StreamingMessage request, AzFunctionInfo functionInfo, PowerShellManager psManager,
FunctionInvocationPerformanceStopwatch stopwatch)
in C:\projects\azure-functions-powershell-worker\src\RequestProcessor.cs:line 305.
Is this something i should be worried about? Even though i get 500 response, everything seems to be working.

You may need to define $body before using it in the Push-OutputBinding response.
Something like:
$body = "My Message"
Push-OutputBinding -Name Response -Value ([HttpResponseContext]#{
StatusCode = $status
Body = $body
})

Related

Azure Function gives "InvokeMethodOnNull" error when calling Start-AzContainerGroup

I have a basic Azure Function running Powershell. I have followed instructions from the Microsoft Learn Tutorial to create an HTTP trigger to start a Container Instance. I have modified the tutorial after various attempts, so that the code now just starts an existing Container Instance on my Azure tenant, as follows:
using namespace System.Net
# Input bindings are passed in via param block.
param($Request, $TriggerMetadata)
# Write to the Azure Functions log stream.
Write-Host "PowerShell HTTP trigger function processed a request."
# Interact with query parameters or the body of the request.
$command = $Request.Query.Command
if (-not $command) {
$command = $Request.Body.Command
}
$body = "This HTTP triggered function executed successfully. Either pass 'start' or 'stop' as the 'command' parameter for the appropriate action to be executed on the acme-dns container."
if ($command) {
$body = "Command received: $command"
if ($command = "start") {
Start-AzContainerGroup -ResourceGroupName test -Name dev
}
elseif ($command = "stop") {
Stop-AzContainerGroup -ResourceGroupName test -Name dev
}
if ($?) {
$body = "This HTTP triggered function executed successfully. Started container group $name"
}
else {
$body = "There was a problem starting the container group."
}
}
# Associate values to output bindings by calling 'Push-OutputBinding'.
Push-OutputBinding -Name Response -Value ([HttpResponseContext]#{
StatusCode = [HttpStatusCode]::OK
Body = $body
})
requirements.psd1:
#{
# For latest supported version, go to 'https://www.powershellgallery.com/packages/Az'.
# To use the Az module in your function app, please uncomment the line below.
'Az' = '9.*'
}
Error text when I try to debug locally:
Azure Functions Core Tools
Core Tools Version: 4.0.4915 Commit hash: N/A (64-bit)
Function Runtime Version: 4.14.0.19631
Functions:
HttpExample: [GET,POST] http://localhost:7071/api/HttpExample
For detailed output, run func with --verbose flag.
[2022-12-13T14:27:14.456Z] Worker process started and initialized.
[2022-12-13T14:27:18.512Z] Host lock lease acquired by instance ID '0000000000000000000000000713F6BA'.
[2022-12-13T14:27:38.039Z] Executing 'Functions.HttpExample' (Reason='This function was programmatically called via the host APIs.', Id=5b3c9e21-290f-4ff4-ac49-21f362414926)
[2022-12-13T14:27:38.238Z] INFORMATION: PowerShell HTTP trigger function processed a request.
[2022-12-13T14:27:56.862Z] EXCEPTION: You cannot call a method on a null-valued expression.
[2022-12-13T14:27:56.863Z]
[2022-12-13T14:27:56.863Z] Exception :
[2022-12-13T14:27:56.864Z] Type : System.Management.Automation.ParentContainsErrorRecordException
[2022-12-13T14:27:56.865Z] Message : You cannot call a method on a null-valued expression.
[2022-12-13T14:27:56.866Z] HResult : -2146233087
[2022-12-13T14:27:56.867Z] CategoryInfo : InvalidOperation: (:) [], ParentContainsErrorRecordException
[2022-12-13T14:27:56.868Z] FullyQualifiedErrorId : InvokeMethodOnNull
[2022-12-13T14:27:56.869Z] InvocationInfo :
[2022-12-13T14:27:56.870Z] ScriptLineNumber : 1777
[2022-12-13T14:27:56.871Z] OffsetInLine : 13
[2022-12-13T14:27:56.872Z] HistoryId : -1
[2022-12-13T14:27:56.873Z] ScriptName : C:\Users\{USER}\Documents\PowerShell\Modules\Az.ContainerInstance\3.1.0\exports\ProxyCmdletDefinitions.ps1
[2022-12-13T14:27:56.874Z] Line : [Microsoft.WindowsAzure.Commands.Utilities.Common.AzurePSCmdlet]::PowerShellVersion = $Host.Runspace.Version.ToString()
[2022-12-13T14:27:56.875Z]
[2022-12-13T14:27:56.876Z] PositionMessage : At C:\Users\{USER}\Documents\PowerShell\Modules\Az.ContainerInstance\3.1.0\exports\ProxyCmdletDefinitions.ps1:1777 char:13
[2022-12-13T14:27:56.877Z] + [Microsoft.WindowsAzure.Commands.Utilities.Common.AzurePS .
[2022-12-13T14:27:56.878Z] + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[2022-12-13T14:27:56.879Z] PSScriptRoot : C:\Users\{USER}\Documents\PowerShell\Modules\Az.ContainerInstance\3.1.0\exports
[2022-12-13T14:27:56.880Z] PSCommandPath : C:\Users\{USER}\Documents\PowerShell\Modules\Az.ContainerInstance\3.1.0\exports\ProxyCmdletDefinitions.ps1
[2022-12-13T14:27:56.881Z] CommandOrigin : Internal
[2022-12-13T14:27:56.882Z] ScriptStackTrace : at Start-AzContainerGroup<Begin>, C:\Users\{USER}\Documents\PowerShell\Modules\Az.ContainerInstance\3.1.0\exports\ProxyCmdletDefinitions.ps1: line 1777
[2022-12-13T14:27:56.883Z] at <ScriptBlock>, C:\Coding\AzFunctions\PsHttpTrigger\HttpExample\run.ps1: line 20
[2022-12-13T14:27:56.884Z]
[2022-12-13T14:27:56.884Z]
[2022-12-13T14:27:56.913Z] Executed 'Functions.HttpExample' (Failed, Id=5b3c9e21-290f-4ff4-ac49-21f362414926, Duration=18894ms)
[2022-12-13T14:27:56.914Z] System.Private.CoreLib: Exception while executing function: Functions.HttpExample. System.Private.CoreLib: Result: Failure
Exception: You cannot call a method on a null-valued expression.
Stack: at System.Management.Automation.Runspaces.PipelineBase.Invoke(IEnumerable input)
[2022-12-13T14:27:56.914Z] at System.Management.Automation.Runspaces.Pipeline.Invoke()
[2022-12-13T14:27:56.915Z] at System.Management.Automation.PowerShell.Worker.ConstructPipelineAndDoWork(Runspace rs, Boolean performSyncInvoke)
[2022-12-13T14:27:56.916Z] at System.Management.Automation.PowerShell.Worker.CreateRunspaceIfNeededAndDoWork(Runspace rsToUse, Boolean isSync)
[2022-12-13T14:27:56.917Z] at System.Management.Automation.PowerShell.CoreInvokeHelper[TInput,TOutput](PSDataCollection`1 input, PSDataCollection`1 output, PSInvocationSettings settings)
[2022-12-13T14:27:56.917Z] at System.Management.Automation.PowerShell.CoreInvoke[TInput,TOutput](PSDataCollection`1 input, PSDataCollection`1 output, PSInvocationSettings settings)
[2022-12-13T14:27:56.918Z] at System.Management.Automation.PowerShell.Invoke[T](IEnumerable input, IList`1 output, PSInvocationSettings settings)
[2022-12-13T14:27:56.919Z] at System.Management.Automation.PowerShell.Invoke[T]()
[2022-12-13T14:27:56.919Z] at Microsoft.Azure.Functions.PowerShellWorker.PowerShell.PowerShellExtensions.InvokeAndClearCommands[T](PowerShell pwsh) in /mnt/vss/_work/1/s/src/PowerShell/PowerShellExtensions.cs:line 45
[2022-12-13T14:27:56.920Z] at Microsoft.Azure.Functions.PowerShellWorker.PowerShell.PowerShellManager.InvokeNonOrchestrationFunction(DurableController durableController, IDictionary outputBindings) in /mnt/vss/_work/1/s/src/PowerShell/PowerShellManager.cs:line 301
[2022-12-13T14:27:56.921Z] at Microsoft.Azure.Functions.PowerShellWorker.PowerShell.PowerShellManager.InvokeFunction(AzFunctionInfo functionInfo, Hashtable triggerMetadata, TraceContext traceContext, RetryContext retryContext, IList`1 inputData, FunctionInvocationPerformanceStopwatch stopwatch) in /mnt/vss/_work/1/s/src/PowerShell/PowerShellManager.cs:line 230
[2022-12-13T14:27:56.922Z] at Microsoft.Azure.Functions.PowerShellWorker.RequestProcessor.InvokeFunction(AzFunctionInfo functionInfo, PowerShellManager psManager, FunctionInvocationPerformanceStopwatch stopwatch, InvocationRequest invocationRequest) in /mnt/vss/_work/1/s/src/RequestProcessor.cs:line 336
[2022-12-13T14:27:56.923Z] at Microsoft.Azure.Functions.PowerShellWorker.RequestProcessor.ProcessInvocationRequestImpl(StreamingMessage request, AzFunctionInfo functionInfo, PowerShellManager psManager, FunctionInvocationPerformanceStopwatch stopwatch) in /mnt/vss/_work/1/s/src/RequestProcessor.cs:line 308.
It seems to me that the $Host variable within the AzurePS module is null, but I have no idea how to go about fixing this.
When I run the function from Azure cloud, it times out (Error: 500 - The request timed out. The web server failed to respond within the specified time.)
Running Start-AzContainerGroup from Powershell on my local PC works just fine.
I have modified your code accordingly for the requirement of Starting and stopping the Azure Container groups using PowerShell Azure Functions and it's working as expected:
using namespace System.Net
# Input bindings are passed in via param block.
param($Request, $TriggerMetadata)
# Write to the Azure Functions log stream.
Write-Host "PowerShell HTTP trigger function processed a request."
Connect-AzAccount -Tenant 'TeantId' -SubscriptionId 'SubscriptionId'
Set-AzContext -Subscription 'SubscriptionId'
$command = $Request.Query.Command
if (-not $command) {
$command = $Request.Body.Command
}
$body = "This HTTP triggered function executed successfully. Either pass 'start' or 'stop' as the 'command' parameter for the appropriate action to be executed on the acme-dns container."
if ($command) {
Write-Host "Command received: $command"
if ($command = "start") {
Start-AzContainerGroup -ResourceGroupName HariTestRG -Name test-cg
$body = "This HTTP triggered function executed successfully. Started container group."
}
elseif ($command = "stop") {
Stop-AzContainerGroup -ResourceGroupName HariTestRG -Name test-cg
$body = "This HTTP triggered function executed successfully. Stopped container group."
}
}
Push-OutputBinding -Name Response -Value ([HttpResponseContext]#{
StatusCode = [HttpStatusCode]::OK
Body = $body
})
Function API Response:
Result of Input Parameter to the Function API as Command = Start:
Before running this function, I have created the Container Group in Azure using these PowerShell Cmdlets:
$port1 = New-AzContainerInstancePortObject -Port 8000 -Protocol TCP
$port2 = New-AzContainerInstancePortObject -Port 8001 -Protocol TCP
$container = New-AzContainerInstanceObject -Name test-container -Image nginx -RequestCpu 1 -RequestMemoryInGb 1.5 -Port #($port1, $port2)
$containerGroup = New-AzContainerGroup -ResourceGroupName HariTestRG -Name test-cg -Location eastus -Container $container -OsType Linux -RestartPolicy "Never" -IpAddressType Public
Note: Few Parts of the above code taken from the MS Doc Reference 1 & 2.

Azure Data Factory webhook execution times out instead of relaying errors

I've attempted to set up a simple webhook execution in Azure Data Factory (v2), calling a simple (parameter-less) webhook for an Azure Automation Runbook I set up.
From the Azure Portal, I can see that the webhook is being executed and my runbook is being run, so far so good. The runbook is (currently) returning an error within 1 minute of execution - but that's fine, I also want to test failure scenarios.
Problem:
Data Factory doesn't seem to be 'seeing' the error result and spins until the timeout (10 minutes) elapses. When I kick off a debug run of the pipeline, I get the same - a timeout and no error result.
Update: I've fixed the runbook and it's now completing successfully, but Data Factory is still timing out and is not seeing the success response either.
Here is a screenshot of the setup:
And here is the portal confirming that the webhook is being run by azure data factory, and is completing in under a minute:
WEBHOOKDATA JSON is:
{"WebhookName":"Start CAMS VM","RequestBody":"{\r\n \"callBackUri\": \"https://dpeastus.svc.datafactory.azure.com/dataplane/workflow/callback/f7c...df2?callbackUrl=AAEAFF...0927&shouldReportToMonitoring=True&activityType=WebHook\"\r\n}","RequestHeader":{"Connection":"Keep-Alive","Expect":"100-continue","Host":"eab...ddc.webhook.eus2.azure-automation.net","x-ms-request-id":"7b4...2eb"}}
So as far as I can tell, things should be in place to pick up on the result (success of failure). Hopefully someone who's done this before knows what I'm missing.
Thanks!
I had assumed that Azure would automatically notify the ADF "callBackUri" with the result once the runbook completed or errored out (since they take care of 99% of the scaffolding without requiring a line of code).
It turns out that is not the case, and anyone wishing to execute a runbook from ADF will have to manually extract the callBackUri from the Webhookdata input parameter, and POST the result to it when done.
I haven't nailed this down yet, since the Microsoft tutorial sites I've found have a bad habit of taking screenshots of the code that does this rather than providing the code itself:
I guess I'll come back and edit this once I have it figured out.
EDIT I ended up implementing this by leaving my original Webhook untouched, and creating a "wrapper"/helper/utility Runbook that will execute an arbitrary webhook, and relay its status to ADF once it's complete.
Here's the full code I ended up with, in case it helps someone else. It's meant to be generic:
Setup / Helper Functions
param
(
[Parameter (Mandatory = $false)]
[object] $WebhookData
)
Import-Module -Name AzureRM.resources
Import-Module -Name AzureRM.automation
# Helper function for getting the current running Automation Account Job
# Inspired heavily by: https://github.com/azureautomation/runbooks/blob/master/Utility/ARM/Find-WhoAmI
<#
Queries the automation accounts in the subscription to find the automation account, runbook and resource group that the job is running in.
AUTHOR: Azure/OMS Automation Team
#>
Function Find-WhoAmI {
[CmdletBinding()]
Param()
Begin { Write-Verbose ("Entering {0}." -f $MyInvocation.MyCommand) }
Process {
# Authenticate
$ServicePrincipalConnection = Get-AutomationConnection -Name "AzureRunAsConnection"
Add-AzureRmAccount `
-ServicePrincipal `
-TenantId $ServicePrincipalConnection.TenantId `
-ApplicationId $ServicePrincipalConnection.ApplicationId `
-CertificateThumbprint $ServicePrincipalConnection.CertificateThumbprint | Write-Verbose
Select-AzureRmSubscription -SubscriptionId $ServicePrincipalConnection.SubscriptionID | Write-Verbose
# Search all accessible automation accounts for the current job
$AutomationResource = Get-AzureRmResource -ResourceType Microsoft.Automation/AutomationAccounts
$SelfId = $PSPrivateMetadata.JobId.Guid
foreach ($Automation in $AutomationResource) {
$Job = Get-AzureRmAutomationJob -ResourceGroupName $Automation.ResourceGroupName -AutomationAccountName $Automation.Name -Id $SelfId -ErrorAction SilentlyContinue
if (!([string]::IsNullOrEmpty($Job))) {
return $Job
}
Write-Error "Could not find the current running job with id $SelfId"
}
}
End { Write-Verbose ("Exiting {0}." -f $MyInvocation.MyCommand) }
}
Function Get-TimeStamp {
return "[{0:yyyy-MM-dd} {0:HH:mm:ss}]" -f (Get-Date)
}
My Code
### EXPECTED USAGE ###
# 1. Set up a webhook invocation in Azure data factory with a link to this Runbook's webhook
# 2. In ADF - ensure the body contains { "WrappedWebhook": "<your url here>" }
# This should be the URL for another webhook.
# LIMITATIONS:
# - Currently, relaying parameters and authentication credentials is not supported,
# so the wrapped webhook should require no additional authentication or parameters.
# - Currently, the callback to Azure data factory does not support authentication,
# so ensure ADF is configured to require no authentication for its callback URL (the default behaviour)
# If ADF executed this runbook via Webhook, it should have provided a WebhookData with a request body.
if (-Not $WebhookData) {
Write-Error "Runbook was not invoked with WebhookData. Args were: $args"
exit 0
}
if (-Not $WebhookData.RequestBody) {
Write-Error "WebhookData did not contain a ""RequestBody"" property. Data was: $WebhookData"
exit 0
}
$parameters = (ConvertFrom-Json -InputObject $WebhookData.RequestBody)
# And this data should contain a JSON body containing a 'callBackUri' property.
if (-Not $parameters.callBackUri) {
Write-Error 'WebhookData was missing the expected "callBackUri" property (which Azure Data Factory should provide automatically)'
exit 0
}
$callbackuri = $parameters.callBackUri
# Check for the "WRAPPEDWEBHOOK" parameter (which should be set up by the user in ADF)
$WrappedWebhook = $parameters.WRAPPEDWEBHOOK
if (-Not $WrappedWebhook) {
$ErrorMessage = 'WebhookData was missing the expected "WRAPPEDWEBHOOK" peoperty (which the user should have added to the body via ADF)'
Write-Error $ErrorMessage
}
else
{
# Now invoke the actual runbook desired
Write-Output "$(Get-TimeStamp) Invoking Webhook Request at: $WrappedWebhook"
try {
$OutputMessage = Invoke-WebRequest -Uri $WrappedWebhook -UseBasicParsing -Method POST
} catch {
$ErrorMessage = ("An error occurred while executing the wrapped webhook $WrappedWebhook - " + $_.Exception.Message)
Write-Error -Exception $_.Exception
}
# Output should be something like: {"JobIds":["<JobId>"]}
Write-Output "$(Get-TimeStamp) Response: $OutputMessage"
$JobList = (ConvertFrom-Json -InputObject $OutputMessage).JobIds
$JobId = $JobList[0]
$OutputMessage = "JobId: $JobId"
# Get details about the currently running job, and assume the webhook job is being run in the same resourcegroup/account
$Self = Find-WhoAmI
Write-Output "Current Job '$($Self.JobId)' is running in Group '$($Self.ResourceGroupName)' and Automation Account '$($Self.AutomationAccountName)'"
Write-Output "Checking for Job '$($JobId)' in same Group and Automation Account..."
# Monitor the job status, wait for completion.
# Check against a list of statuses that likely indicate an in-progress job
$InProgressStatuses = ('New', 'Queued', 'Activating', 'Starting', 'Running', 'Stopping')
# (from https://learn.microsoft.com/en-us/powershell/module/az.automation/get-azautomationjob?view=azps-4.1.0&viewFallbackFrom=azps-3.7.0)
do {
# 1 second between polling attempts so we don't get throttled
Start-Sleep -Seconds 1
try {
$Job = Get-AzureRmAutomationJob -Id $JobId -ResourceGroupName $Self.ResourceGroupName -AutomationAccountName $Self.AutomationAccountName
} catch {
$ErrorMessage = ("An error occurred polling the job $JobId for completion - " + $_.Exception.Message)
Write-Error -Exception $_.Exception
}
Write-Output "$(Get-TimeStamp) Polled job $JobId - current status: $($Job.Status)"
} while ($InProgressStatuses.Contains($Job.Status))
# Get the job outputs to relay to Azure Data Factory
$Outputs = Get-AzureRmAutomationJobOutput -Id $JobId -Stream "Any" -ResourceGroupName $Self.ResourceGroupName -AutomationAccountName $Self.AutomationAccountName
Write-Output "$(Get-TimeStamp) Outputs from job: $($Outputs | ConvertTo-Json -Compress)"
$OutputMessage = $Outputs.Summary
Write-Output "Summary ouput message: $($OutputMessage)"
}
# Now for the entire purpose of this runbook - relay the response to the callback uri.
# Prepare the success or error response as per specifications at https://learn.microsoft.com/en-us/azure/data-factory/control-flow-webhook-activity#additional-notes
if ($ErrorMessage) {
$OutputJson = #"
{
"output": { "message": "$ErrorMessage" },
"statusCode": 500,
"error": {
"ErrorCode": "Error",
"Message": "$ErrorMessage"
}
}
"#
} else {
$OutputJson = #"
{
"output": { "message": "$OutputMessage" },
"statusCode": 200
}
"#
}
Write-Output "Prepared ADF callback body: $OutputJson"
# Post the response to the callback URL provided
$callbackResponse = Invoke-WebRequest -Uri $callbackuri -UseBasicParsing -Method POST -ContentType "application/json" -Body $OutputJson
Write-Output "Response was relayed to $callbackuri"
Write-Output ("ADF replied with the response: " + ($callbackResponse | ConvertTo-Json -Compress))
At a high-level, steps I've taken are to:
Execute the "main" Webhook - get back a "Job Id"
Get the current running job's "context" (resource group and automation account info) so that I can poll the remote job.
Poll the job until it is complete
Put together either a "success" or "error" response message in the format that Azure Data Factory expects.
Invoke the ADF callback.
For those looking, I created a secondary approach to the above solution - one that executes a Runbook (with parameters) from a Webhook, rather than a invoking a nested Webhook. This has a couple of benefits:
Parameters can be passed through to the Runbook (rather than requiring parameters to be baked-into a new Webhook.
A Runbook from another Azure Automation Account / Resource Group can be invoked.
There's no need to poll the status of the job, since the Start-AzureRmAutomationRunbook commandlet has a -Wait parameter.
Here's the code:
param
(
# Note: While "WebhookData" is the only root-level parameter (set by Azure Data Factory when it invokes this webhook)
# The user should ensure they provide (via the ADF request body) these additional properties required to invoke the runbook:
# - RunbookName
# - ResourceGroupName (TODO: Can fill this in by default if not provided)
# - AutomationAccountName (TODO: Can fill this in by default if not provided)
# - Parameters (A nested dict containing parameters to forward along)
[Parameter (Mandatory = $false)]
[object] $WebhookData
)
Import-Module -Name AzureRM.resources
Import-Module -Name AzureRM.automation
Function Get-TimeStamp {
return "[{0:yyyy-MM-dd} {0:HH:mm:ss}]" -f (Get-Date)
}
# If ADF executed this runbook via Webhook, it should have provided a WebhookData with a request body.
if (-Not $WebhookData) {
Write-Error "Runbook was not invoked with WebhookData. Args were: $args"
exit 0
}
if (-Not $WebhookData.RequestBody) {
Write-Error "WebhookData did not contain a ""RequestBody"" property. Data was: $WebhookData"
exit 0
}
$parameters = (ConvertFrom-Json -InputObject $WebhookData.RequestBody)
# And this data should contain a JSON body containing a 'callBackUri' property.
if (-Not $parameters.callBackUri) {
Write-Error 'WebhookData was missing the expected "callBackUri" property (which Azure Data Factory should provide automatically)'
exit 0
}
$callbackuri = $parameters.callBackUri
# Check for required parameters, and output any errors.
$ErrorMessage = ''
$RunbookName = $parameters.RunbookName
$ResourceGroupName = $parameters.ResourceGroupName
$AutomationAccountName = $parameters.AutomationAccountName
if (-Not $RunbookName) {
$ErrorMessage += 'WebhookData was missing the expected "RunbookName" property (which the user should have added to the body via ADF)`n'
} if (-Not $ResourceGroupName) {
$ErrorMessage += 'WebhookData was missing the expected "ResourceGroupName" property (which the user should have added to the body via ADF)`n'
} if (-Not $AutomationAccountName) {
$ErrorMessage += 'WebhookData was missing the expected "AutomationAccountName" property (which the user should have added to the body via ADF)`n'
} if ($ErrorMessage) {
Write-Error $ErrorMessage
} else {
# Set the current automation connection's authenticated account to use for future Azure Resource Manager cmdlet requests.
# TODO: Provide the user with a way to override this if the target runbook doesn't support the AzureRunAsConnection
$ServicePrincipalConnection = Get-AutomationConnection -Name "AzureRunAsConnection"
Add-AzureRmAccount -ServicePrincipal `
-TenantId $ServicePrincipalConnection.TenantId `
-ApplicationId $ServicePrincipalConnection.ApplicationId `
-CertificateThumbprint $ServicePrincipalConnection.CertificateThumbprint | Write-Verbose
Select-AzureRmSubscription -SubscriptionId $ServicePrincipalConnection.SubscriptionID | Write-Verbose
# Prepare the properties to pass on to the next runbook - all provided properties exept the ones specific to the ADF passthrough invocation
$RunbookParams = #{ }
if($parameters.parameters) {
$parameters.parameters.PSObject.Properties | Foreach { $RunbookParams[$_.Name] = $_.Value }
Write-Output "The following parameters will be forwarded to the runbook: $($RunbookParams | ConvertTo-Json -Compress)"
}
# Now invoke the actual runbook desired, and wait for it to complete
Write-Output "$(Get-TimeStamp) Invoking Runbook '$($RunbookName)' from Group '$($ResourceGroupName)' and Automation Account '$($AutomationAccountName)'"
try {
# Runbooks have this nice flag that let you wait on their completion (unlike webhook-invoked)
$Result = Start-AzureRmAutomationRunbook -Wait -Name $RunbookName -AutomationAccountName $AutomationAccountName -ResourceGroupName $ResourceGroupName –Parameters $RunbookParams
} catch {
$ErrorMessage = ("An error occurred while invoking Start-AzAutomationRunbook - " + $_.Exception.Message)
Write-Error -Exception $_.Exception
}
# Digest the result to be relayed to ADF
if($Result) {
Write-Output "$(Get-TimeStamp) Response: $($Result | ConvertTo-Json -Compress)"
$OutputMessage = $Result.ToString()
} elseif(-Not $ErrorMessage) {
$OutputMessage = "The runbook completed without errors, but the result was null."
}
}
# Now for the entire purpose of this runbook - relay the response to the callback uri.
# Prepare the success or error response as per specifications at https://learn.microsoft.com/en-us/azure/data-factory/control-flow-webhook-activity#additional-notes
if ($ErrorMessage) {
$OutputJson = #{
output = #{ message = $ErrorMessage }
statusCode = 500
error = #{
ErrorCode = "Error"
Message = $ErrorMessage
}
} | ConvertTo-Json -depth 2
} else {
$OutputJson = #{
output = #{ message = $OutputMessage }
statusCode = 200
} | ConvertTo-Json -depth 2
}
Write-Output "Prepared ADF callback body: $OutputJson"
# Post the response to the callback URL provided
$callbackResponse = Invoke-WebRequest -Uri $callbackuri -UseBasicParsing -Method POST -ContentType "application/json" -Body $OutputJson
Write-Output "Response was relayed to $callbackuri"
Write-Output ("ADF replied with the response: " + ($callbackResponse | ConvertTo-Json -Compress))

Powershell Runbook [Invoke-ASCmd], FileNotFoundException for xmla file

I'm trying to make a script that creates automatic partions for SSAS via Powershell Runbook, but whenever I try to read in the xmla file i get the following error:
My code that calls this is as followed:
$StorageAccount = Get-AzureRmStorageAccount -ResourceGroupName $ResourceGroupName -Name $StorageAccountName
$blob = Get-AzureStorageBlob -Context $StorageAccount.Context -Container "Database name" -Blob "CreateNewPartition.xmla"
$file = $blob.ICloudBlob.DownloadText()
Invoke-ASCmd `
-Database $AnalysisServiceDatabase `
-InputFile $file `
-server $AnalysisServiceServer
When using the following code:
$memStream = New-Object System.IO.MemoryStream
$blob.ICloudBlob.DownloadToStream($memStream)
$readStream = New-Object System.IO.StreamReader($memStream, [System.Text.Encoding]::Unicode)
$memStream.Position = 0
$file = ($readStream.ReadToEnd() -replace "`0",'' | ConvertFrom-Json)
I get this error:
And when trying this code:
$byteArray = New-Object Byte[] $blob.Length
$file = $blob.ICloudBlob.DownloadToByteArray($byteArray, 0)
I get this error:
Easy fix.
In the first example, you are correctly reading the contents of the file from the blob. But, -InputFile is expecting a file path (e.g. C:\arst.xmla), and can't handle the raw contents of the .xmla file.
Instead, use the -Query parameter to pass the contents of the file to Invoke-ASCmd e.g.:
...
$query = $blob.ICloudBlob.DownloadText()
Invoke-ASCmd `
-Database $AnalysisServiceDatabase `
-Query $query `
-server $AnalysisServiceServer

Cannot POST Cosmos DB Stored Procedures using PowerShell

I'm trying to deploy stored procedures to a collection within an Azure Cosmos DB account as part of my deployment pipeline in Azure DevOps. Due to security reasons, I have to use the REST API (cannot use or import PowerShell modules to do this).
The build agents that I'm using are on-premise agents. Again, security reasons.
In order to generate an authorization token to make requests, I have the following PowerShell function:
Add-Type -AssemblyName System.Web
# Generate Authorization Key for REST calls
Function Generate-MasterKeyAuthorizationSignature
{
[CmdletBinding()]
Param
(
[Parameter(Mandatory=$true)][String]$verb,
[Parameter(Mandatory=$true)][String]$resourceLink,
[Parameter(Mandatory=$true)][String]$resourceType,
[Parameter(Mandatory=$true)][String]$dateTime,
[Parameter(Mandatory=$true)][String]$key,
[Parameter(Mandatory=$true)][String]$keyType,
[Parameter(Mandatory=$true)][String]$tokenVersion
)
$hmacSha256 = New-Object System.Security.Cryptography.HMACSHA256
$hmacSha256.Key = [System.Convert]::FromBase64String($key)
$payLoad = "$($verb.ToLowerInvariant())`n$($resourceType.ToLowerInvariant())`n$resourceLink`n$($dateTime.ToLowerInvariant())`n`n"
$hashPayLoad = $hmacSha256.ComputeHash([System.Text.Encoding]::UTF8.GetBytes($payLoad))
$signature = [System.Convert]::ToBase64String($hashPayLoad);
[System.Web.HttpUtility]::UrlEncode("type=$keyType&ver=$tokenVersion&sig=$signature")
}
I then call this function within the POST function that I'm using to POST the stored procedure to Azure Cosmos DB:
Function Post-StoredProcToCosmosDb
{
[CmdletBinding()]
Param
(
[Parameter(Mandatory=$true)][String]$EndPoint,
[Parameter(Mandatory=$true)][String]$DataBaseId,
[Parameter(Mandatory=$true)][String]$CollectionId,
[Parameter(Mandatory=$true)][String]$MasterKey,
[Parameter(Mandatory=$true)][String]$JSON
)
$Verb = 'Post'
$ResourceType = "sprocs"
$ResourceLink = "dbs/$DataBaseId/colls/$CollectionId"
$dateTime = [DateTime]::UtcNow.ToString("r")
$authHeader = Generate-MasterKeyAuthorizationSignature -verb $Verb -resourceLink $ResourceLink -resourceType $ResourceType -key $MasterKey -keyType "master" -tokenVersion "1.0" -dateTime $dateTime
$header = #{authorization=$authHeader;"x-ms-version"="2017-02-22";"x-ms-date"=$dateTime;"xs-ms-session-token"="28"}
$contentType = "application/json"
$queryUri = "$EndPoint$ResourceLink/sprocs"
$result = Invoke-RestMethod -Headers $header -Method $Verb -ContentType $contentType -Uri $queryUri -Body $JSON
return $result.statuscode
}
I see from the documentation that I need to pass my stored procedure in the body as a string, so I set the path of my stored procedure to a variable like so:
$HelloWorld = Get-Content -Path '.\Databases\MyCosmosDB\MyCosmosCollection\Stored Procedures\HelloWorld.js' | Out-String
I then call my POST function like so:
Post-StoredProcToCosmosDb -EndPoint $env:COSMOSENDPOINT -DataBaseId $env:MYCOSMOSDB -CollectionId $MyCollection -MasterKey $env:MASTERKEY -JSON $HelloWorld
However, when I run the task, I get the following error:
Invoke-RestMethod : The remote server returned an error: (400) Bad Request
At D:_workAzure\r12\a_Cosmos\Deployment\scripts\postStoredProcedures.ps1:61 char:15
$result = Invoke-RestMethod -Headers $header -Method $Verb -Content ...
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand
I have followed the example request body as outlined in the documentation for my stored procedure.
Here is what it looks like for your info:
"function () {\r\n var context = getContext();\r\n var response = context.getResponse();\r\n\r\n response.setBody(\"Hello, World\");\r\n}"
I'm fairly new to PowerShell, so I'm wondering where I am going wrong. I've tried setting the contentType to both application/json and application/query+json but otherwise, I'm not sure where I am going wrong?
If anyone can provide any guidance on this, I'd be most grateful.
So turns out the request body was wrong. It should be like this:
{
"body":"function HelloWorld() {\r\n var context = getContext();\r\n var response = context.getResponse();\r\nresponse.setBody(\"Hello, World\");\r\n}",
"id": "HelloWorld"
}
That's the acceptable request body for Stored Procedures. So what I should have done is set my $HelloWorld variable to:
$HelloWorld = Get-Content -Path '.\Databases\MyCosmosDB\MyCosmosCollection\Stored Procedures\HelloWorld.json' | Out-String
Hope my stupidity helps someone someday :/

Start-AzureSqlDatabaseExport Object Reference not set to an instance of an object

I'm not sure how to debug this, assuming it's not a problem with the cmdlet. I'm trying to replace the automated SQL export with an automation workflow, but I can't seem to get Start-AzureSqlDatabaseExport to work -- it keeps getting the following warning and error messages.
d4fc0004-0c0b-443e-ad1b-310af7fd4e2a:[localhost]:Client Session Id: 'c12c92eb-acd5-424d-97dc-84c4e9c4f914-2017-01-04
19:00:23Z'
d4fc0004-0c0b-443e-ad1b-310af7fd4e2a:[localhost]:Client Request Id: 'd534f5fd-0fc0-4d68-8176-7508b35aa9d8-2017-01-04
19:00:33Z'
Start-AzureSqlDatabaseExport : Object reference not set to an instance of an object.
At DBBackup:11 char:11
+
+ CategoryInfo : NotSpecified: (:) [Start-AzureSqlDatabaseExport], NullReferenceException
+ FullyQualifiedErrorId : Microsoft.WindowsAzure.Commands.SqlDatabase.Database.Cmdlet.StartAzureSqlDatabaseExport
This seems similar to some other questions, but they seem to be unanswered or not applicable. I did have a similar procedure working in the Powershell environment. I replaced that procedure with the automated export from Azure, which seems like a poor choice now! I've tried a number of variations, using sqlcontext and databasename instead of database, for example.
Here's my code with sensitive parts replaced with ****:
workflow DBBackup {
param(
[parameter(Mandatory=$true)]
[string] $dbcode
)
$cred = Get-AutomationPSCredential -Name "admindbcredentials"
$VerbosePreference = "Continue"
inlineScript {
$dbcode = $using:dbcode
$cred = $using:cred
if ($dbcode -eq $null)
{
Write-Output "Database code must be specified"
}
Else
{
$dbcode = $dbcode.ToUpper()
$dbsize = 1
$dbrestorewait = 10
$dbserver = "kl8p7d444a"
$stacct = $dbcode.ToLower()
$stkey = "***storagekey***"
Write-Verbose "DB Server '$dbserver' DB Code '$dbcode'"
Write-Verbose "Storage Account '$stacct'"
$url = "https://$dbserver.database.windows.net"
$sqlctx = New-AzureSqlDatabaseServerContext -ManageUrl $url -Credential $cred
# $sqlctx = New-AzureSqlDatabaseServerContext -ManageUrl $url -Credential $cred
$stctx = New-AzureStorageContext -StorageAccountName $stacct -StorageAccountKey $stkey
$dbname = "FSUMS_" + $dbcode
$dt = Get-Date
$timestamp = $dt.ToString("yyyyMMdd") + "_" + $dt.ToString("HHmmss")
$bkupname = $dbname + "_" + $timestamp + ".bacpac"
$stcon = Get-AzureStorageContainer -Context $stctx -Name "backups"
$db = Get-AzureSqlDatabase -Context $sqlctx -DatabaseName $dbname
Write-Verbose "Backup $dbname to $bkupname in storage account $stacct"
Start-AzureSqlDatabaseExport $sqlctx -DatabaseName $dbname -StorageContainer $stcon -BlobName $bkupname
}
}
}
Try New-AzureRmSqlDatabaseExport instead. This command will return export status object. If you want a synchronous export you can check for "export status" in a loop.
Adding the following lines corrected the problem:
In the workflow before inlineScript:
$cred = Get-AutomationPSCredential -Name "admincredentials"
(where admincredentials was an asset with my admin login credentials)
and inside the inlineScript:
Add-AzureAccount $cred
Select-AzureSubscription "My subscription"
Some runbooks don't seem to need this, but probably best to always include it.

Resources