Azure Automation runbook fails when triggered from a Webhook
We have a runbook that runs just fine when launched via the “Start” button in the Azure Portal.
The exact same runbook fails when launched from a webhook using the exact same parameters.
The issue is that after the code in the runbook opens a valid connection to Azure, it tries to get one of the databases we are going to work with and it fails with an exception that reads “Resource group '''' could not be found.”
The same exact code and parameters work without any issues when run outside the Webhook.
We added code to get the context after the code does the login and we checked that the context name, subscription ID and Tenant ID are valid, and we even pass the context to the Get-AzSqlDatabase call and still it fails when triggered via the webhook. We are not sure what else we can try to diagnose and correct the issue.
Here is a sample of the code we run and where it fails...
. . .
Function Login {
Disable-AzContextAutosave -Scope Process
Write-Output "Logging in to Azure..."
$connectionName = "GoodTestingConnection"
try{
Connect-AzAccount -ServicePrincipal -Tenant $servicePrincipalConnection.TenantId `
-ApplicationId $servicePrincipalConnection.ApplicationId `
-CertificateThumbprint $servicePrincipalConnection.CertificateThumbprint
}
catch {
Write-Output "Error loging in to Azure: $_"
throw "Error loging in to Azure: $_"
}
}
$ErrorActionPreference = "Stop"
if ($WebHookData) {
Write-Output "Running runbook from web hook request"
$wbRunParams = (ConvertFrom-Json -InputObject $WebHookData.RequestBody)
Write-Output "Webhook parameters $wbRunParams"
$client = $wbRunParams.client
Write-Output "Running process for client $client"
}
Login
$ctx = Get-AzContext
Write-Output "Context nanme: $($ctx.Name)"
Write-Output "Environment name: $($ctx.Environment.Name)"
Write-Output "Subscription ID: $($ctx.Subscription.Id)"
Write-Output "Tenant ID: $($ctx.Tenant.Id)"
Write-Output "Getting the 'from' Db..."
$fromDb = Get-AzSqlDatabase `
-DatabaseName $copyFromDbName `
-ServerName $copyFromServer `
-ResourceGroupName $copyFromRG `
-DefaultProfile $ctx
$fromDb
#The code never reaches this point as it fails on the previous call
. . .
I had the same issue. Here is what worked for me.
When the automation runbook is triggered via the start option or via the test pane, the, the format needs to be converted. This is when you would use
$wbRunParams = (ConvertFrom-Json -InputObject $WebHookData.RequestBody)
When the runbook has to be triggered directly by webhook, no need to convert and the below would work.
$wbRunParams = $WebHookData.RequestBody
Related
I am trying to make a code using PowerShell so that secrets are not hardcoded on my runbook so that it will not be exposed in the script. I created encrypted variables in my automation account. These variables are AppID, AppSecret and TenantID.
This is the part of the script to login automatically to Azure. I didn't use managed identity for some compatibility reasons with the script.
My script is running fine when secrets and IDs are hardcoded but when I created variables it is not working. Error message is "Run Connect-AzAccount". Below is my code. Need help on how to correct this. Thank you in advance.
$AzVariableApplicationID = 'AppID'
$AzVariableAppSecret = 'AppSecret'
$AzVariableTenantID = 'TenantID'
$AppID = Get-AzAutomationVariable -Name $AzVariableApplicationID
$AppSecret = Get-AzAutomationVariable -Name $AzVariableAppSecret
$TenantID = Get-AzAutomationVariable -Name $AzVariableTenantID
$SecureSecret = $AppSecret | ConvertTo-SecureString -AsPlainText -Force
$Credential = New-Object -TypeName System.Management.Automation.PSCredential `
-ArgumentList $AppID, $SecureSecret
Connect-AzAccount -ServicePrincipal -Credential $Credential -Tenant $TenantID
As described in PsCustom Object - Hitchikers GUID(e) to Automation
, It is not possible to retrieve values for encrypted variables as they’re available within the runbook at runtime via the Get-AutomationVariable cmdlet
I found an alternative approach to "Connect Azure" by using "Certificate-based authentication" inside PowerShell runbook without hardcoding the values:
Created a new Service principal and provided the "Owner" role access to avoid any restrictions.
To authenticate via service principal, I create a new self-signed certificate with the command:
$cert=New-SelfSignedCertificate -Subject "CN=xxxxxCert" -CertStoreLocation "Cert:\CurrentUser\My" -KeyExportPolicy Exportable -KeySpec Signature
Upload a certificate under Certifications & Secrets:
Click windows + R to open the run box and give certmgr.msc as shown here.
Export a certificate without private key.
Upload a certificate in the below path:
AzureAD -> App registrations -> Serviceprincipal
Now, I have exported the same certificate with key and uploaded inside my automation account to authenticate Service principal connection:
Added an "Azure Service Principal" connection inside automation accounts by providing "ApplicationID, TenantID, Certificate Thumbprint" of my Service principal as shown:
Inside PowerShell runbook, I ran the below script that works for me:
$connectionName = "serviceprincipalname"
try
{
$servicePrincipalConnection=Get-AutomationConnection -Name $connectionName
"Logging in to Azure..."
Add-AzureRMAccount
-ServicePrincipal `
-TenantID $servicePrincipalConnection.TenantID `
-ApplicationID $servicePrincipalConnection.applicationID `
-CertificateThumbprint $servicePrincipalConnection.CertificateThumbprint
}
catch {
if (!$servicePrincipalConnection)
{
$ErrorMessage = "Connection $connectionName not found. "
throw $ErrorMessage
} else{
Write-Error -Message $_.Exception
throw $_.Exception
}
}
Logged in to Azure:
Register an App in App registrations and create a service principal in AzureAD
I have an Azure Analysis Services with S1 SKU. There is an SPN who has OWNER RBAC over the AAS. I am trying to use a PowerShell 5.1 Runbook through an Automation Account to run the Restart-AzAnalysisServicesInstance cmdlet.
When I run the Runbook, I get to see the error:
Restart-AzAnalysisServicesInstance : Response status code does not indicate success: 401 (Unauthorized).
However, when I run the cmdlet locally, using my credentials to Connect-AzAccount in Windows Powershell ISE, it works. I am also an OWNER over the AAS.
Here's the Runbook:
# Init
$ErrorActionPreference = 'Stop'
$AutomationAccountConnectionName = "Name of my Connection that uses the Owner SPN"
# Get Automation connection (SPN connection details)
$servicePrincipalConnection = Get-AutomationConnection -Name $AutomationAccountConnectionName
Write-Output "Connected using SPN:"
$servicePrincipalConnection
# Connect using SPN
Write-Output "Connecting to AZ using the SPN connection:"
$Connection | ConvertTo-Json
$azContext = Connect-AzAccount -CertificateThumbprint $servicePrincipalConnection.CertificateThumbprint `
-ApplicationId $servicePrincipalConnection.ApplicationId `
-Tenant $servicePrincipalConnection.TenantId -ServicePrincipal
Write-Output ("Connected to azure using certificate with app id : " + $Connection.AppId)
# Get AAS
$aasServer = "test113aas"
$subscriptionId = "GUID of my azure subscrition"
Select-AzSubscription -Subscription $subscriptionId
$resourceObj = Get-AzAnalysisServicesServer -Name $aasServer
$ResourceObj
$AnalysisServer = $resourceObj.Name
$AnalysisServerLocation = 'northeurope'
$ModelName = 'adventureworks'
# # Connect AAS Account => This did not help as well
# Write-Host "Adding AAS Account"
# Add-AzAnalysisServicesAccount -RolloutEnvironment "$AnalysisServerLocation.asazure.windows.net" `
# -ServicePrincipal -ApplicationId $servicePrincipalConnection.ApplicationId `
# -CertificateThumbprint $servicePrincipalConnection.CertificateThumbprint -TenantId $servicePrincipalConnection.TenantId
# Restart AAS server
Write-Host "Server's full name is $($resourceObj.ServerFullName)"
Write-Host "$AnalysisServer : Preparing to Restart the Analysis Server"
$result = Restart-AzAnalysisServicesInstance –Instance $resourceObj.ServerFullName -PassThru # returns true if successful
$result
Any idea as to what I am missing out here? Documentation: https://learn.microsoft.com/en-us/powershell/module/az.analysisservices/restart-azanalysisservicesinstance?view=azps-7.2.0
I even tried running the same within an Azure PowerShell Core Function, since the documentation is for PowerShell 7+, but to no avail.
Turns out that we need to whitelist the client's IP before calling the cmdlet.
The error message could have been more precise.
The following code works great via Azure Cloud Shell (completes, App Service deleted as expected).
Remove-AzWebApp -ResourceGroupName "ResourceGroup1" -Name "AppService1" -Force
It also completes without error within my Runbook workflow, but the App Service remains operational. This feels like a permissions problem, but I've tried adding the Owner role at the subscription level without success.
Any ideas/tips for how to make this work for the AzureRunAsConnection account?
This feels like a permissions problem, but I've tried adding the Owner role at the subscription level without success.
It is not a permission problem, when you create the automation account along with the RunAsAccount, it will add the service principal related to the RunAsAccount to the subscription as a Contributor role, which is enough to remove the web app.
If you are using the PowerShell Workflow Runbook, try the sample below, it works for me. (First, make sure you have installed the Az.Accounts, Az.Websites modules in the automation account -> Modules.)
workflow testrun3
{
$connectionName = "AzureRunAsConnection"
try
{
# Get the connection "AzureRunAsConnection "
$servicePrincipalConnection=Get-AutomationConnection -Name $connectionName
Connect-AzAccount `
-ServicePrincipal `
-TenantId $servicePrincipalConnection.TenantId `
-ApplicationId $servicePrincipalConnection.ApplicationId `
-CertificateThumbprint $servicePrincipalConnection.CertificateThumbprint
}
catch {
if (!$servicePrincipalConnection)
{
$ErrorMessage = "Connection $connectionName not found."
throw $ErrorMessage
} else{
Write-Error -Message $_.Exception
throw $_.Exception
}
}
Remove-AzWebApp -ResourceGroupName "<group-name>" -Name "joywebapp1234" -Force
}
Check the result in the portal:
To create or update a Run As account, you must have specific privileges and permissions. An Application administrator in Azure Active Directory and an Owner in a subscription can complete all the tasks. Use Remove-AzAutomationConnection to remove an Automation connection.
For more details, you could refer to this article about Run As account permissions.
I am using Automation Account with execution of PowerShell Runbook for Pausing my Azure Datawarehouse database.
Setup is completed with creation of crendentials, certificate for Thumbprints.
When I executed the PS script and completed with error message
No Azure SQL Data Warehouse named ADWPOC exist on SQL Server At SuspendOrPauseAzureSQLDataWarehouse:72 char:72 + + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException
I have provided the name of database and server details correctly while executing this script. I am not sure what is the problem here. Please advise.
I have imported the PS script from runbook gallery "Suspend Or Pause Azure SQL Data Warehouse". Please let me know if you want me to attach the script here?
Not sure, but the script looks too old, it still uses the AzureRm module which has been deprecated.
To pause the data warehouse in the runbook, I recommend you to use new Az module, sample here, to use that in runbook, just follow the steps as below.
1.Navigate to your automation account in the portal -> Modules, make sure you have imported the Az.Accounts and Az.Sql modules, if not, in the Modules -> Browse Gallery, search for the modules and import them.
2.After importing the modules successfully, navigate to the Runbooks -> create a PowerShell runbook(not the PowerShell Workflow), then use the script as below in the runbook.
Note: The <server-name> should be like testserver, not testserver.database.windows.net.
$connectionName = "AzureRunAsConnection"
try
{
# Get the connection "AzureRunAsConnection "
$servicePrincipalConnection=Get-AutomationConnection -Name $connectionName
"Logging in to Azure..."
Connect-AzAccount `
-ServicePrincipal `
-TenantId $servicePrincipalConnection.TenantId `
-ApplicationId $servicePrincipalConnection.ApplicationId `
-CertificateThumbprint $servicePrincipalConnection.CertificateThumbprint
}
catch {
if (!$servicePrincipalConnection)
{
$ErrorMessage = "Connection $connectionName not found."
throw $ErrorMessage
} else{
Write-Error -Message $_.Exception
throw $_.Exception
}
}
$database = Get-AzSqlDatabase –ResourceGroupName "<resource-group-name>" –ServerName "<server-name>" –DatabaseName "<data-warehouse-name>"
if($database){
if($database.Status -eq 'Paused'){
Write-Output "The Data Warehouse was already paused."
}else{
$database | Suspend-AzSqlDatabase
Write-Output "The Data Warehouse has been paused."
}
}else{
Write-Output "The Data Warehouse does not exist."
}
I would like to know how to create an alert for an Azure VM which tells me if the server(s) is running at a specific time.
The scenario:
Servers for the Azure network need to start at 7:30am to be ready for the users as they shut down at 7:30pm each day to save $$.
Today the azure automation script could not find any vms for the resource group! So that meant the servers where not started. I want to create an alert that will only tell me if the server(s) are not running at say 7:45am. So I can start them.
(Running the script now does find all of the servers now but didn't before for some reason... maybe Azure was moving the vms in the resource group?)
I have looked at:
- Microsoft Operations Management Suit > Log Search > Add Alert Rule.
- Resource Manager > Virtual Machines > Monitoring > Alert Rules > Add metic alert & Add activity log alert.
But I can't see where to only run the alert at a specific time.
Update/Edit:
Script used:
param (
[Parameter(Mandatory=$false)]
[String]$AzureCredentialAssetName = 'AzureCred',
[Parameter(Mandatory=$false)]
[String]$AzureSubscriptionIDAssetName = 'AzureSubscriptionId'
)
# Setting error and warning action preferences
$ErrorActionPreference = "SilentlyContinue"
$WarningPreference = "SilentlyContinue"
# Connecting to Azure
$Cred = Get-AutomationPSCredential -Name $AzureCredentialAssetName -ErrorAction Stop
$null = Add-AzureAccount -Credential $Cred -ErrorAction Stop -ErrorVariable err
$null = Add-AzureRmAccount -Credential $Cred -ErrorAction Stop -ErrorVariable err
# Selecting the subscription to work against
$SubID = Get-AutomationVariable -Name $AzureSubscriptionIDAssetName
Select-AzureRmSubscription -SubscriptionId $SubID
# Getting all resource groups
$ResourceGroup = "Servers"
# Getting all virtual machines
$RmVMs = (Get-AzureRmVM -ResourceGroupName $ResourceGroup -ErrorAction $ErrorActionPreference -WarningAction $WarningPreference).Name
# Managing virtual machines deployed with the Resource Manager deployment model
"Loop through all VMs in resource group $ResourceGroup."
if ($RmVMs)
{
foreach ($RmVM in $RmVMs)
{
"`t$RmVM found ..."
$RmPState = (Get-AzureRmVM -ResourceGroupName $ResourceGroup -Name $RmVM -Status -ErrorAction $ErrorActionPreference -WarningAction $WarningPreference).Statuses.Code[1]
if ($RmPState -eq 'PowerState/deallocated')
{
"`t$RmVM is starting up ..."
$RmSState = (Start-AzureRmVM -ResourceGroupName $ResourceGroup -Name $RmVM -ErrorAction $ErrorActionPreference -WarningAction $WarningPreference).IsSuccessStatusCode
if ($RmSState -eq 'True')
{
"`t$RmVM has been started."
}
else
{
"`t$RmVM failed to start."
}
}
}
}
else
{
"No VMs for $ResourceGroup deployed with the Resource Manager deployment model."
}
"Runbook Completed."
I just want a fail safe to know if the servers are not running when they should be.
Expected output:
Loop through all VMs in resource group Servers.
DOMAINCONTROLLER found ...
SQLSERVER found ...
GATEWAY found ...
APPLICATIONHOST found ...
Runbook Completed.
instead of:
Loop through all VMs in resource group Servers.
No VMs for Servers deployed with the Resource Manager deployment model.
Runbook Completed.
I.e. rerunning the same script manually gave expected results.
Based on my knowledge, Azure metric alert could not send mail when your VM has been started or stopped at a specific time.
According to your description, Start/Stop VMs during off-hours [Preview] solution in Automation is a good solution for you to save money.
The Start/Stop VMs during off-hours [Preview] solution starts and stops your Azure Resource Manager virtual machines on a user-defined schedule and provides insight into the success of the Automation jobs that start and stop your virtual machines with OMS Log Analytics.
It could send email notifications when the start and stop VM runbooks complete.
You also could use Azure automation to start or stop VM for a specific time, more information please refer to this link.
Update:
Your script works for me. I check your script, $RmVMs = (Get-AzureRmVM -ResourceGroupName $ResourceGroup -ErrorAction $ErrorActionPreference -WarningAction $WarningPreference).Name returns null. Do you have multiple subscriptions? It seems that your subscription ID is wrong. Please ensure your subscription ID is right. You could get your subscription ID on your local PowerShell.
Get-AzureRmSubscription
Update2:
You could save your save your subscription to automation connection. You could use the script below:
param (
[Parameter(Mandatory=$false)]
[String]$AzureCredentialAssetName
)
# Authenticate to Azure with certificate
Write-Verbose "Get connection asset: $ConnectionAssetName" -Verbose
$connectionName = Get-AutomationConnection -Name $AzureCredentialAssetName
try
{
# Get the connection "AzureRunAsConnection "
$servicePrincipalConnection=Get-AutomationConnection -Name $AzureCredentialAssetName
"Logging in to Azure..."
Add-AzureRmAccount `
-ServicePrincipal `
-TenantId $servicePrincipalConnection.TenantId `
-ApplicationId $servicePrincipalConnection.ApplicationId `
-CertificateThumbprint $servicePrincipalConnection.CertificateThumbprint
}
catch {
if (!$servicePrincipalConnection)
{
$ErrorMessage = "Connection $connectionName not found."
throw $ErrorMessage
} else{
Write-Error -Message $_.Exception
throw $_.Exception
}
}
$ResourceGroup = "shui2"
.....