I am trying to add subscription to storage account using New-AzEventGridSubscription. The subscription should be triggered by blob modification in a container and put message to a certain queue. I created the following script:
$ResourceGroup = "test"
$includedEventTypes = "Microsoft.Storage.BlobCreated", "Microsoft.Storage.BlobDeleted"
New-AzEventGridSubscription `
-ResourceId "/subscriptions/[id]/resourceGroups/[group]/providers/Microsoft.Storage/storageAccounts/[name]" `
-EventSubscriptionName DummyName `
-Endpoint "/subscriptions/[id]/resourceGroups/[group]/providers/Microsoft.Storage/storageAccounts/[name]/queueServices/default/queues/my-queue" `
-ResourceGroup $ResourceGroup `
-EndpointType "storagequeue" `
-SubjectBeginsWith "prefix" `
-SubjectEndsWith "suffix"
but it throws an error:
New-AzEventGridSubscription : Parameter set cannot be resolved using the specified named parameters.
At line:1 char:1
+ New-AzEventGridSubscription `
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (:) [New-AzEventGridSubscription], ParameterBindingException
+ FullyQualifiedErrorId : AmbiguousParameterSet,Microsoft.Azure.Commands.EventGrid.NewAzureEventGridSubscription
I made similar command to add subscription to a custom topic, which worked well:
New-AzEventGridSubscription `
-ResourceGroup $ResourceGroup `
-EventSubscriptionName SubscriptionName `
-TopicName MyCustomTopic `
-EndpointType "storagequeue" `
-Endpoint "/subscriptions/[id]/resourceGroups/[group]/providers/Microsoft.Storage/storageAccounts/[account]/queueServices/default/queues/my-queue" `
-SubjectBeginsWith "prefix" `
-SubjectEndsWith "suffix"
I tried several modifications, but to no avail. What am I doing wrong?
After throwing out "ResourceGroup" parameter, reordering other parameters, and putting some arguments into variables, I finally managed to get my script up and running... If anyone ever needs similar stuff, here it is:
$subscriptionId = "<id>"
$ResourceGroup = "<group>"
$includedEventTypes = "Microsoft.Storage.BlobCreated", "Microsoft.Storage.BlobDeleted"
$storageAccount = "<account name>"
$endpoint = "/subscriptions/"+$subscriptionId+"/resourceGroups/"+$ResourceGroup+"/providers/Microsoft.Storage/storageAccounts/"+$storageAccount+"/queueServices/default/queues/my-queue"
$resourceId = "/subscriptions/"+$subscriptionId+"/resourceGroups/"+$ResourceGroup+"/providers/Microsoft.Storage/storageAccounts/"+$storageAccount
$subject = "prefix"
New-AzEventGridSubscription `
-EventSubscriptionName MySubscriptionName `
-Endpoint $endpoint `
-ResourceId $resourceId `
-EndpointType "storagequeue" `
-SubjectBeginsWith $subject `
-SubjectEndsWith "suffix" `
-IncludedEventType $includedEventTypes
Related
I am trying to figure out how to restore a database from one managed SQL instance to another. I'm following the tutorials, but I keep running into inscrutable error messages.
Here's my command:
Restore-AzSqlInstanceDatabase `
-Name "SomeDatabase" `
-InstanceName "our-oltp-dev" `
-ResourceGroupName "dev-managedsqlinstances" `
-PointInTime "4/7/2020 12:00:00" `
-TargetInstanceDatabaseName "SomeDatabase_FROM_DEV" `
-TargetInstanceName "our-oltp-sandbox" `
-TargetResourceGroupName "sandbox-managedsqlinstances"
Here's the output:
PS C:\WINDOWS\system32> Restore-AzSqlInstanceDatabase `
-Name "SomeDatabase" `
-InstanceName "our-oltp-dev" `
-ResourceGroupName "dev-managedsqlinstances" `
-PointInTime "4/7/2020 12:00:00" `
-TargetInstanceDatabaseName "SomeDatabase_FROM_DEV" `
-TargetInstanceName "our-oltp-sandbox" `
-TargetResourceGroupName "sandbox-managedsqlinstances"
Restore-AzSqlInstanceDatabase : Parameter set cannot be resolved using the specified named parameters.
At line:1 char:1
+ Restore-AzSqlInstanceDatabase `
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (:) [Restore-AzSqlInstanceDatabase], ParameterBindingException
+ FullyQualifiedErrorId : AmbiguousParameterSet,Microsoft.Azure.Commands.Sql.ManagedDatabase.Cmdlet.RestoreAzureRmSqlManagedDatabase
It's a copy-and-paste from the Azure docs; so I'm not sure what I'm doing wrong. Any help would be appreciated.
As mentioned in the comment, you need to pass the PointInTime as a DateTime instead of String, you could specify the 4/7/2020 12:00:00 as below.
Sample:
$PointInTime = Get-Date -Year 2020 -Month 4 -Day 7 -Hour 12 -Minute 0 -Second 0
Restore-AzSqlInstanceDatabase `
-Name "SomeDatabase" `
-InstanceName "our-oltp-dev" `
-ResourceGroupName "dev-managedsqlinstances" `
-PointInTime $PointInTime `
-TargetInstanceDatabaseName "SomeDatabase_FROM_DEV" `
-TargetInstanceName "our-oltp-sandbox" `
-TargetResourceGroupName "sandbox-managedsqlinstances"
Attempting to add an extension when not detected but keep failing to find the secret sauce to get this to work. Mind you I am a BASH guy and this is a first foray into PowerShell..
#requires -version 2
# Required parameter $subscription: name of the subscription to enable Custom Script Extensions in
param (
# NOTE: See below for reason...
# [Parameter(Mandatory = $true)] [String] $subscription
# NOTE: Prompting is great for using the script interactively, but if this will also be executed
# from a build server or ...
# NOTE: Once the parameter is marked as mandatory PowerShell it will prompt for value. That said,
# if you remove the mandatory attribute then you can set a default value as a T_THROW ...
# NOTE: This _does_ contain shortcomings if this will be used as a pipeline param ...
# https://stackoverflow.com/questions/33600279/is-it-possible-to-force-powershell-script-to-throw-if-a-required-pipeline-para
[Parameter()]
[ValidateNotNullOrEmpty()]
[String]$SubscriptionName=$(Throw "`SubscriptionName` is mandatory, please provide a value...")
)
# Connect to the current Azure account
Write-Output "Pulling Azure account credentials..."
Start-Process "https://microsoft.com/devicelogin" # steals focus...
# Login to Azure account
Connect-AzAccount
# Set the active subscription
$null = Get-AzSubscription -SubscriptionName "$SubscriptionName" |Set-AzContext
# TODO: error handling
$vms = Get-AzVM
$cseName = "VulnerabilityManagementTools"
ForEach ($vm in $vms) {
try {
$cseStatus = Get-AzVMCustomScriptExtension `
-ResourceGroupName $vm.ResourceGroupName `
-VMName $vm.Name `
-Name $cseName `
-Status
}
catch {
Write-Output "Enabling Custom Script Extension for $vm."
Set-AzVMCustomScriptExtension `
-ResourceGroupName $vm.ResourceGroup `
-Location $vm.Location `
-VMName $vm.Name `
-Name $cseName `
-TypeHandlerVersion "1.1" `
-StorageAccountName "VulnerabilityManagementTools" `
-FileName "VulnerabilityManagementInstaller.ps1" `
-ContainerName "VulnerabilityManagementTools"
}
}
End up err'ing out with
PS /.../automation-scripts> ./EnableCustomScriptExtension.ps1 SubscriptionName
Pulling Azure account credentials...
WARNING: To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code XXXXXX to authenticate.
Account SubscriptionName TenantId Environment
------- ---------------- -------- -----------
XXXX#analytics.com SubName XXXXXX-XXXX AzureCloud
Get-AzVMCustomScriptExtension : The Resource 'Microsoft.Compute/virtualMachines/XXXX/extensions/VulnerabilityManagementTools' under resource group '{NAME}' was not found.
ErrorCode: ResourceNotFound
ErrorMessage: The Resource 'Microsoft.Compute/virtualMachines/XXXX/extensions/VulnerabilityManagementTools' under resource group '{NAME}' was not found.
ErrorTarget:
StatusCode: 404
ReasonPhrase: Not Found
At /.../automation-scripts/EnableCustomScriptExtension.ps1:59 char:18
+ $cseStatus = Get-AzVMCustomScriptExtension `
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : CloseError: (:) [Get-AzVMCustomScriptExtension], ComputeCloudException
+ FullyQualifiedErrorId : Microsoft.Azure.Commands.Compute.GetAzureVMCustomScriptExtensionCommand
Get-AzVMCustomScriptExtension : The Resource 'Microsoft.Compute/virtualMachines/XXXXX/extensions/VulnerabilityManagementTools' under resource group '{RESOURCE_GROUPNAME}' was not found.
ErrorCode: ResourceNotFound
ErrorMessage: The Resource 'Microsoft.Compute/virtualMachines/XXXX/extensions/VulnerabilityManagementTools' under resource group '{RESOURCE_GROUPNAME}' was not found.
ErrorTarget:
StatusCode: 404
ReasonPhrase: Not Found
At /.../automation-scripts/EnableCustomScriptExtension.ps1:59 char:18
+ $cseStatus = Get-AzVMCustomScriptExtension `
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : CloseError: (:) [Get-AzVMCustomScriptExtension], ComputeCloudException
+ FullyQualifiedErrorId : Microsoft.Azure.Commands.Compute.GetAzureVMCustomScriptExtensionCommand
Get-AzVMCustomScriptExtension : The Resource 'Microsoft.Compute/virtualMachines/{VMName}/extensions/VulnerabilityManagementTools' under resource group '{RESOURCEX_GROUPNAME}' was not found.
ErrorCode: ResourceNotFound
ErrorMessage: The Resource 'Microsoft.Compute/virtualMachines/{VMName}/extensions/VulnerabilityManagementTools' under resource group '{RESOURCEX_GROUPNAME}' was not found.
ErrorTarget:
StatusCode: 404
ReasonPhrase: Not Found
At /.../automation-scripts/EnableCustomScriptExtension.ps1:59 char:18
+ $cseStatus = Get-AzVMCustomScriptExtension `
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : CloseError: (:) [Get-AzVMCustomScriptExtension], ComputeCloudException
+ FullyQualifiedErrorId : Microsoft.Azure.Commands.Compute.GetAzureVMCustomScriptExtensionCommand`
In your case, you just need to use the if(){}else{} statement, try the script as below instead of the ForEach part of yours, it works fine on my side.
ForEach ($vm in $vms) {
$cseStatus = Get-AzVMCustomScriptExtension `
-ResourceGroupName $vm.ResourceGroupName `
-VMName $vm.Name `
-Name $cseName `
-Status `
-ErrorAction SilentlyContinue
if ($cseStatus){
Write-Host "The extension has been set for" $vm.Name
}else{
Write-Host "Enabling Custom Script Extension for" $vm.Name
Set-AzVMCustomScriptExtension `
-ResourceGroupName $vm.ResourceGroup `
-Location $vm.Location `
-VMName $vm.Name `
-Name $cseName `
-TypeHandlerVersion "1.1" `
-StorageAccountName "VulnerabilityManagementTools" `
-FileName "VulnerabilityManagementInstaller.ps1" `
-ContainerName "VulnerabilityManagementTools"
}
}
Test result:
You'll need to create an Azure AD Service Principal using password authentication and use the credentials of this to pass to the Connect-AzAccount cmdlet as follows:
$credentials = Get-Credential
Connect-AzAccount -ServicePrincipal -Credentials $credentials
The service account will need to have the necessary permissions to use the Set-AzVMCustomScriptExtensions cmdlet.
More information on creating the service account here:
https://learn.microsoft.com/en-us/powershell/azure/create-azure-service-principal-azureps?view=azps-2.8.0
I have a script to handle some parameters and deploy an arm template with those parameters, and then create a database using a bacpac file in my storage account. When my script is trying to invoke the cmdlet "New-AzSqlDatabaseImport" i'm getting "NotFound: Entity not found to invoke import"
I have trying updating all of my Az modules, and tried to run the cmdlet manually and fill in the parameters manually.
Write-Host "Creating database and importing data"
$bloblink = (Get-AzStorageAccount -ResourceGroupName $ResourceGroup -Name $StorageName).PrimaryEndpoints.Blob
$uri = $bloblink + $containerName + "/" + $bacpacFile
$importRequest = New-AzSqlDatabaseImport `
-ResourceGroupName $ResourceGroup `
-ServerName $SQLServer `
-DatabaseName $database_ql_name `
-DatabaseMaxSizeBytes $databaseMaxSizeBytes `
-StorageKeyType "StorageAccessKey" `
-StorageKey $(Get-AzStorageAccountKey -ResourceGroupName $ResourceGroup -StorageAccountName $StorageName).Value[0] `
-StorageUri $uri `
-Edition $DatabaseEdition `
-ServiceObjectiveName $DatabaseSize `
-AdministratorLogin $dbUsername `
-AdministratorLoginPassword $dbPassword
I expect it to start a job on Azure to create the database and start the import job.
This is the error im getting instead (i replaced the full path).
At *PATH*/Deployment.ps1:262 char:18
+ $importRequest = New-AzSqlDatabaseImport `
+ ~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : CloseError: (:) [New-AzSqlDatabaseImport], CloudException
+ FullyQualifiedErrorId : Microsoft.Azure.Commands.Sql.ImportExport.Cmdlet.NewAzureSqlDatabaseImport
I just encountered the same error. I found that the -ServerName argument is case-sensitive. So check your $SQLServer-variable.
I am using this code found on stack , and all connections are correct.
Import-Module Azure
Import-Module Azure.Storage
Get-AzureRmSubscription –SubscriptionName “Production” | Select-AzureRmSubscription
# Username for Azure SQL Database server
$ServerLogin = "username"
# Password for Azure SQL Database server
$serverPassword = ConvertTo-SecureString "abcd" -AsPlainText -Force
# Establish credentials for Azure SQL Database Server
$ServerCredential = new-object System.Management.Automation.PSCredential($ServerLogin, $serverPassword)
# Create connection context for Azure SQL Database server
$SqlContext = New-AzureSqlDatabaseServerContext -FullyQualifiedServerName “myspecialsqlserver.database.windows.net” -Credential $ServerCredential
$StorageContext = New-AzureStorageContext -StorageAccountName 'prodwad' -StorageAccountKey 'xxxxx'
$Container = Get-AzureStorageContainer -Name 'automateddbbackups' -Context $StorageContext
$exportRequest = Start-AzureSqlDatabaseExport -SqlConnectionContext $SqlContext -StorageContainer $Container -DatabaseName 'Users' -BlobName 'autobackupotest.bacpac' -Verbose -Debug
I am getting this error. I have spent hours on this.
Start-AzureSqlDatabaseExport : Cannot bind parameter 'StorageContainer'. Cannot convert the
"Microsoft.WindowsAzure.Commands.Common.Storage.ResourceModel.AzureStorageContainer" value of type
"Microsoft.WindowsAzure.Commands.Common.Storage.ResourceModel.AzureStorageContainer" to type
"Microsoft.WindowsAzure.Commands.Common.Storage.ResourceModel.AzureStorageContainer".
At line:31 char:99
+ ... SqlConnectionContext $SqlContext -StorageContainer $Container -Databa ...
+ ~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (:) [Start-AzureSqlDatabaseExport], ParameterBindingException
+ FullyQualifiedErrorId : CannotConvertArgumentNoMessage,Microsoft.WindowsAzure.Commands.SqlDatabase.Database.Cmdlet.StartAzureSqlDatabaseExport
I am using AzureRM 3.8.0
According to your description and codes, If you want to start a new sqldatabase export.I suggest you could try below codes. It will work well.
$subscriptionId = "YOUR AZURE SUBSCRIPTION ID"
Login-AzureRmAccount
Set-AzureRmContext -SubscriptionId $subscriptionId
# Database to export
$DatabaseName = "DATABASE-NAME"
$ResourceGroupName = "RESOURCE-GROUP-NAME"
$ServerName = "SERVER-NAME"
$serverAdmin = "ADMIN-NAME"
$serverPassword = "ADMIN-PASSWORD"
$securePassword = ConvertTo-SecureString -String $serverPassword -AsPlainText -Force
$creds = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $serverAdmin, $securePassword
# Generate a unique filename for the BACPAC
$bacpacFilename = $DatabaseName + (Get-Date).ToString("yyyyMMddHHmm") + ".bacpac"
# Storage account info for the BACPAC
$BaseStorageUri = "https://STORAGE-NAME.blob.core.windows.net/BLOB-CONTAINER-NAME/"
$BacpacUri = $BaseStorageUri + $bacpacFilename
$StorageKeytype = "StorageAccessKey"
$StorageKey = "YOUR STORAGE KEY"
$exportRequest = New-AzureRmSqlDatabaseExport -ResourceGroupName $ResourceGroupName -ServerName $ServerName `
-DatabaseName $DatabaseName -StorageKeytype $StorageKeytype -StorageKey $StorageKey -StorageUri $BacpacUri `
-AdministratorLogin $creds.UserName -AdministratorLoginPassword $creds.Password
$exportRequest
# Check status of the export
Get-AzureRmSqlDatabaseImportExportStatus -OperationStatusLink $exportRequest.OperationStatusLink
Then you could use Get-AzureRmSqlDatabaseImportExportStatus to see the details information, like below:
I got the same problem after updating some powershell packages which I do not remember exactly. After the update my scripts started to fail.
My solution is :
Install the latest AzureRM from nuget via powershell
Use the other overload of Start-AzureSqlDatabaseExport which utilizes the parameters
-StorageContainerName and -StorageContext rather than -StorageContainer
It looks like if you pass the parameters to the function it will create the container object internally without crashing.
I get error when I try to create Azure HDInsight cluster using powershell cmdlet:
New-AzureRmHDInsightClusterConfig `
| Add-AzureRmHDInsightMetastore `
-SqlAzureServerName "$sqlDatabaseServerName.database.windows.net" `
-DatabaseName $hiveMetaStoreDBName `
-Credential $sqlServerCred `
-MetastoreType HiveMetaStore `
| New-AzureRmHDInsightCluster `
-ResourceGroupName $resourceGroupName `
-HttpCredential $clusterCreds `
-ClusterName $clusterName `
-Location $location `
-ClusterType $clusterType `
-OSType $OSType `
-Version "$hdVersion" `
-SshCredential $clusterCreds `
-DefaultStorageAccountName "$storageAccountName.blob.core.windows.net" `
-DefaultStorageAccountKey $storageAccountKey `
-ClusterSizeInNodes $clusterNodes
Looks like parameters are not recognized by powershell because it asks to input them (see below). I input required parameters (Location, ClusterName, ClusterSizeInNodes) and then error occurs.
cmdlet New-AzureRmHDInsightCluster at command pipeline position 3
Supply values for the following parameters:
(Type !? for Help.)
Location: West Europe
ClusterName: xxxxxxxxx
ClusterSizeInNodes: 1
New-AzureRmHDInsightCluster : BadRequest: ParameterNullOrEmpty,Parameter 'ASVAccount.AccountName' cannot be null or
empty.
At line:117 char:11
+ | New-AzureRmHDInsightCluster `
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [New-AzureRmHDInsightCluster], CloudException
+ FullyQualifiedErrorId : Hyak.Common.CloudException,Microsoft.Azure.Commands.HDInsight.NewAzureHDInsightClusterCom
mand
Does somebody know why its happens or what is wrong in smdlet?
From the error message, it seems like your $storageAccountName parameter to the New-AzureRmHDInsightCluster cmdlet is Null or Empty, you may want to inspect further on this.
Besides, I would also strongly recommend you to specify the -DefaultStorageContainer to the New-AzureRmHDInsightCluster cmdlet as well. This will ensures that the cmdlet will be able to resolve the FQDN of your storage account Uri.
E.g. asv://YourDefaultContainer#YourDefaultStorageAccountName.blob.core.windows.net/
Hope this helps!
Use below command for Cluster with Hive Metastore.
Here is a working PowerShell script, to be used with Azure ARM PowerShell, 1.0.1 or later – you can install Azure RM PS via web platform installer or follow this blog https://azure.microsoft.com/en-us/blog/azps-1-0/
Add-AzureRmAccount
$MyClusterName = "clustername";
$MyClusterLocation = "East US 2";
$NumClusterNodes = 2;
$MyClusterVersion = "3.2";
$MyHDInsightUserName = ""
$MyHDInsightPwd = ""
$MySqlAzureUserName = ""
$MySqlAzurePwd = ""
$MySqlAzureServerName = "*.database.windows.net"
$MySqlAzureDbName = "Dbtest"
$MyDefaultContainerName = "tastoreps"
$clusterResourceGroupName = "dirg"
# Use the correct Azure Subscription!
$subid = ""
Select-AzureRmSubscription -SubscriptionId $subid
# Storage key
$primaryStorageAcctName = "toragesouth"
$primaryStorageResourceGroupName = "storagerg"
# you need to use an ARM based storage as the primary account , classic storage won’t work as a primary account, known issue
$storageAccountKey = Get-AzureRmStorageAccountKey -ResourceGroupName $primaryStorageResourceGroupName -Name $primaryStorageAcctName | %{ $_.Key1 }
# credentials
$HdInsightPwd = ConvertTo-SecureString $MyHDInsightPwd -AsPlainText -Force
$HdInsightCreds = New-Object System.Management.Automation.PSCredential ($MyHDInsightUserName, $HdInsightPwd)
$SqlAzurePwd = ConvertTo-SecureString $MySqlAzurePwd -AsPlainText -Force
$SqlAzureCreds = New-Object System.Management.Automation.PSCredential ($MySqlAzureUserName, $SqlAzurePwd)
$config = New-AzureRmHDInsightClusterConfig -ClusterType Hadoop |
Add-AzureRmHDInsightMetastore -SqlAzureServerName $MySqlAzureServerName -DatabaseName $MySqlAzureDbName -Credential $SqlAzureCreds -MetastoreType HiveMetastore |
Add-AzureRmHDInsightMetastore -SqlAzureServerName $MySqlAzureServerName -DatabaseName $MySqlAzureDbName -Credential $SqlAzureCreds -MetastoreType OozieMetastore
$config.DefaultStorageAccountName="$StorageAcctName.blob.core.windows.net"
$config.DefaultStorageAccountKey=$storageAccountKey
#create cluster
New-AzureRmHDInsightCluster -config $config -OSType Windows -clustername $MyClusterName -HttpCredential $HdInsightCreds -DefaultStorageContainer $MyDefaultContainerName -Location $MyClusterLocation -ResourceGroupName $clusterResourceGroupName -ClusterSizeInNodes $NumClusterNodes -Version $MyClusterVersion