Auto sql of azure SQL based on DTU consumption alert - azure

I have to scale my Azure SQL if DTU utilization is high at any moment and scale down if consumption is low for a certain duration.
I know this can be done by setting alert rule and web hook but looking for some redimate stuff which i can refer and utilize it....

Scale up and down takes time and sometimes it takes more than expected. It usually takes 5-10 minutes but sometimes it takes 25-30 minutes. The busier the database is at the time of the scale, the more time it takes to scale up or down. The size of the database also matters.
Remember also that transactions are rolled back when a scale is in progress.
My suggestion is to use the following query to identify patterns in resource consumption and then automate the scale up and down of the database.
SELECT *
FROM sys.dm_db_resource_stats
ORDER BY end_time
Tabulate that data on hours and days of the week to facilitate the pattern identification. After that use Azure Automation and parts of the following PowerShell to automate the scale up/down of the database. The following PowerShell does all what you want, it monitors DTU usage and then based on consumption triggers a scale process.
# Login-AzureRmAccount
# Set the resource group name and location for your server
$resourcegroupname = "myResourceGroup-$(Get-Random)"
$location = "southcentralus"
# Set an admin login and password for your server
$adminlogin = "ServerAdmin"
$password = "ChangeYourAdminPassword1"
# The logical server name has to be unique in the system
$servername = "server-$(Get-Random)"
# The sample database name
$databasename = "mySampleDatabase"
# The ip address range that you want to allow to access your server
$startip = "0.0.0.0"
$endip = "0.0.0.0"
# Create a new resource group
$resourcegroup = New-AzureRmResourceGroup -Name $resourcegroupname -Location $location
# Create a new server with a system wide unique server name
$server = New-AzureRmSqlServer -ResourceGroupName $resourcegroupname `
-ServerName $servername `
-Location $location `
-SqlAdministratorCredentials $(New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $adminlogin, $(ConvertTo-SecureString -String $password -AsPlainText -Force))
# Create a server firewall rule that allows access from the specified IP range
$serverfirewallrule = New-AzureRmSqlServerFirewallRule -ResourceGroupName $resourcegroupname `
-ServerName $servername `
-FirewallRuleName "AllowedIPs" -StartIpAddress $startip -EndIpAddress $endip
# Create a blank database with S0 performance level
$database = New-AzureRmSqlDatabase -ResourceGroupName $resourcegroupname `
-ServerName $servername `
-DatabaseName $databasename -RequestedServiceObjectiveName "S0"
# Monitor the DTU consumption on the imported database in 5 minute intervals
$MonitorParameters = #{
ResourceId = "/subscriptions/$($(Get-AzureRMContext).Subscription.Id)/resourceGroups/$resourcegroupname/providers/Microsoft.Sql/servers/$servername/databases/$databasename"
TimeGrain = [TimeSpan]::Parse("00:05:00")
MetricNames = "dtu_consumption_percent"
}
(Get-AzureRmMetric #MonitorParameters -DetailedOutput).MetricValues
# Scale the database performance to Standard S1
$database = Set-AzureRmSqlDatabase -ResourceGroupName $resourcegroupname `
-ServerName $servername `
-DatabaseName $databasename `
-Edition "Standard" `
-RequestedServiceObjectiveName "S1"
# Set an alert rule to automatically monitor DTU in the future
Add-AzureRMMetricAlertRule -ResourceGroup $resourcegroupname `
-Name "MySampleAlertRule" `
-Location $location `
-TargetResourceId "/subscriptions/$($(Get-AzureRMContext).Subscription.Id)/resourceGroups/$resourcegroupname/providers/Microsoft.Sql/servers/$servername/databases/$databasename" `
-MetricName "dtu_consumption_percent" `
-Operator "GreaterThan" `
-Threshold 90 `
-WindowSize $([TimeSpan]::Parse("00:05:00")) `
-TimeAggregationOperator "Average" `
-Actions $(New-AzureRmAlertRuleEmail -SendToServiceOwners)
# Clean up deployment
# Remove-AzureRmResourceGroup -ResourceGroupName $resourcegroupname

Related

Need help spiting Azure snapshots into another tenant/subscription

I have a script which snapshots all my disks in a certain RG.
However when I do the snapshotting, I need them to be spat out into another tenant/subscription for a migration project!
I've got as far as snapshotting everything and spitting them into a different RG but I need to take it a step further and spit them into the same named RG but in a different tenant/sub.
My script is below:
Login-AzureRmAccount -Credential $psCred –SubscriptionId $SubscriptionId -ErrorAction Stop | out-null
Connect-AzureRmAccount
Get-AzureRmSubscription -SubscriptionId $SubscriptionId | Select-AzureRmSubscription
$tagResList = Get-AzureRmResource -TagName Environment -TagValue Staging
#$tagResList = Find-AzureRmResource -ResourceGroupNameEquals testrs
#$tagRsList[0].ResourceId.Split("//")
#subscriptions
#<SubscriptionId>
#resourceGroups
#<ResourceGroupName>
#providers
#Microsoft.Compute
#virtualMachines
#<vmName>
foreach($tagRes in $tagResList) {
if($tagRes.ResourceId -match "Microsoft.Compute")
{
$vmInfo = Get-AzureRmVM sandbox207478603000 #$tagRes.ResourceId.Split("//")[4] -Name $tagRes.ResourceId.Split("//")[8]
#Set local variables
$location = $vmInfo.Location
$resourceGroupName = $vmInfo.ResourceGroupName
$timestamp = Get-Date -f MM-dd-yyyy_HH_mm_ss
#Snapshot name of OS data disk
$snapshotName = $vmInfo.Name + $timestamp
#Create snapshot configuration
$snapshot = New-AzureRmSnapshotConfig -SourceUri $vmInfo.StorageProfile.OsDisk.ManagedDisk.Id -Location $location -CreateOption copy
#Take snapshot
New-AzureRmSnapshot -Snapshot $snapshot -SnapshotName $snapshotName snapshots $resourceGroupName
if($vmInfo.StorageProfile.DataDisks.Count -ge 1){
#Condition with more than one data disks
for($i=0; $i -le $vmInfo.StorageProfile.DataDisks.Count - 1; $i++){
#Snapshot name of OS data disk
$snapshotName = $vmInfo.StorageProfile.DataDisks[$i].Name + $timestamp
#Create snapshot configuration
$snapshot = New-AzureRmSnapshotConfig -SourceUri $vmInfo.StorageProfile.DataDisks[$i].ManagedDisk.Id -Location $location -CreateOption copy
#Take snapshot
New-AzureRmSnapshot -Snapshot $snapshot -SnapshotName $snapshotName snapshots $ResourceGroupName
}
}
else{
Write-Host $vmInfo.Name + " doesn't have any additional data disk."
}
}
else{
$tagRes.ResourceId + " is not a compute instance"
}
}
$tagRgList = Get-AzureRmResourceGroup -Tag #{ Environment = "Staging" }
I am not sure if you can save snapshot in another tenant in one command as you'd need to be authenticated there.
I would suggest using azcopy tool to move snapshot files between storage accounts
#######################################
Reviewed your comment and found that indeed you can't use azcopy on vm images.
But you may create access to the snapshot
#Generate the SAS for the snapshot
$sas = Grant-AzSnapshotAccess -ResourceGroupName $ResourceGroupName -SnapshotName $SnapshotName -DurationInSecond $sasExpiryDuration -Access Read
and save it to the destination storage account:
#Copy the snapshot to the storage account
Start-AzStorageBlobCopy -AbsoluteUri $sas.AccessSAS -DestContainer $storageContainerName -DestContext $destinationContext -DestBlob $destinationVHDFileName
More details canbe found here

The Azure PowerShell session has not been properly initialized. Please import the module and try again

I'm writing my first powershell script to load data from a CSV to an Azure Storage table. I'm not sure why the line
$storageAccountKey = (Get-AzStorageAccountKey -ResourceGroupName $resourceGroupName -Name $storageAccountName)[0].Value
is throwing an error:
Running Get-Module gives this result:
This is a snippet of the code that I have written till now:
# Step 1, Set variables
# Enter Table Storage location data
$resourceGroupName = "ComputeTesting"
$storageAccountName = 'computetestingdiag'
$tableName = 'strtable'
$dateTime = get-date
# Step 2, Login to your Azure subscription
$sub = Get-AzSubscription -ErrorAction SilentlyContinue
if(-not($sub))
{
Connect-AzAccount
}
# If you have multiple subscriptions, set the one to use
# Select-AzSubscription -SubscriptionId "<SUBSCRIPTIONID>"
# Step 3, Get the access key for the Azure Storage account
$storageAccountKey = (Get-AzStorageAccountKey -ResourceGroupName $resourceGroupName -Name $storageAccountName)[0].Value
# Step 4, Connect to Azure Table Storage
$storageCtx = New-AzureStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $storageAccountKey
$table = Get-AzureStorageTable -Name $tableName -Context $storageCtx
I checked some of the similar questions and what I understand that uninstalling and re-installing the Azure modules might help. Although I didn't try this yet, is there any other workaround for this? Any help whatsoever would be highly helpful.
According to the script you provided, you use the Az and AzureRM modules at the same PowerShell session. It may cause conflicts. I suggest you use the one module in one session.
For example
$resourceGroupName = "<>"
$storageAccountName = '<>'
$tableName = '<>'
Connect-AzAccount
$storageAccountKey = (Get-AzStorageAccountKey -ResourceGroupName $resourceGroupName -Name $storageAccountName)[0].Value
$storageCtx = New-AzStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $storageAccountKey
$table = Get-AzStorageTable -Name $tableName -Context $storageCtx
For more details about how to manage Azure table storage, please refer to the document

How to set value of -DefaultProfile (of type IAzureContextContainer) in New-AzSqlSyncMember

I am trying to create a powershell script to create azure data sync between 2 azure SQL databases.
My member database is on another subscription.
I need to set -DefaultProfile which is of type on 'New-AzSqlSyncMember' command.
I am not aware of the syntax for setting this parameter.
My current script without -DefaultProfile looks like below:
New-AzSqlSyncMember -ResourceGroupName $resourceGroupName `
-ServerName $serverName `
-DatabaseName $databaseName `
-SyncGroupName $syncGroupName `
-Name $syncMemberName `
-MemberDatabaseType $memberDatabaseType `
-SyncDirection $syncDirection
I want to set the value of the subscription field using powershell like in the image below using powershell:
Possible cross post from https://social.msdn.microsoft.com/Forums/en-US/4ad3dd3e-314a-4442-957f-da77c17ef85b/how-to-set-value-of-defaultprofile-of-type-iazurecontextcontainer-in-newazsqlsyncmember?forum=azurescripting&prof=required
You want to call Connect-AzAccount with the credentials for the account you want to use in the -DefaultProfile parameter and store that in a variable. You can use that variable to set the parameter:
$DefaultProfile = Connect-AzAccount <params> -SubscriptionId $SubscriptionId
New-AzSqlSyncMember <params> -DefaultContext $DefaultProfile
If this throws a type error that it can't convert from a PSAzureContext to a IAzureContextContainer there is an explicit converter available.
$DefaultProfile = Connect-AzAccount <params> -SubscriptionId $SubscriptionId
$Converter = New-Object -TypeName Microsoft.Azure.Commands.Profile.Models.AzureContextConverter
$Container = $converter.ConvertFrom($DefaultProfile, [Microsoft.Azure.Commands.Common.Authentication.Abstractions.Core.IAzureContextContainer], $null, $true)
New-AzSqlSyncMember <params> -DefaultContext $Container

Azure Powershell Scripting parameters

Good Day
I have been tasked to gather a list of all the VMs in our different subscriptions, and I have a script that gathers: Mode, Name, ResourceGroupName, location, VMSize, and Status. There are two empty columns: subscription and availability set.
A bit of a backstory, the company I am contracted to hired a vendor to deploy the resources and now they don't know if the vendor did what they were suppose to.
What I am trying to figure out:
How to list the ASG the resource is in
The NSG the resource is in
The vCPUs and RAM
Storage account
I have been scouring google to find the PSObjects I need to define. Any assistance would be greatly appreciated
1.How to list the ASG the resource is in
If you want to list all the ASGs in the VM, you need to list them via all the NICs, you could refer to the command below, the $asgnames is the list of all ASGs, note if different NICs have the same ASG, it will output several times.
$nics = (Get-AzureRmVM -ResourceGroupName "<ResourceGroupName>" -Name "<VM Name>").NetworkProfile.NetworkInterfaces
$nicnames = #()
$asgnames = #()
foreach($nic in $nics){
$a = $nic.Id -split"/"
$nicname = $a[8]
$nicnames += $nicname
$ipconfig = Get-AzureRmResource -ResourceGroupName <ResourceGroupName> -ResourceType Microsoft.Network/networkInterfaces/ipConfigurations -ResourceName "$nicname/ipconfig1" -ApiVersion 2018-07-01
$asgids = $ipconfig.Properties.applicationSecurityGroups.id
foreach ($asgid in $asgids){
$a = $asgid -split"/"
$asganme = $a[8]
$asgnames += $asganme
}
}
2.The NSG the resource is in
If you want to get NSGs in the VM, you need to get them via all NICs and VNET Subnet.
Get the NSG of the NIC, the $nsgnic is the NSG:
$nic = Get-AzureRmResource -ResourceGroupName <ResourceGroupName> -ResourceType Microsoft.Network/networkInterfaces -ResourceName "<NIC name>" -ApiVersion 2018-07-01
$a = $nic.properties.networkSecurityGroup.id -split"/"
$nsgnic = $a[8]
Get the NSG of subnet, the $nsgsub is the NSG:
$subnet = Get-AzureRmResource -ResourceGroupName <ResourceGroupName> -ResourceType Microsoft.Network/virtualNetworks/subnets -ResourceName "<Vnet name>/<Subnet name>" -ApiVersion 2018-07-01
$b = $subnet.properties.networkSecurityGroup.id -split"/"
$nsgsub = $a[8]
Then the NSGs of the VM will be the sum of $nsgnic and $nsgsub.
3.The vCPUs and RAM
Try the command below, NumberOfCores is the vCPUs , MemoryInMB is the RAM, note the RAM in result is in MB, you could convert it to GB if necessary.
Get-AzureRmVMSize -ResourceGroupName "<ResourceGroupName>" -VMName "<VMName>"
4.Storage account
If I do not misunderstand, the storage account here you mean the Diagnostics storage account, try the command below, $storage is the storage account uri.
$a = Get-AzureRmResource -ResourceGroupName <ResourceGroupName> -ResourceType Microsoft.Compute/virtualMachines -ResourceName "<VM Name>" -ApiVersion 2018-06-01
$storage = $a.properties.diagnosticsProfile.bootDiagnostics.storageUri

How to move an azure VM image to a different location

I have an azure VM image with a managed disk in east US and I want to move/copy it to West Europe.
Is there an easy way to do it?
I saw that there's an azure cli extension called az-image-copy but it doesn't work for me because it outputs an error which says that it can't find the OS disk (even though the resource ID is correct and I can see it in the azure portal)
ERROR: Resource ServerLinux_OsDisk_1_c208679747734937b10a1525aa84a7d7 is not found
So is there any other way to do it?
You could use azure powershell to copy the managed images, create a snapshot and copy it to anther region, then create the image. Here is a similar issue.
Create a snapshot:
<# -- Create a snapshot of the OS (and optionally data disks) from the generalized VM -- #>
$vm = Get-AzureRmVM -ResourceGroupName $resourceGroupName -Name $vmName
$disk = Get-AzureRmDisk -ResourceGroupName $resourceGroupName -DiskName $vm.StorageProfile.OsDisk.Name
$snapshot = New-AzureRmSnapshotConfig -SourceUri $disk.Id -CreateOption Copy -Location $region
$snapshotName = $imageName + "-" + $region + "-snap"
New-AzureRmSnapshot -ResourceGroupName $resourceGroupName -Snapshot $snapshot -SnapshotName $snapshotName
Copy the snapshot:
# Create the name of the snapshot, using the current region in the name.
$snapshotName = $imageName + "-" + $region + "-snap"
# Get the source snapshot
$snap = Get-AzureRmSnapshot -ResourceGroupName $resourceGroupName -SnapshotName $snapshotName
# Create a Shared Access Signature (SAS) for the source snapshot
$snapSasUrl = Grant-AzureRmSnapshotAccess -ResourceGroupName $resourceGroupName -SnapshotName $snapshotName -DurationInSecond 3600 -Access Read
# Set up the target storage account in the other region
$targetStorageContext = (Get-AzureRmStorageAccount -ResourceGroupName $resourceGroupName -Name $storageAccountName).Context
New-AzureStorageContainer -Name $imageContainerName -Context $targetStorageContext -Permission Container
# Use the SAS URL to copy the blob to the target storage account (and thus region)
Start-AzureStorageBlobCopy -AbsoluteUri $snapSasUrl.AccessSAS -DestContainer $imageContainerName -DestContext $targetStorageContext -DestBlob $imageBlobName
Get-AzureStorageBlobCopyState -Container $imageContainerName -Blob $imageBlobName -Context $targetStorageContext -WaitForComplete
# Get the full URI to the blob
$osDiskVhdUri = ($targetStorageContext.BlobEndPoint + $imageContainerName + "/" + $imageBlobName)
# Build up the snapshot configuration, using the target storage account's resource ID
$snapshotConfig = New-AzureRmSnapshotConfig -AccountType StandardLRS `
-OsType Windows `
-Location $targetRegionName `
-CreateOption Import `
-SourceUri $osDiskVhdUri `
-StorageAccountId "/subscriptions/${sourceSubscriptionId}/resourceGroups/${resourceGroupName}/providers/Microsoft.Storage/storageAccounts/${storageAccountName}"
# Create the new snapshot in the target region
$snapshotName = $imageName + "-" + $targetRegionName + "-snap"
$snap2 = New-AzureRmSnapshot -ResourceGroupName $resourceGroupName -SnapshotName $snapshotName -Snapshot $snapshotConfig
Create the image:
<# -- In the second subscription, create a new Image from the copied snapshot --#>
Select-AzureRmSubscription -SubscriptionId $targetSubscriptionId
$snap = Get-AzureRmSnapshot -ResourceGroupName $resourceGroupName -SnapshotName $snapshotName
$imageConfig = New-AzureRmImageConfig -Location $destinationRegion
Set-AzureRmImageOsDisk -Image $imageConfig `
-OsType Windows `
-OsState Generalized `
-SnapshotId $snap.Id
New-AzureRmImage -ResourceGroupName $resourceGroupName `
-ImageName $imageName `
-Image $imageConfig
For more details, refer to this link.
If the resources are under the same subscription then you can get move resources to from the Resource Group 01 (East Us) to the Resource Group 02 (West Europe).
For your help you can check these documents:
Move Managed Disks and VMs
Move resources to new resource group or subscription
Copy managed disks in the same subscription or different subscription with PowerShell
Migrating Azure IaaS solutions using MigAz
The last link is about the MigAz tool, which allow to migrate resources on Azure.

Resources