Powershell Runbook [Invoke-ASCmd], FileNotFoundException for xmla file - azure

I'm trying to make a script that creates automatic partions for SSAS via Powershell Runbook, but whenever I try to read in the xmla file i get the following error:
My code that calls this is as followed:
$StorageAccount = Get-AzureRmStorageAccount -ResourceGroupName $ResourceGroupName -Name $StorageAccountName
$blob = Get-AzureStorageBlob -Context $StorageAccount.Context -Container "Database name" -Blob "CreateNewPartition.xmla"
$file = $blob.ICloudBlob.DownloadText()
Invoke-ASCmd `
-Database $AnalysisServiceDatabase `
-InputFile $file `
-server $AnalysisServiceServer
When using the following code:
$memStream = New-Object System.IO.MemoryStream
$blob.ICloudBlob.DownloadToStream($memStream)
$readStream = New-Object System.IO.StreamReader($memStream, [System.Text.Encoding]::Unicode)
$memStream.Position = 0
$file = ($readStream.ReadToEnd() -replace "`0",'' | ConvertFrom-Json)
I get this error:
And when trying this code:
$byteArray = New-Object Byte[] $blob.Length
$file = $blob.ICloudBlob.DownloadToByteArray($byteArray, 0)
I get this error:

Easy fix.
In the first example, you are correctly reading the contents of the file from the blob. But, -InputFile is expecting a file path (e.g. C:\arst.xmla), and can't handle the raw contents of the .xmla file.
Instead, use the -Query parameter to pass the contents of the file to Invoke-ASCmd e.g.:
...
$query = $blob.ICloudBlob.DownloadText()
Invoke-ASCmd `
-Database $AnalysisServiceDatabase `
-Query $query `
-server $AnalysisServiceServer

Related

How to convert JSON file content into Powershell object in Powershell runbook?

I'm trying to convert JSON file which is present in storage account data into Powershell object. But I'm not getting the proper output. Output does not contain proper values.
My code:
$storageAccountKey = "xxxx"
$Context = New-AzStorageContext -StorageAccountName 'xxxx' -StorageAccountKey $storageAccountKey
$b='C:\Temp\Scheduled.json'
Get-AzureStorageFileContent -Path $b -ShareName "file-share-name"
$newScheduledRules = Get-Content -Raw $b | ConvertFrom-Json
Write-Output ("########newScheduledRules::###########" + $newScheduledRules)
Output:
Could not get the storage context. Please pass in a storage context or set the current storage context.
########newScheduledRules::############{Scheduled=System.Object[]; Fusion=System.Object[]; MLBehaviorAnalytics=System.Object[]; MicrosoftSecurityIncidentCreation=System.Object[]}
I have reproduced in my environment and got expected results as below and followed below process and followed Microsoft-Document:
Scheduled.json:
{
"Rithwik":"Hello",
"Chotu":"Bojja"
}
Firstly, I have created Storage account and then added a Scheduled.json in file share as below:
Now i have created a runbook and excuted below script in runbook as below:
$storageAccountKey = "PFHxFbVmAEvwBM6/9kW4nORJYA+AStA2QQ1A=="
$Context = New-AzStorageContext -StorageAccountName 'rithwik' -StorageAccountKey $storageAccountKey
$out = "$($env:TEMP)\$([guid]::NewGuid())"
New-Item -Path $out -ItemType Directory -Force
Get-AzStorageFileContent -ShareName "rithwik" -Context $Context -Path 'Scheduled.json' -Destination $out -Force
$newScheduledRules = Get-Content -Path "$out\Scheduled.json" -Raw | ConvertFrom-Json
Write-Output ("########newScheduledRules::###########" + $newScheduledRules)
Output:
Here $out is the Destination Variable.
-Path should be Only the file name Scheduled.json in
Get-AzStorageFileContent command.
It seems the Get-AzureStorageFileContent is missing -Context parameter. It should be something like this
$OutPath = "$($env:TEMP)\$([guid]::NewGuid())"
New-Item -Path $OutPath -ItemType Directory -Force
$storageContext = (Get-AzStorageAccount -ResourceGroupName xxxx -Name xxxx).Context
Get-AzStorageFileContent -ShareName "file-share-name" -Context $storageContext -Path 'Scheduled.json' -Destination $OutPath -Force
$newScheduledRules = Get-Content -Path "$OutPath\Scheduled.json" -Raw | ConvertFrom-Json
Write-Output ("########newScheduledRules::###########" + $newScheduledRules)

Encrypt Azure Storage account key in powershell script

I'm developing a new powershell script in order to download any blobs from a specific container and the problem is due to security reasons because I do not want to paste in text plain the azure account key.
So I have implemented a solution using 'ConvertTo-SecureString' command but the problem still exists because when I create a connection string to the blob, there appears a message who said: "Server Failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. HTTP Status Code 403 - HTTP Error".
With the key in plain text I'm able to create the connection string properly and then list and download all blobs from the container.
I tried other solutions for example ' $Credential= New-Object System.Management.Automation.PSCredential ('$ShareUser, $SharePassword)'
but there is other problem related with the input is not valid base64 string.
Do you know how to avoid this issues and create a secure connection string with an Azure Storage Account?
Best regards and thanks in advance
Here a part of my powershell script
$SecurePassword= Read-Host -AsSecureString | ConvertFrom-SecureString
$SecurePassword | Out-File -FilePath C:\test_blob\pass_file.xml
$ConfigFile= 'C:\Users\\config_file.xml'
IF (Test-Path) {
[xml]$Config= Get-Content $ConfigFile
[string] $Server = $Config.Config.Server;
[string] $SharePassword = $Config.Config.SharePassword;
} ELSE
{
write-host "File do not exists: $ConfigFile"
}
#BlobStorageInformation
$StorageAccountName='test_acc'
$Container='test'
$DestinationFolder= 'C:\Users\user1\Blobs'
$Context = New-AzStorageConext -StorageAccountName $StorageAccountName -StorageAccountKey $SharePassword
#List of Blobs
$ListBlob=#()
$ListBlob+= Get-AzStorageBlob -context $Context -container $Container | Where-Object {$_.LastModified -lt (Get-Date).AddDAys(-1)}
Why would you maintain the password files or enter storage key manually when you have az powershell. Just login using az powershell, set the subscription and enjoy !
$ResourceGroupName = "YOURRESOURCEGROUPNAME"
$StorageAccountName = "YOURSTORAGEACCOUNTNAME"
$ContainerName = "YOURCONTAINERNAME"
$LocalPath = "D:\Temp"
Write-Output 'Downloading Content from Azure blob to local...'
$storageKey = (Get-AzStorageAccountKey -ResourceGroupName $ResourceGroupName -AccountName $StorageAccountName).value[0]
$storageContext = New-AzStorageContext -StorageAccountName $StorageAccountName -StorageAccountKey $storageKey
$blobs = Get-AzStorageBlob -Container $ContainerName -Context $storageContext
foreach($blob in $blobs)
{
Get-AzStorageBlobContent -Container $ContainerName -Context $storageContext -Force -Destination $LocalPath -Blob $blob.Name
}
Write-Output 'Content Downloaded Successfully !!!'

Argument not serializable PowerShell

I have a strange error, I'm running a PowerShell script in DevOps and at the end I want the contents sent to a storage account, I use the snippet below, which works in PowerShell ISE
$strgrg = "storage-rg"
$StorageAccountName = "storageaccount2021"
$ContainerName = "container1"
$Filename = "nsg.txt"
$endtime = (get-date).AddHours("2")
$ctx = (Get-AzStorageAccount -ResourceGroupName $strgrg -Name $StorageAccountName).context
$StorageAccountKey = New-AzStorageContainerSASToken -Context $ctx -Name $ContainerName -Permission racwdl -ExpiryTime $endtime
Set-AzStorageBlobContent -file ./$Filename -Container $ContainerName -Blob $Filename -Context $ctx -force
The error I get is:
WARNING: System.ArgumentException: Argument passed in is not serializable.
Parameter name: value at System.Collections.ListDictionaryInternal.set_Item(Object key, Object value)
at Microsoft.WindowsAzure.Commands.Common.MetricHelper.PopulatePropertiesFromQos(AzurePSQoSEvent qos, IDictionary`2 eventProperties, Boolean populateException)
at Microsoft.WindowsAzure.Commands.Common.MetricHelper.LogUsageEvent(AzurePSQoSEvent qos)
I put some debugging in and it looks like the error occurs after $endtime = (get-date).AddHours("2") when it's running the following line, but that seems fine $ctx = (Get-AzStorageAccount -ResourceGroupName $strgrg -Name $StorageAccountName).context
Can anyone see where I've gone wrong?
Thanks in advance :)

Azure Automation Runbook missing mandatory parameters

I'm trying to set a Tag on all virtual machines in my subscription but I keep getting errors when running the Runbook.
The error is the following:
Get-AzureRmVM : Cannot process command because of one or more missing mandatory parameters: ResourceGroupName. At line:30
Here is my Runbook:
$azureConnection = Get-AutomationConnection -Name 'AzureRunAsConnection'
#Authenticate
try {
Clear-Variable -Name params -Force -ErrorAction Ignore
$params = #{
ServicePrincipal = $true
Tenant = $azureConnection.TenantID
ApplicationId = $azureConnection.ApplicationID
CertificateThumbprint = $azureConnection.CertificateThumbprint
}
$null = Add-AzureRmAccount #params
}
catch {
$errorMessage = $_
Throw "Unable to authenticate with error: $errorMessage"
}
# Discovery of all Azure VM's in the current subscription.
$azurevms = Get-AzureRmVM | Select-Object -ExpandProperty Name
Write-Host "Discovering Azure VM's in the following subscription $SubscriptionID Please hold...."
Write-Host "The following VM's have been discovered in subscription $SubscriptionID"
$azurevms
foreach ($azurevm in $azurevms) {
Write-Host "Checking for tag $vmtagname on $azurevm"
$tagRGname = Get-AzureRmVM -Name $azurevm | Select-Object -ExpandProperty ResourceGroupName
$tags = (Get-AzureRmResource -ResourceGroupName $tagRGname -Name $azurevm).Tags
If ($tags.UpdateWindow){
Write-Host "$azurevm already has the tag $vmtagname."
}
else
{
Write-Host "Creating Tag $vmtagname and Value $tagvalue for $azurevm"
$tags.Add($vmtagname,$tagvalue)
Set-AzureRmResource -ResourceGroupName $tagRGname -ResourceName $azurevm -ResourceType Microsoft.Compute/virtualMachines -Tag $tags -Force `
}
}
Write-Host "All tagging is done"
I tried importing the right modules but this doesn't seem to affect the outcome.
Running the same commands in Cloud Shell does work correctly.
I can reproduce your issue, the error was caused by this part Get-AzureRmVM -Name $azurevm, when running this command, the -ResourceGroupName is needed.
You need to use the Az command Get-AzVM -Name $azurevm, it will work.
Running the same commands in Cloud Shell does work correctly.
In Cloud shell, azure essentially uses the new Az module to run your command, you can understand it runs the Enable-AzureRmAlias before the command, you could check that via debug mode.
Get-AzureRmVM -Name joyWindowsVM -debug
To solve your issue completely, I recommend you to use the new Az module, because the AzureRM module was deprecated and will not be updated.
Please follow the steps below.
1.Navigate to your automation account in the portal -> Modules, check if you have imported the modules Az.Accounts, Az.Compute, Az.Resources, if not, go to Browse Gallery -> search and import them.
2.After import successfully, change your script to the one like below, then it should work fine.
$azureConnection = Get-AutomationConnection -Name 'AzureRunAsConnection'
#Authenticate
try {
Clear-Variable -Name params -Force -ErrorAction Ignore
$params = #{
ServicePrincipal = $true
Tenant = $azureConnection.TenantID
ApplicationId = $azureConnection.ApplicationID
CertificateThumbprint = $azureConnection.CertificateThumbprint
}
$null = Connect-AzAccount #params
}
catch {
$errorMessage = $_
Throw "Unable to authenticate with error: $errorMessage"
}
# Discovery of all Azure VM's in the current subscription.
$azurevms = Get-AzVM | Select-Object -ExpandProperty Name
Write-Host "Discovering Azure VM's in the following subscription $SubscriptionID Please hold...."
Write-Host "The following VM's have been discovered in subscription $SubscriptionID"
$azurevms
foreach ($azurevm in $azurevms) {
Write-Host "Checking for tag $vmtagname on $azurevm"
$tagRGname = Get-AzVM -Name $azurevm | Select-Object -ExpandProperty ResourceGroupName
$tags = (Get-AzResource -ResourceGroupName $tagRGname -Name $azurevm).Tags
If ($tags.UpdateWindow){
Write-Host "$azurevm already has the tag $vmtagname."
}
else
{
Write-Host "Creating Tag $vmtagname and Value $tagvalue for $azurevm"
$tags.Add($vmtagname,$tagvalue)
Set-AzResource -ResourceGroupName $tagRGname -ResourceName $azurevm -ResourceType Microsoft.Compute/virtualMachines -Tag $tags -Force `
}
}
Write-Host "All tagging is done"

Escaping characters on AzureBlobContent

I have got a problem to set my content in AzureBlobStorage.
In local, I have succeeded to replace characters for each files in a directory.
$sourceFolder = "C:\MyDirectory"
$targetFolder = "C:\MyDirectoryEncodeded"
$fileList = Dir $sourceFolder -Filter *.dat
MkDir $targetFolder -ErrorAction Ignore
ForEach($file in $fileList) {
$file | Get-Content | %{$_ -replace '"',''} | %{$_ -replace ',','.'} | Set-Content -Path "tempDirectory\$file"
$newFile = Get-Content "tempDirectory\$file"
$Utf8NoBomEncoding = New-Object System.Text.UTF8Encoding $False
[System.IO.File]::WriteAllLines("targetDirectory\$file" , $newFile,$Utf8NoBomEncoding)
}
exit
But now, I need to do the same in Microsoft Azure.
I get the content into an Azure Blob Storage, I escape characters, I encoding my file in UTF-8NoBom and then I set the encode file into a new Blob Directory.
Nevertheless, I faced an issue when I want to set the new content with escape characters (First line in my loop).
$storageContext = New-AzureStorageContext -ConnectionString "DefaultEndpointsProtocol=https;AccountName=<myAccountName>;AccountKey=<myAccountKey>;"
$sourceFolder = Get-AzureStorageBlob -Container "datablobnotencoded" -Blob "*.dat" -Context $storageContext
$targetFolder = Get-AzureStorageBlob -Container "datablob" -Context $storageContext
MkDir $targetFolder -ErrorAction Ignore
ForEach($file in $sourceFolder) {
Get-AzureStorageBlob -Container "datablobnotencoded" -Blob $file.Name -Context $storageContext | Get-AzureStorageBlobContent | %{$_ -replace '"',''} | %{$_ -replace ',','.'} | Set-AzureStorageBlobContent -File $file.Name -Context $storageContext -CloudBlob $file
$newFile = Get-AzureStorageFileContent -Path $file
$Utf8NoBomEncoding = New-Object System.Text.UTF8Encoding $False
[System.IO.File]::WriteAllLines($file , $newFile, $Utf8NoBomEncoding)
}
I've got this error:
Set-AzureStorageBlobContent : Cannot bind parameter 'CloudBlob'.
Cannot convert the
"Microsoft.WindowsAzure.Commands.Storage.Model.ResourceModel.AzureStorageBlob"
value of type
"Microsoft.WindowsAzure.Commands.Storage.Model.ResourceModel.AzureStorageBlob"
to type "Microsoft.WindowsAzure.Storage.Blob.CloudBlob". At line:7
char:264
+ ... lobContent -File $file.Name -Context $storageContext -CloudBlob $file
+ ~~~~~
+ CategoryInfo : InvalidArgument: (:) [Set-AzureStorageBlobContent], ParameterBindingException
+ FullyQualifiedErrorId : CannotConvertArgumentNoMessage,Microsoft.WindowsAzure.Commands.Storage.Blob.SetAzureBlobContentCommand
Thank you for your answers!
There are some mistakes in your powershell scripts:
1.You may misunderstand the usage of Get-AzureStorageBlobContent, it's used to download blob to local, you cann't get the content of the blob, more details refer here.
2.In the loop, you used $newFile = Get-AzureStorageFileContent -Path $file, the Get-AzureStorageFileContent cmdlet is for file share storage, not for the blob storage.
You can use Get-AzureStorageBlobContent to download the blobs to a local folder, then operate on the local file which is downloaded from blob storage. After the file is modified, you can use Set-AzureStorageBlobContent to upload the local files to the specified azure blob storage.
Sample code as below, and works fine at my side:
$context = New-AzureStorageContext -ConnectionString "xxxx"
#download the blobs in specified contianers
$sourceFolder_blob = Get-AzureStorageBlob -Container "test-1" -Blob "*.txt" -Context $context
#the target azure container, which you want to upload the modifed blob to
$taget_container="test-2"
#the local path which is used to store the download blobs, and make sure the folders exist before use.
$sourceFolder_local="d:\test\blob1\"
$targetFolder_local="d:\test\blob2\"
foreach($file in $sourceFolder_blob)
{
#download the specified blob to local path
Get-AzureStorageBlobContent -Container "test-1" -Blob $file.name -Destination $sourceFolder_local -Context $context
#get the local file path
$local_file_path=$sourceFolder_local + $file.name
#set content to the file in target local folder
$local_target_file_path = "$targetFolder_local"+$file.name
#since the files are downloaded to local, you can any operation for the local file
Get-Content $local_file_path | %{$_ -replace '-','!'} | %{$_ -replace ',','.'} | Set-Content -Path $local_target_file_path
$newFile = Get-Content -Path $local_target_file_path
$Utf8NoBomEncoding = New-Object System.Text.UTF8Encoding $False
[System.IO.File]::WriteAllLines($local_target_file_path , $newFile,$Utf8NoBomEncoding)
#the last step, upload the modified file to another azure container
Set-AzureStorageBlobContent -File $local_target_file_path -Context $context -Container $taget_container
}

Resources