I am trying to create new directories in Azure datalake using azure automation powershell workflow.
The code that i have is like this:
$ctx = New-AzStorageContext -ConnectionString $connectionstring
New-AzDataLakeGen2Item -Context $ctx -FileSystem $filesystemName -Path $dirname -Directory
the error message that I get:
New-AzDataLakeGen2Item : Cannot bind parameter 'Context'. Cannot convert the "Microsoft.WindowsAzure.Commands.Storage.AzureStorageContext" value of type "Deserialized.Microsoft.WindowsAzure.Commands.Storage.AzureStorageContext" to type "Microsoft.Azure.Commands.Common.Authentication.Abstractions.IStorageContext
I dont know how to solve this error? any help is greatly appreciated.
In our local environment, We have created a Powershell runbook running with PowerShell Version 5.1.
Using the same above shared cmdlets, we are able to create a new directory in the Azure Data lake storage account. We have used System managed identity to connect to subscription resources from the automation account.
Here is the Powershell script that we have used in Automation Account:
Disable-AzContextAutosave -Scope Process # Ensures you do not inherit an AzContext in your runbook
$AzureContext = (Connect-AzAccount -Identity).context # Connect to Azure with system-assigned managed identity
$AzureContext = Set-AzContext -SubscriptionName $AzureContext.Subscription -DefaultProfile $AzureContext # set and store context
Import-module -name Az.Storage
$ctx=New-AzStorageContext -StorageAccountName "<strgaccountName>" -StorageAccountKey "<storageaccountKey>"
New-AzDataLakeGen2Item -Context $ctx -FileSystem "testfiler" -Path "dir1/dir2" -Directory
Here is the Sample output for reference:
Related
After create a runbook and edit content, I want to create variable and set value for them. How can I do it by ansible or azure cli ?
Please help me
Azure Automation stores each encrypted variable securely. When you create a variable, you can specify its encryption and storage by Azure Automation as a secure asset.
You must set the value with the Set-AzAutomationVariable cmdlet or the internal Set-AutomationVariable cmdlet. You use the Set-AutomationVariable in your runbooks that are intended to run in the Azure sandbox environment, or on a Windows Hybrid Runbook Worker.
You can create variables and set value for them using PowerShell script.
$rgName = "ResourceGroup01"
$accountName = "MyAutomationAccount"
$vm = Get-AzVM -ResourceGroupName "ResourceGroup01" -Name "VM01" | Select Name, Location,Extensions
New-AzAutomationVariable -ResourceGroupName "ResourceGroup01" -AutomationAccountName "MyAutomationAccount" -Name "MyComplexVariable" -Encrypted $false -Value $vm
$vmValue = Get-AzAutomationVariable -ResourceGroupName "ResourceGroup01" -AutomationAccountName "MyAutomationAccount" -Name "MyComplexVariable"
$vmName = $vmValue.Value.Name
$vmTags = $vmValue.Value.Tags
Reference: Manage variables in Azure Automation | Microsoft Docs
Azure runbook.
The question:
How to run/call powershell scripts on remote Azure VM via runbook? The script is placed on a remote VM.
There is no Azure AD, powershell has Az module installed.
Thank you.
Have your Azure Automation runbook something like shown below. It will accomplish your requirement.
$ServicePrincipalConnection = Get-AutomationConnection -Name 'AzureRunAsConnection'
Add-AzAccount -ServicePrincipal -TenantId $ServicePrincipalConnection.TenantId -ApplicationId $ServicePrincipalConnection.ApplicationId -CertificateThumbprint $ServicePrincipalConnection.CertificateThumbprint
$rgname ="rrrrrrrrrrrrrr"
$vmname ="vvvvvvvvvvvvvv"
$ScriptToRun = "ssssssssssssss"
Out-File -InputObject $ScriptToRun -FilePath ScriptToRun.ps1
Invoke-AzVMRunCommand -ResourceGroupName $rgname -Name $vmname -CommandId 'RunPowerShellScript' -ScriptPath ScriptToRun.ps1
Remove-Item -Path ScriptToRun.ps1
Note: Before you run your runbook, make sure you update "rrrrrrrrrrrrrr" with your resource group name, "vvvvvvvvvvvvvv" with your VM name and "ssssssssssssss" with the path of the script along with script name
For reference, you may refer the source from here.
Hope this helps!! Cheers!!
I have prepared a script for executing .ps1 script on a Windows in Azure. Powershell script is stored on a storage account. Script output is directed to a local C:\ drive on a Windows VM. In order to run a script on a VM i am using Custom Script Extension, extract from my code below. Question is, how can I copy received output.csv from my C:\ drive on a Windows VM to a storage account?
##Storage Account credentials
$StorageAccountName = (Get-AutomationPSCredential -Name 'xxx').UserName
$StorageAccountKey = (Get-AutomationPSCredential -Name 'xxx').GetNetworkCredential().Password
$Context = New-AzureStorageContext -StorageAccountName $StorageAccountName -StorageAccountKey $StorageAccountKey
$CSEName = ((Get-AzureRmVm -ResourceGroupName $ResourceGroup -Name $VM |
Where-Object {$_.Extensions.VirtualMachineExtensionType -eq "CustomScriptExtension"}).Extensions |
Where-Object {$_.VirtualMachineExtensionType -eq "CustomScriptExtension"}).Name
write-output "CSE Name: $CSEName"
Set-AzureRmVMCustomScriptExtension -ResourceGroupName $ResourceGroup `
-VMName $VM `
-FileName $psfilename `
-ContainerName $ContainerName `
-StorageAccountName $StorageAccountName `
-StorageAccountKey $StorageAccountKey `
-Run $psfilename `
-Location "North Europe"`
-Name $CSEName
$LogFilePath = "C:\Users\xxx\Desktop\Output.csv"
Note: I re-edited my entire Ansewr
if I understand your script correctly, you are using it for internal usage.
in that case, the solution is very simple.
as you have all the information on the source and destination, all you need is to connect them to the proper Blob to Copy.
You will need either a Storage Account Access-Key (easy to get/ use but less secure)
or a SAS with at least the following Privileges =Read, wright, List. (a bit harder to configure but more secure)
here is an example using Storage Account Access-Key:
$ResourceGroupName = "Resource-Group"
$storageAccountName = "Storge account"
#full path to the file
$localPath = "C:\test\vm01.5f9e0fc2-cf60-444f-9cba-61c6f66b6373.serialconsole.log"
#blob container name
$storageContainerName = "test"
#Blob name (the name of the file in after the transfer)
$BlobName = "destFileName.log"
#getting the SA Key
$SAkey = (Get-AzStorageAccountKey -ResourceGroupName $ResourceGroupName -AccountName $storageAccountName).Value[0]
#getting Storage account context
$destinationContext = New-AzStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $SAkey
#copy the file
Set-AzStorageBlobContent -File $localPath -Container $storageContainerName -Blob $BlobName -Context $destinationContext
the script is working if your connected to azure, if not, you can pass a SAS URL instedof the $SAkey and it will work the same.
to run it, just add at the end of your ps1 script.
I am deploying Azure VMs (with unmanaged disks), that are based on a VHD image. JSON templates used for deployment are stored in my Azure subscription.
Sometimes I need to reset the machine to the original state - the manual way to achieve this through the Azure web portal is:
Open the resource group, delete VM (while keeping other resources).
Going to the storage account and deleting VHD that served as OS disk for the machine.
Go back to the Resource group -> Deployment -> select last Deployment -> Redeploy.
I want to do this programmatically using PowerShell. All the steps are quite easily achievable except for the last one - running redeployment.
This is my PowerShell code:
# Authenticate to Azure Account
Login-AzAccount
$vm = Get-AzVM | Out-GridView -Title "Select machine to be reset to factory state" -PassThru
$groupName = $vm.ResourceGroupName
#Stop the VM
Stop-AzureRmVM -ResourceGroupName $vm.ResourceGroupName -Name $vm.Name -Force
#Delete VM
#Remove-AzureRmVM -ResourceGroupName $vm.ResourceGroupName -Name $vm.Name
#Getting storage context, blob name and deleting VHD (blob)
$disk = $vm.StorageProfile.OsDisk
$storageAccount = Get-AzStorageAccount -ResourceGroupName "myStorageAccountResourceGroupName" -Name "myStorageAccountName"
#Get storage context
$storageKey = (Get-AzStorageAccountKey -ResourceGroupName $storageAccount.ResourceGroupName -Name $storageAccount.StorageAccountName)[0].Value
$context = New-AzStorageContext -StorageAccountName $storageAccount.StorageAccountName -StorageAccountKey $storageKey
$container = Get-AzStorageContainer -Context $context -Name 'vhds'
$blobName = $disk.Name + ".vhd"
$blob = Get-AzStorageBlob -Container $container.Name -Context $context -Blob $blobName
#Delete Blob
$blob | Remove-AzStorageBlob
Now for the last step - I can get the last Resource group deployment and set up a new deployment with -RollbackToLastDeployment parameter.
#Redeploy Group
$deployments = Get-AzResourceGroupDeployment $groupName
$deployment = $deployments[$deployments.size - 1]
New-AzResourceGroupDeployment -Name $deployment.DeploymentName -ResourceGroupName $groupName -TemplateFile <Expects template in local storage> -RollbackToLastDeployment
The problem is that the New-AzResourceGroupDeployment command expects a JSON template that is on my local disk, but I have my templates stored in the Azure subscription.
Is there any way to use a template that is located in Azure subscription for redeployment of a resource group?
No matter where is the template file located, you could convert/copy the template to the .json file in local, then upload it to the storage, then you will be able to use the -TemplateUri parameter to deploy the remote template.
Sample:
Set-AzCurrentStorageAccount -ResourceGroupName ManageGroup -Name {your-unique-name}
# get the URI with the SAS token
$templateuri = New-AzStorageBlobSASToken -Container templates -Blob storage.json -Permission r `
-ExpiryTime (Get-Date).AddHours(2.0) -FullUri
# provide URI with SAS token during deployment
New-AzResourceGroup -Name ExampleGroup -Location "South Central US"
New-AzResourceGroupDeployment -ResourceGroupName ExampleGroup -TemplateUri $templateuri
For more details, you could refer to this link.
Update:
Seems we could not find the uri of the Template(preview) in the portal, my workaround is copy the template as a .json file in local manually, then upload to the azure blob storage, then use the sample above.
Follow the steps:
1.In the portal, click the View Template, you can copy the template and save it as a .json file in local.
2.Then go to the container of your storage account, upload the .json file.
3.Click the ... of your .json file -> Generate SAS -> Generate blob SAS token and URL, copy the Blob SAS URL, it is the $templateuri what you need in the New-AzResourceGroupDeployment -ResourceGroupName ExampleGroup -TemplateUri $templateuri. Or you can use New-AzStorageBlobSASToken generate it like the sample above.
I wan't to create a Runbook that will start a specific (or parameter controlled) VM, and then run a script (locally or from blob storage) on the VM.
I have checked a lot of documentation, but so far without luck in getting it to work.
What I got so far under the same Resource Group:
VM created
Automation account created incl. Run As account
Azure Automation solution (OMS)
Credential (to my own account) under the automation account
Used several Runbook galleries and other code examples using functions as e.g.
Start-AzureVM ...
Invoke-Command ...
Anyone of you good people out there who can sample a guideline on what is needed depending on methods being used?
The VM start part is working, but I cannot get the login + executing of script to work!
I'm not a high skilled developer, and I have even doubts about choosing between the script languages in Azure.
Any help will be highly appreciated.
Thanks,
Tom
Invoke-Command
Invoke-AzureRmVMRunCommand
Set-AzureRmVMCustomScriptExtension
New-SSHSession + Invoke-SSHCommand
Code taken from e.g. gallary "Connect-AzureVM"
the parameter -ScriptPath of Invoke-AzureRmVMRunCommand should not point to the path in the remote computer, but should point to the local path of runbook environment.
Sample code like below(create a file named atestfile.txt in the remote vm):
$ServicePrincipalConnection = Get-AutomationConnection -Name 'AzureRunAsConnection'
Add-AzureRmAccount `
-ServicePrincipal `
-TenantId $ServicePrincipalConnection.TenantId `
-ApplicationId $ServicePrincipalConnection.ApplicationId `
-CertificateThumbprint $ServicePrincipalConnection.CertificateThumbprint
#define resource group and vm name
$resourceGroup ="xxx"
$VmName ="xxx"
#define the scripts in $scriptblock, and add the content of $scriptblock to aa.ps1 in current directory of runbook
write-output "create test file"
$scriptblock = "New-Item -path c:\test -name atestfile.txt -itemtype file -force"
Out-File -FilePath aa.ps1 -InputObject $scriptblock
#Note that the -ScriptPath should not point to the remote path(in remote vm), it should point to the local path where you execute the command Invoke-AzureRmVMRunCommand
Invoke-AzureRmVMRunCommand -ResourceGroupName $resourceGroup -Name $VmName -CommandId 'RunPowerShellScript' -ScriptPath aa.ps1
#after execution, you can remove the file
Remove-Item -Path aa.ps1
write-output "done now"
Test result: