I am uploading the result of some computation to an azure storage account.
I use these statements:
$context = New-AzureStorageContext -StorageAccountName "csb5387a20f4e8cx435ax9b8" -StorageAccountKey "xxxx "
Set-AzureStorageBlobContent -Container "azurescan" -File $FileName -context $context
It has worked and this statments still work on the Cloud Shell.
But in the runbook they get the following error-messages, which are not very helpful.
Can anybody help me?
Set-AzureStorageBlobContent : Method not found:
'System.Net.HttpWebRequest
Microsoft.WindowsAzure.Storage.RequestEventArgs.get_Request()'. At
line:329 char:5
+ Set-AzureStorageBlobContent -Container "azurescan" -File $FileNam ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : CloseError: (:) [Set-AzureStorageBlobContent], StorageException
+ FullyQualifiedErrorId : StorageException,Microsoft.WindowsAzure.Commands.Storage.Blob.SetAzureBlobContentCommand
Related
I am trying to upload a csv file to azure file share. Below is the cmdlets I am using:
$folderPath="/"
$ctx=(Get-AzStorageAccount -ResourceGroupName 'rg-aexch' -Name 'storaexch').Context
$fileShare = Get-AZStorageShare -Context $ctx -Name "fs-aexch"
Set-AzStorageFileContent -Share $fileShare -Source "E:\ Exc\dash\batch1.csv" -Path $folderPath -Force
But I am getting the below error:
Set-AzStorageFileContent : Cannot bind parameter 'Share'. Cannot convert the "Microsoft.WindowsAzure.Commands.Common.Storage.ResourceModel.AzureStorageFileShare" value of type
"Microsoft.WindowsAzure.Commands.Common.Storage.ResourceModel.AzureStorageFileShare" to type "Microsoft.Azure.Storage.File.CloudFileShare".
At line:13 char:37
Set-AzStorageFileContent –Share $s –Source $localFile
~~
CategoryInfo : InvalidArgument: (:) [Set-AzStorageFileContent], ParameterBindingException
FullyQualifiedErrorId : CannotConvertArgumentNoMessage,Microsoft.WindowsAzure.Commands.Storage.File.Cmdlet.SetAzureStorageFileContent
Below script works for one recovery vault but fails for another.
Script
$Sub = Get-AzSubscription -SubscriptionName ''
$context = $Sub | Set-AzContext
$rv = Get-AzRecoveryServicesVault -ResourceGroupName '' -Name ''
Set-AzRecoveryServicesAsrVaultContext -Vault $rv -defaultprofile $context
Error
Set-AzRecoveryServicesAsrVaultContext : Operation failed.
Download vault credential file using cmdlet Get-AzRecoveryServicesVaultSettingsFile and Import-AzRecoveryServicesAsrVaultSettingsFile
At line:1 char:1
+ Set-AzRecoveryServicesAsrVaultContext -Vault $rv -defaultprofile $con
+ CategoryInfo : CloseError: (:) [Set-AzRecoveryServicesAsrVaultContext], Exception
+ FullyQualifiedErrorId : Microsoft.Azure.Commands.RecoveryServices.SiteRecover
Hi the issue was related to vnet configuration. now it is resolved.
error was was coming as data was not passing from one vnet to another.
I have a powershell script which I want to run through azure automation account. The script worked fine when running on cloudshell but when running it through a powershell runbook it is not doing the intended tasks and throwing errors.When I am using Select-AzSubscription -SubscriptionName 'xxx' it tells me to use Connect-AzAccount and when I use that it gives me Cannot find an open port error.
Can anyone help with this? I am listing down the four different error messages I am getting.
Connect-AzAccount : Cannot find an open port. At line:6 char:1 + Connect-AzAccount + ~~~~~~~~~~~~~~~~~ + CategoryInfo : CloseError: (:) [Connect-AzAccount], Exception + FullyQualifiedErrorId : Microsoft.Azure.Commands.Profile.ConnectAzureRmAccountCommand
Get-AzVM : Argument passed in is not serializable. Parameter name: value At line:19 char:12 + $vmOSDisk=(Get-AzVM -ResourceGroupName $resourceGroupName -Name $vmNa ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : CloseError: (:) [Get-AzVM], ArgumentException + FullyQualifiedErrorId : Microsoft.Azure.Commands.Compute.GetAzureVMCommand
New-AzStorageContext : Context cannot be null. Please log in using Connect-AzAccount. At line:50 char:23 + ... onContext = New-AzStorageContext -StorageAccountName $destinationstor ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : CloseError: (:) [New-AzStorageContext], InvalidOperationException + FullyQualifiedErrorId : Microsoft.WindowsAzure.Commands.Storage.Common.Cmdlet.NewAzureStorageContext
Start-AzStorageBlobCopy : Cannot bind argument to parameter 'AbsoluteUri' because it is null. At line:55 char:38 + Start-AzStorageBlobCopy -AbsoluteUri $sas.AccessSAS -DestContainer $d ... + ~~~~~~~~~~~~~~ + CategoryInfo : InvalidData: (:) [Start-AzStorageBlobCopy], ParameterBindingValidationException + FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.WindowsAzure.Commands.Storage.Blo [![enter image description here][1]][1]b.Cmdlet.StartAzureStorageBlobCopy
Here is a snip of a part of code for reference. Az module is used in the script and new Azure automation account does not have Az module installed by default, even after importing Az.Accounts, Az.Automation, Az.Compute I am getting these errors.
In cloud shell, it will login your user account automatically, actually Connect-AzAccount does not work, but it will not give you an error, just a warning, so the script will work.
In the automation runbook, it does not support the interactive way to login your user account, if you use Connect-AzAccount directly, you will get the error, the other errors are follow-up issues based on this.
So to solve the issues, we always use the Run As Account of your automation account in this scenario, essentially it is an AD App along with a service principal in your AAD tenant. Make sure you have enabled it and give all the permissions to it the same as your user account to run the script, its name is like automationname_xxxxxxxxx, you can check it in the Run As Account blade in the automation account in the portal.
Then use the commands below to login.
$connectionName = "AzureRunAsConnection"
try
{
# Get the connection "AzureRunAsConnection "
$servicePrincipalConnection=Get-AutomationConnection -Name $connectionName
Connect-AzAccount `
-ServicePrincipal `
-TenantId $servicePrincipalConnection.TenantId `
-ApplicationId $servicePrincipalConnection.ApplicationId `
-CertificateThumbprint $servicePrincipalConnection.CertificateThumbprint
}
catch {
if (!$servicePrincipalConnection)
{
$ErrorMessage = "Connection $connectionName not found."
throw $ErrorMessage
} else{
Write-Error -Message $_.Exception
throw $_.Exception
}
}
After login, please use Set-AzContext -Subscription <subscription-id> instead of Select-AzSubscription, also please import Az.Storage module in the automation account, because some commands like New-AzStorageContext and Start-AzStorageBlobCopy belong to this module, after doing them, your script should work.
Update:
I test with the script you used with Storage Blob Data Contributor, it works fine.
$connectionName = "AzureRunAsConnection"
try
{
# Get the connection "AzureRunAsConnection "
$servicePrincipalConnection=Get-AutomationConnection -Name $connectionName
Connect-AzAccount `
-ServicePrincipal `
-TenantId $servicePrincipalConnection.TenantId `
-ApplicationId $servicePrincipalConnection.ApplicationId `
-CertificateThumbprint $servicePrincipalConnection.CertificateThumbprint
}
catch {
if (!$servicePrincipalConnection)
{
$ErrorMessage = "Connection $connectionName not found."
throw $ErrorMessage
} else{
Write-Error -Message $_.Exception
throw $_.Exception
}
}
$sas = Grant-AzSnapshotAccess -SnapshotName "joyvmsnap" -ResourceGroupName "xxxxx" -DurationInSecond 3600 -Access Read
$destinationContext = New-AzStorageContext -StorageAccountName "joystoragev2" -UseConnectedAccount
$storageContainerName="image"
$destinationVHDFileName="test.vhd"
Start-AzStorageBlobCopy -AbsoluteUri $sas.AccessSAS -DestContainer $storageContainerName -DestContext $destinationContext -DestBlob $destinationVHDFileName -Force
Make sure the firewall of the storage account is set to allow access from all networks and the Storage Blob Data Contributor role is assigned at the storage account level or higher, not the container level.
I need to remove all of the blobs in a specific container before doing a backup to that container. The following does not work. What is the proper way to do this?
Get-AzStorageBlob -Container $ContainerName -Context $ctx | Remove-AzStorageBlob -DeleteSnapshot -Force
I get the following error message for each blob returned by Get-AzStorageBlob:
Remove-AzStorageBlob : The specified blob does not exist. HTTP Status Code: 404 - HTTP Error Message: The specified blob does not exist.
ErrorCode: BlobNotFound
ErrorMessage: The specified blob does not exist.
RequestId:eb2612f4-f01e-0067-5471-3e9f69000000
Time:2019-07-19T20:33:50.1261168Z
At line:15 char:62
+ ... inerName -Context $ctx | Remove-AzStorageBlob -DeleteSnapshot -Force
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : CloseError: (:) [Remove-AzStorageBlob], StorageException
+ FullyQualifiedErrorId : StorageException,Microsoft.WindowsAzure.Commands.Storage.Blob.RemoveStorageAzureBlobCommand
I test with your ps script, it's caused by the -DeleteSnapshot -Force, delete it and then it works well.
Get-AzureStorageBlob -Container $ContainerName -Context $ctx | Remove-AzureStorageBlob
And if you want to delete snapshot, just add -Force. Here is the Parameters description, -Force indicates that this cmdlet removes the blob and its snapshot without confirmation.
You could have a try, hope this could help you.
Note: If you are using the newer Az Module, the above code will look like:
Get-AzStorageBlob -Container $ContainerName -Context $ctx | Remove-AzStorageBlob
I don't think the Remove-AzStorageBlob can delete blobs in bulk, so you will have to loop through the blobs you get returned from Get-AzureStorageBlob.
Get-AzStorageBlob -Container $ContainerName -Context $ctx | ForEach-Object {
$_ | Remove-AzureStorageBlob # or: Remove-AzureStorageBlob -ICloudBlob $_.ICloudBlob -Context $ctx
}
I want to copy from Azure file share to blob and I am following below refs, and I wanted a incremental copy which is available in azcopy with not in Start-AzureStorageBlobCopy. And ofcourse Start-AzureStorageBlobIncrementalCopy wont do copy from file share to blob.
https://learn.microsoft.com/en-us/powershell/module/azure.storage/start-azurestorageblobincrementalcopy?view=azurermps-6.8.1
https://learn.microsoft.com/en-us/powershell/module/azure.storage/start-azurestorageblobcopy?view=azurermps-6.8.1
I wrote small commands,
$StorageContext = New-AzureStorageContext -StorageAccountName 'neverdelete' -StorageAccountKey 'XXX'
$Srcsh = Get-AzureStorageFile -ShareName "filetest" -Context $StorageContext
$DestBlob = Get-AzureStorageBlob -Container "client1" -Context $StorageContext
Start-AzureStorageBlobCopy -SrcShare $Srcsh --DestContainer $DestBlob
But this throws an error below:
Start-AzureStorageBlobCopy : Cannot convert 'System.Object[]' to the type 'Microsoft.WindowsAzure.Storage.File.CloudFileShare' required by parameter 'SrcShare'. Specified method
is not supported.
At line:4 char:38
+ Start-AzureStorageBlobCopy -SrcShare $Srcsh --DestContainer $DestBlob
+ ~~~~~~
+ CategoryInfo : InvalidArgument: (:) [Start-AzureStorageBlobCopy], ParameterBindingException
+ FullyQualifiedErrorId : CannotConvertArgument,Microsoft.WindowsAzure.Commands.Storage.Blob.Cmdlet.StartAzureStorageBlobCopy
I dont know what I am doing wrong here and how do I script it so that it could be incremental.
Thanks,
Akshay
You get the error because the
-SrcShare parameter expects a CloudFileShare and you pass a list of AzureStorageFile / AzureStorageDirectory. Also your -DestContainer has two -- and you are missing the context.
So you could e. g. iterate over your AzureStorageFiles and invoke the Start-AzureStorabeBlobCopy cmdlet for each of them:
$Srcsh | ForEach-Object {
Start-AzureStorageBlobCopy -SrcShare ($Srcsh[0] | Select-Object -ExpandProperty Share) -SrcFilePath $_.Name -DestContainer "client1" -Context $StorageContext
}
I suppose that you want to copy the files in the storage fileshare to storage container, you could try the command below.
$StorageContext = New-AzureStorageContext -StorageAccountName 'StorageAccountName' -StorageAccountKey 'xxxxx'
$Srcsh = Get-AzureStorageFile -ShareName "filesharename" -Context $StorageContext | Get-AzureStorageFile
$DestBlob = Get-AzureStorageContainer -Container "containername" -Context $StorageContext
foreach ($item in $Srcsh)
{
Start-AzureStorageBlobCopy -SrcFile $item -DestContainer "DestContainername" -Context $StorageContext
}
My fileshare:
Result in the container:
For Start-AzureStorageBlobCopy (Start-AzureStorageFileCopy), the cmdlets can only copy single File/Blob, can't copy a share/container, and the copy can't be resume if fail.
The incremental copy (https://learn.microsoft.com/en-us/rest/api/storageservices/incremental-copy-blob) can only copy page blob snapshot, and currently File not support it. As you say AzCopy support it, what do you means about "incremental copy"?
If you want to copy all files in the root directory in a share, you can use following command. (Please note, Get-AzureStorageFile will only get the first level file/dir in a share or dir, it won't get the files in subdir.)
Get-AzureStorageFile -ShareName $shareName -Context $StorageContext
| where {$_.GetType().Name -eq "CloudFile"} |Start-AzureStorageBlobCopy -DestContainer $containerName -DestContext $StorageContext
-context $StorageContext
BTW, for any azure powershell problem, the formal way is to open an issue in https://github.com/Azure/azure-powershell/issues, and the proper team will follow up it.