Is there a way to set up Cache headers for Images in Azure Blob Storage - Please Suggest - we have around 2M images wanted to add Maxage . I tried Powershell solutions but it says Properties not exist Null refrence error giving...
Is there a work around to set it up ?
Please advise , thank you !!
Edited:
Powershell Script :
#connection info
$StorageName="storagename given here"
$StorageKey="Key Replaced here”
$ContainerName="container name given here"
$BlobUri="added correct URI here"+$ContainerName
#get all the blob under your container
$StorageCtx = New-AzureStorageContext -StorageAccountName $StorageName -StorageAccountKey $StorageKey
$blobs = Get-AzureStorageBlob -Container $ContainerName -Context $StorageCtx
#creat CloudBlobClient
Add-Type -Path "C:\Program Files\Microsoft SDKs\Azure\.NET SDK\v2.9\ref\Microsoft.WindowsAzure.StorageClient.dll"
$storageCredentials = New-Object Microsoft.WindowsAzure.StorageCredentialsAccountAndKey -ArgumentList $StorageName,$StorageKey
$blobClient = New-Object Microsoft.WindowsAzure.StorageClient.CloudBlobClient($BlobUri,$storageCredentials)
#set Properties and Metadata
$cacheControlValue = "public, max-age=60480"
foreach ($blob in $blobs)
{
$blobRef = $blobClient.GetBlobReference($blob.Name)
#set Properties
$blobRef.Properties.CacheControl = $cacheControlValue
$blobRef.SetProperties()
}
Related
I am making a ReST api call and uploading the JSON payload to azure blob storage using the below powershell.
PoSh:
$Params = #{"URI" = 'https://myapiexample.com/api/data/'}
$Result = Invoke-RestMethod #Params | ConvertTo-Json
$context=New-AzStorageContext -StorageAccountName "mystorage" -StorageAccountKey ""
$container=Get-AzStorageContainer -Name "input" -Context $context
$content = [system.Text.Encoding]::UTF8.GetBytes($Result)
$blobpath = "azblob/mydata.json"
$container.CloudBlobContainer.GetBlockBlobReference($blobpath).UploadFromByteArray($content,0,$content.Length)
$container.CloudBlobContainer.GetBlockBlobReference($blobpath).Properties.ContentType = "application/json"
$container.CloudBlobContainer.GetBlockBlobReference($blobpath).SetProperties()
I notice that when the blob is stored in azure blob storage the content-type is application/octet-stream whereas I would want it to be application/json. The code I use is not working as expected.
After reproducing from my end, I could able to achieve your requirement using Set-AzStorageBlobContent. Below is the complete code that is working for me that sets the content type of the blob.
$url = "<YOUR_URL>"
$dest = "samplex.json"
Invoke-WebRequest -Uri $url -OutFile $dest
$context = New-AzStorageContext -StorageAccountName "<YOUR_ACCOUNT_NAME>" -StorageAccountKey "<YOUR_ACCOUNT_KEY>"
Set-AzStorageBlobContent -Context $context -Container "containers" -Blob "mydata.json" -File $dest -Properties #{"ContentType" = "application/json"};
RESULTS:
I'm developing a new powershell script in order to download any blobs from a specific container and the problem is due to security reasons because I do not want to paste in text plain the azure account key.
So I have implemented a solution using 'ConvertTo-SecureString' command but the problem still exists because when I create a connection string to the blob, there appears a message who said: "Server Failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. HTTP Status Code 403 - HTTP Error".
With the key in plain text I'm able to create the connection string properly and then list and download all blobs from the container.
I tried other solutions for example ' $Credential= New-Object System.Management.Automation.PSCredential ('$ShareUser, $SharePassword)'
but there is other problem related with the input is not valid base64 string.
Do you know how to avoid this issues and create a secure connection string with an Azure Storage Account?
Best regards and thanks in advance
Here a part of my powershell script
$SecurePassword= Read-Host -AsSecureString | ConvertFrom-SecureString
$SecurePassword | Out-File -FilePath C:\test_blob\pass_file.xml
$ConfigFile= 'C:\Users\\config_file.xml'
IF (Test-Path) {
[xml]$Config= Get-Content $ConfigFile
[string] $Server = $Config.Config.Server;
[string] $SharePassword = $Config.Config.SharePassword;
} ELSE
{
write-host "File do not exists: $ConfigFile"
}
#BlobStorageInformation
$StorageAccountName='test_acc'
$Container='test'
$DestinationFolder= 'C:\Users\user1\Blobs'
$Context = New-AzStorageConext -StorageAccountName $StorageAccountName -StorageAccountKey $SharePassword
#List of Blobs
$ListBlob=#()
$ListBlob+= Get-AzStorageBlob -context $Context -container $Container | Where-Object {$_.LastModified -lt (Get-Date).AddDAys(-1)}
Why would you maintain the password files or enter storage key manually when you have az powershell. Just login using az powershell, set the subscription and enjoy !
$ResourceGroupName = "YOURRESOURCEGROUPNAME"
$StorageAccountName = "YOURSTORAGEACCOUNTNAME"
$ContainerName = "YOURCONTAINERNAME"
$LocalPath = "D:\Temp"
Write-Output 'Downloading Content from Azure blob to local...'
$storageKey = (Get-AzStorageAccountKey -ResourceGroupName $ResourceGroupName -AccountName $StorageAccountName).value[0]
$storageContext = New-AzStorageContext -StorageAccountName $StorageAccountName -StorageAccountKey $storageKey
$blobs = Get-AzStorageBlob -Container $ContainerName -Context $storageContext
foreach($blob in $blobs)
{
Get-AzStorageBlobContent -Container $ContainerName -Context $storageContext -Force -Destination $LocalPath -Blob $blob.Name
}
Write-Output 'Content Downloaded Successfully !!!'
I'm trying to write a function in Powershell that sets blobs in a specific container to a certain type, as they are always written with the type application/octet-stream which causes issues with downstream applications. I have written the below function but it returns the error 'ContentType' is a ReadOnly property.
I was wondering if there was any way around this? I know the property can be set manually in Azure Storage Explorer, however this is a daily task.
Function:
Function Set-ContentType {
Param (
[string]$accountName,
[string]$accessKey,
[string]$storageContainer
)
# Connect to blob storage and get blobs
$context = New-AzureStorageContext -StorageAccountName $accountName -StorageAccountKey $accessKey
$blobs = Get-AzureStorageBlob -Container $storageContainer -Context $context -Blob $fileMask
foreach ($blob in $blobs) {
if ($blob.ContentType -eq $genericMIME) {
$blob.ContentType = $targetMIME
}
}
}
So based on the link from the comments, please give this solution a try.
Function Set-ContentType {
Param (
[string]$accountName,
[string]$accessKey,
[string]$storageContainer
)
# Connect to blob storage and get blobs
$context = New-AzureStorageContext -StorageAccountName $accountName -StorageAccountKey $accessKey
$blobs = Get-AzureStorageBlob -Container $storageContainer -Context $context -Blob $fileMask
foreach ($blob in $blobs) {
if ($blob.ContentType -eq $genericMIME) {
$blob.Properties.ContentType = $targetMIME
$blob.SetProperties()
}
}
}
I have solved my own issue by writing an alternative upload script that defines the ContentType at the time of writing the blob:
Function UploadFile {
Param (
[string]$accountName,
[string]$accessKey
)
$context = New-AzureStorageContext -StorageAccountName $accountName -StorageAccountKey $accessKey
$files = Get-ChildItem $workingDir -Filter $fileMask
foreach ($file in $files) {
Set-AzureStorageBlobContent -File $file.FullName -Container $container -Properties #{"ContentType" = "$targetMIME"} -Context $context -Force
}
}
It's very easy to list azure managed disks in PS, but unmanaged ones are very tricky to list, as they're not objects from azure POV. I tried to wrote foreach loop to list me all unamanged disks (i.e. *.vhd files) for each storage account. This is the code I wrote:
$StorageAccounts = Get-AzureRmStorageAccount
$sa = $StorageAccounts | foreach-object {
#Get the Management key for the storage account
$key1 = (Get-AzureRmStorageAccountKey -ResourceGroupName $_.ResourceGroupName -name $_.StorageAccountName)[0].value
#Get the Storage Context to access the Storage Container
$storageContext = New-AzureStorageContext -StorageAccountName $_.StorageAccountName -StorageAccountKey $key1
#Get the Storage Container in the Variable
$storageContainer = Get-AzureStorageContainer -Context $storageContext
$blob = Get-AzureStorageBlob -Container $storageContainer.name -Context $storageContext
[PSCustomObject]#{
"Name" = $blob.Name
"Length" = $blob.Length
"Storage Account Name" = $_.StorageAccountName
}
}
I want the loop to fetch all the vhd's for each storageaccount and parse it into pscustomobject to list me all vhd*s from all storage accounts, but I get an error:
Get-AzureStorageBlob : Cannot validate argument on parameter
'Container'. The argument is null or empty. Provide an argument that
is not null or empty, and then try the command again. At line:13
char:41
Get-AzureStorageBlob : Cannot convert 'System.Object[]' to the type
'System.String' required by parameter 'Container'. Specified method is not > supported.
At line:13 char:41
Why loop is not parsing data to $storageContainer in line 11? I can see what's inside other two vars like $key1 and $storageContext.
you can rewrite your script in this fashion:
$StorageAccounts = Get-AzureRmStorageAccount
$StorageAccounts.foreach{
$ctx = $_.Context
$containers = Get-AzureStorageContainer -Context $ctx
$containers.foreach{
$blobs = Get-AzureStorageBlob -Container $_.name -Context $ctx
$blobs.foreach{
do_something
}
}
}
you dont need to get keys to construct context, because storage account variable contains the context. and then you need to iterate containers and blobs
How do I break a lease on an item in Blob Storage utilizing PowerShell?
I'm receiving the following when trying to upload something over the current image:
Add-AzureRmVhd : The remote server returned an error: (412) There is currently a lease on the blob and no lease ID was specified in the request..
At line:1 char:1
+ Add-AzureRmVhd -Destination $osDiskUri -LocalFilePath $localFileName ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : CloseError: (:) [Add-AzureRmVhd], StorageException
+ FullyQualifiedErrorId : Microsoft.Azure.Commands.Compute.StorageServices.AddAzureVhdCommand
Login to the old portal and navigate to the Virtual Machines then the Images tab the url will be https://manage.windowsazure.com/#yourname.onmicrosoft.com#Workspaces/VirtualMachineExtension/images. Select the image and choose Delete on the bottom.
After that go to your storage and delete it.
You can also try the following which will remove blobs for a given container and then remove the container.
Add-AzureAccount
Get-AzureSubscription | Format-Table SubscriptionName, IsDefault, IsCurrent, CurrentStorageAccountName
$SubscriptionName = 'Your subsscription name'
Select-AzureSubscription -SubscriptionName $SubscriptionName
Get-AzureSubscription -Default
Get-AzureStorageAccount | Format-Table -Property StorageAccountName, Location, AccountType, StorageAccountStatus
$StorageAccountName = "Your storage account"
$StorageAccountKey = (Get-AzureStorageKey -StorageAccountName $StorageAccountName).Primary
$ContainerName = "Your container name"
$Context = New-AzureStorageContext -StorageAccountName $StorageAccountName -StorageAccountKey $StorageAccountKey
#Get a reference to all the blobs in the container.
$blobs = Get-AzureStorageBlob -Container $ContainerName -Context $Context
#Remove lease on each Blob
$blobs | %{$_.ICloudBlob.BreakLease()}
#Delete blobs in a specified container.
$blobs| Remove-AzureStorageBlob
Remove-AzureStorageContainer -Container $ContainerName -Context $Context
If you want to break a seal on a blob you can use the How to break the locked lease of blob storage in Microsoft Azure (PowerShell)
$key = (Get-AzureRmStorageAccountKey -ResourceGroupName $selectedStorageAccount.ResourceGroupName -name $selectedStorageAccount.StorageAccountName -ErrorAction Stop)[0].value
$storageContext = New-AzureStorageContext -StorageAccountName $selectedStorageAccount.StorageAccountName -StorageAccountKey $key -ErrorAction Stop
$storageContainer = Get-AzureStorageContainer -Context $storageContext -Name $ContainerName -ErrorAction Stop
$blob = Get-AzureStorageBlob -Context $storageContext -Container $ContainerName -Blob $BlobName -ErrorAction Stop
$leaseStatus = $blob.ICloudBlob.Properties.LeaseStatus;
If($leaseStatus -eq "Locked")
{
$blob.ICloudBlob.BreakLease()
Write-Host "Successfully broken lease on '$BlobName' blob."
}
Else
{
#$blob.ICloudBlob.AcquireLease($null, $null, $null, $null, $null)
Write-Host "The '$BlobName' blob's lease status is unlocked."
}
If you want to a script for ARM resources you can use the
How to break the locked lease of blob storage by ARM in Microsoft Azure(PowerShell)
The lease is likely from something like a VM, or something else using the Blog Storage. As a result manually releasing the lease could cause problems.
With that said, the following PowerShell command should do the trick:
Get-AzureRmStorageAccount -Name "STORAGE_ACCOUNT_NAME" | Get-AzureStorageBlob -name "CONTAINER_NAME").ICloudBlob.BreakLease()
If its a VM, you should see the following post on removing the disk:
Cannot delete blob: There is currently a lease on the blob and no lease ID was specified in the request
However, if you simply want to replace the drive used by every machine that uses the given blob, stopping the VM, releasing the lease, uploading a new image, and starting the VM appears to work.