I am using Azure CDN with blob storage account as endpoint. My data is static and changes only once per year. I am not sure: does it has some impact when I choose blob storage account "Redundancy" as "LRS" vs "GRS"?
I hope that CDN will cache my data in different regions and that blob storage is needed only for first time when CDN gets data from blob storage.
From the CDN perspective, I don't think the redundancy matters because CDN will cache the content and the content will be served from CDN nodes. Redundancy becomes important from data protection perspective.
If you go with LRS and assuming that the datacenter becomes completely inoperable, then you will lose all the source content.
If you opt for GRS, at least you have a copy of your content (though it will not be directly accessible to you) and in case datacenter becomes completely inoperable, your data is not lost and Microsoft will switch to the secondary location.
Recommendation would be to go with GRS over LRS. If you want access to the content in secondary region, then you should go with RA-GRS.
Related
I want to implement a multi-region architecture in azure. My current architecture is running in one region and I am using blob storage to save my data. I was wondering if the blob storage can be shared between users in different regions? because I have seen that the replication is read-only otherwise I have to create another blob storage for the other regions ? and how I synchronize it to have the same data so the users in different regions can see the same content in the software?
I was wondering if the blob storage can be shared between users in
different regions?
Blob storage can certainly be shared between users in different regions. Blob storage resources are accessible over HTTP protocol so it doesn't really matter where your users are.
However please note that you may incur extra charges for data egress if the blob storage data is consumed by the application in your secondary regions.
Furthermore, you will notice some increased latency for both reads and writes. You can reduce the read latency by fronting your blob storage with CDN (but then you will pay extra for the CDN).
How to best use Azure CDN to enhance the reliability of accessing Azure storage?
Hi,
In our current system implementation, we have use an azure storage account to store some essential information of the system. Thus, the storage becomes the single point of failure. In order to enhance the reliability of this mechanism, I am considering to use Azure CDN at the top of storage account.
Since I am new to this product, wondering what is the best practice here. And also here are some questions.
I figure out Azure CDN could provide a cache at the top of storage, but what if I updated the blobs, but the content inside cache doesn't expire, how to force cdn to bypass the cache to fetch the up-to-date content. Could I setup the CDN to detect the update in the storage and catch the new stuff?
Need to confirm, if I create the content delivery network (CDN) in a resource group locating in East US, the cdn is not only available in this region, which means if Azure East US went down, we could still access the endpoint empowered by another edge node in different region.
Also, there is another approach I could think of, Azure storage Geo-redundant storage(GRS). So in this case, I could simply add a try catch in the code, whenever I got a failure from the original storage endpoint, I route to grs endpoint and do the same get.
Which way do you think is better?
The ideal solution would be use the CDN with (RAGRS) storage account. CDN is a global resource which deploys at each of the edges providing far better Caching and Acceleration.
You can purge the CDN cache every time you update the storage blob. Normally this is done using your CI/CD pipeline at deployment.
In Azure DevOps there's already a task, or you can google for various ways to purge the cache.
Using the azure portal:
https://learn.microsoft.com/en-us/azure/cdn/cdn-purge-endpoint
Script:
https://gallery.technet.microsoft.com/scriptcenter/Purge-Azure-CDN-endpoint-042fb00d
I am thinking of using Azure Blob Storage for document management system which I am developing. All Blobs ( images,videos, word/excel/pdf etc) will be stored in Azure Blob storage. As I understand, I need to create container and these files can be stored within the container.
I would like to know how to safeguard against accidental/malicious deletion of the container. If a container is deleted, all the files it contains will be lost. I am trying to figure out how to put backup and recovery mechanism in place for my storage account so that it is always guaranteed that if something happens to a container, I can recover files inside it.
Is there any way provided by Microsoft Azure for such backup and recovery or Do I need explicitly write a code in such a way that files are stored in two separate Blob storage account.
Anyone with access to your storage account's key (primary or secondary; there are two keys for a storage account) can manipulate the storage account in any way they see fit. The only way to ensure nothing happens? Don't give anyone access to the key(s). If you place the storage account within a resource group that only you have permissions on, you'll at least prevent others with access to the subscription from discovering the storage account and accessing it.
Within the subscription itself, you can place a lock on the actual resource (the storage account), so that nobody with access to the subscription accidentally deletes the entire storage account.
Note: with storage account keys, you do have the ability to regenerate the keys at any time. So if you ever suspected a key was compromised, you can perform a re-gen action.
Backups
There are several backup solutions offered for blob storage in case if containers get deleted.more product info can be found here:https://azure.microsoft.com/en-us/services/backup/
Redundancy
If you are concerned about availability, "The data in your Microsoft Azure storage account is always replicated to ensure durability and high availability. Replication copies your data, either within the same data center, or to a second data center, depending on which replication option you choose." , there are several replication options:
Locally redundant storage (LRS)
Zone-redundant storage (ZRS)
Geo-redundant storage (GRS)
Read-access geo-redundant storage (RA-GRS)
More details can be found here:
https://learn.microsoft.com/en-us/azure/storage/common/storage-redundancy
Managing Access
Finally, managing access to your storage account would be the best way to secure and ensure you'll avoid any loss on your data. You can provide read access only if you don't want anyone to delete files,folders etc.. through the use of SAS: Shared Access Signatures, allows you to create policies and provide access based on Read, Write, List, Delete, etc.. A quick GIF demo can be seen here: https://azure.microsoft.com/en-us/updates/manage-stored-access-policies-for-storage-accounts-from-within-the-azure-portal/
We are using blob to store documents and for documents management.
To prevent deletion of the blob, you can now enable soft deletion as described in here:
https://azure.microsoft.com/en-us/blog/soft-delete-for-azure-storage-blobs-ga/
You can also create your own automation around powershell,azcopy to do incremental and full backups.
The last element would be to use RA-GRS blobs where you can read from a secondary blob in read mode in another region in case the data center goes down.
Designing Highly Available Applications using RA-GRS
https://learn.microsoft.com/en-us/azure/storage/common/storage-designing-ha-apps-with-ragrs?toc=%2fazure%2fstorage%2fqueues%2ftoc.json
Use Microsoft's Azure Storage Explorer. It will allow you to download the full contents of blob containers including folders and subfolders with blobs. Conversely, you can upload to containers in the same way. Simple and free!
I've searched the web and contacted technical support yet no one seems to be able to give me a straight answer on whether items in Azure Blob Storage are backed up or not.
What I mean is, do I need to create a twin storage account as a "backup" and program copies of all content from one storage to another, or are the contents of a client's Blob Storage automatically redundantly backed up by Microsoft?
I know with AWS, storage is redundantly backed up via onsite drives as well as across other nodes in the cluster.
do I need to create a twin storage account as a "backup" and program
copies of all content from one storage to another, or are the contents
of a client's Blob Storage automatically redundantly backed up by
Microsoft?
Yes, you will need to do backup manually. Azure Storage does not back up the contents of your storage account automatically.
Azure Storage does provide geo-redundant replication (provided you configure the redundancy level for your storage account as GRS or RA-GRS) but that is not back up. Once you delete content from your primary account (location, it will automatically be removed from secondary account (geo-redundant location).
Both AWS (EBS) and Azure(Blob Storage) options provides durability by replicating the data across different data centers. This is for the high availability and durability of the data to provide the guarantee by the cloud provider.
In order to ensure that your data is durable, Azure Storage has the
ability to keep (and manage) multiple copies of your data. This is
called replication, or sometimes redundancy. When you set up your
storage account, you select a replication type. In most cases, this
setting can be modified after the storage account is set up.
For more details refer the replication section in documentation.
If you need to capture changes to the storage and allow restore to previous versions (e.g In situations like data corruption or application feature requirements like restore points, backups), you need to take a SnapShot manually. This is common for both AWS and Azure.
For more details on creating a Snapshot of Blob in Azure refer the documentation.
We have written an application which writes media to and reads media from Azure Blob Storage.
I feel this may be a stupid question but is Azure Blob storage backed up?
Just wondering whether it is necessary to set this up explicitly?
Simple answer to your question is No. Azure Storage does not create back up of your data. This is something you would need to do.
#Pradeep mentions about data replication which is true but please do not confuse data replication with backup. Data replication is NOT data backup.
While it is true that a minimum of 3 copies of your blobs are maintained however if you delete the blob, all 3 copies are removed immediately.
Note: The data in your Microsoft Azure storage account is always replicated to ensure durability and high availability.
Replication copies your data, either within the same data center, or to a second data center, depending on which replication option you choose. Replication protects your data and preserves your application up-time in the event of transient hardware failures. If your data is replicated to a second data center, it's protected from a catastrophic failure in the primary location.
When you create a storage account, you can select one of the following replication options:
Locally redundant storage (LRS)
Zone-redundant storage (ZRS)
Geo-redundant storage (GRS)
Read-access geo-redundant storage (RA-GRS)
For more details, refer “Azure Storage replication”.