I am developing a site that stores text in Azure Blob Storage. The text may be sensitive (not necessarily passwords, but personal information). I am trying to decide whether or not I should encrypt the text before I store it in Azure Blob Storage. My understanding is that this could mitigate a risk of exposing the data should the Azure key and account name get out and a malicious user download the blob. My questions are:
Are Azure Blobs already being encrypted when they land on disk at Microsoft? Is the account key used as an encryption key, or just an access token?
IF I were to do this in Azure Websites by using the .NET AES algorithm, where should I store the encryption key(s) or passphrase/salt used to generate a key? (ie is web.config an ok place for this?)
Blob content is not encrypted; that step would be completely up to you. Blob access is strictly controlled by access key (and there are two keys: primary and secondary, both working equally). Here are my thoughts on this:
If Storage access is exclusive to your app tier (that is, the key is never exposed outside of your app), risk is fairly low (vs. embedding the key in a desktop or mobile app, or using it with online storage browser services). Someone would need to steal the key from you somehow (like stealing source / config files). You mentioned using Websites, which doesn't provide RDP access, further protecting your running code.
If, somehow, your key were compromised, you can invalidate the key by generating a new one. This immediately cuts off access to anyone holding the old key. As a general pattern, when I use external tools (such as the Cerebrata tool), I always use my secondary key, reserving my primary key for my app. That way, I can always invalidate my secondary key as often as I like, preventing these tools from accessing my storage but not interfering with my running apps.
If you need to expose specific blobs to your customers, you have two ways to do it. First, you can download the blob to your web server, and then stream content down. Second: You can generate a Shared Access Signature (SAS) for the specific blob, and then give that resultant URI to the user (e.g. as the href of of an <a> tag). By using SAS, you permit access to a private blob for a given amount of time, like 10-20 minutes. Even if someone took an SAS URL and posted it on the Internet, it would only be valid for the time window you specified (it's hashed, preventing modification).
Consider multiple storage accounts for multiple apps (or even per app). This way, if there were a security breach, damage is limited to the specific compromised storage account.
EDIT April 2016
Azure Storage Service encryption for data at rest, just announced, is now in preview and available for any storage account created via the Azure Resource Manager (ARM). It is not available for "Classic" storage accounts (the rest of my answer, above, still applies). You can enable/disable encryption via the portal, for your storage account:
The service is available for blobs in both standard and premium storage accounts. More details are in this post.
David's answer is spot-on, but for people looking to actually implement the encryption the poster asked about, I've put together some samples and libraries at Azure Encryption Extensions.
Related
Can data stored in Azure using BYOK (storage accounts, databases etc) be technically (rather than contractually) assured to be not to be access even from Microsoft? For example we assume the HSM key has been securely transferred to HSM backed Key Vault. How can application write to and read from storage using BYOK without Microsoft being able to peek in configuration, or in memory process, or while data is saved to storage?
In public preview now you can use Managed HSM (MHSM). You can provision an MHSM similar to a Key Vault (KV), but to activate and use it you need to set up 3 or more keys to download a security domain from the HSM. Microsoft has no access to decrypt the key - only a quorum of the 3 or more public keys you uploaded. While a bit specific to our testing environment, we have a script that shows how we create certificates and download the security domain using those public keys in order to test MHSM.
You can use the existing Key Vault SDKs and tools like the Azure CLI to access MHSM just like you would KV. For the Azure CLI you need to pass --hsm-name instead of --vault-name, but otherwise works the same for keys.
We are soon releasing another beta of the Azure SDKs for .NET, Java, JavaScript, and Python that support other algorithms supported by MHSM (AES-CBC, AES-CBC-PAD, and AES-GCM). Check out our blog for announcements.
I have sensitive data in my application, just because encryption is not available with the Azure Table Storage, I can't store the sensitive data into it.
As of 31 August 2017 the Azure storage service encrypts all data at rest using 256 bit AES for blobs, files, tables and queues as per this post Announcing Default Encryption for Azure Blobs, Files, Table and Queue Storage. This has also been applied to any storage accounts created before this date.
This is automatic and cannot be disabled but you can choose to use your own encryption keys instead of the Microsoft managed ones. It applies to both standard and premium performance tiers and Azure Resource Manager and classic deployment models.
Docs here Azure Storage Service Encryption for Data at Rest
If you have secure data then it's better to encrypt, you can encrypt with Azure Key Vault one more link
Basically, in your entity, you will mark properties with attribute
[EncryptProperty]
public string EncryptedProperty1 { get; set; }
The benefit would be that you will always communicate with encrypted data, even if somebody, somehow get access to storage he will not be able to extract data.
I guess that depends on how you're managing your data. If only parts of your data need to be encrypted and the rest can remain clear then you could potentially encrypt those sensitive fields in your app before storing them in Table Storage. You'd have to implement the encryption yourself, of course, but you should be able to find plenty of libraries to help with that.
If you had to encrypt everything before storing I don't think it would work well. Table Storage is key:value pairs and all your values would be encrypted, rendering Table Storage useless for any kind of lookup. You'd have to read in all your data, sort and index it in your app, then get on with your regular work.
I'm a bit confused about how to store keys (for data encryption) in Windows Azure.
According to the following two links (#1, #2), it is recommended to store the keys/key library in the Windows Azure Storage:
Storing your own key library within the Windows Azure Storage services is a good way to persist some secret information since you can rely on this data being secure in the multi-tenant environment and secured by your own storage keys.
But the "Security Best Practices For Developing Windows Azure Applications" (#3) recommends NOT to store any key related material in Windows Azure:
Also, developers should not upload
the key or any keying material to Windows Azure Storage, regardless of how careful they are about hiding it. If
any computer or storage services were compromised, it could lead to encryption keys being exposed.
What is the best approach to store keys for encryption in Windows Azure?
You'll see from my comment in that first link that I agree with your concerns. :)
Azure has no secure way of storing keys other than it's own Certificate Storage. Here is an article on using this method:
Field Note: Using Certificate-Based Encryption in Windows Azure Applications
You'll notice I've also commented on that article's shortcomings too, linking to this question:
Read azure ServiceConfiguration file's certificate section using c#
An example of using Azure's built in certificate storage to encrypt AES keys (avoiding the RSA restrictions on encrypted data length, while keeping the AES key secure) can be found in this project:
Codeplex: Azure Table Encryption via Attribute
The SymmetricKeyHelper class in the EncryptDecrypt project is of particular interest.
Kudos to #breischl for mentioning it, and for his contributions to the project.
The Azure Key vault service that has been released recently might be a perfect fit for the problem. This has been introduced so that keys can be managed in a central place and access can be easily controlled. It also supports HSM-backed service making it very secure.
Here is a artice on Getting Started with Azure Key Vault
For future Googlers - I've implemented the solution that Stuart Pegg describes above, but decoupled from Azure Tables.
See https://www.fasterweb.io/Blog/two-way-encryption-for-azure-web-roles for a writeup, or https://gist.github.com/strommen/20905504949072fe5e16 for just the code.
There’s always a risk. If someone gains access to your storage account using any means (such as using a tool), they may be able to find out your key. So in the end, it is needed to protect the storage account itself from accessed by unauthorized access.
For example, please do not allow a developer to access the production storage account. This includes don’t allow them to access the account using tools. Please protect the storage account key and do not leak any information in any application.
Only storage administrators (and developers who you 100% trust) can have full access to the production storage account. Then you’re safe to store the key in your storage account.
I know that this may be a bit late, but if anyone is looking for a quick and easy implementation of encryption for Azure Websites, I've created a (Azure.Security and the source code is currently on GitHub. The project is loosely based on the Codeplex: Azure Table Encryption via Attribute project but it is a lot more straightforward and easy to use. A blog post will follow shortly with instructions on how to set it up and use it.
I need to store multiple files that users upload, and then provide these users with the capability of accessing their files via http. There are two key considerations:
- Storage (which is my primary concern here)
- Security (which let's leave aside for now)
The question is:
What is the most cost efficient and performant way of storing all these files and giving access to them later? I believe the answer is:
- Store files within Azure Storage Account, and have a key that references them in an SQL Azure database.
I am correct on this?
Is a blob storage flat? Or can I create something like folders inside it to better organize my files?
The idea of using SQL Azure to store metadata for your blobs is a pretty common scenario, which allows you to take advantage of SQL for searching, and blobs for storage.
Blobs are organized by container. So you'd have something like:
http://mystorage.blob.core.windows.net/mycontainer/myfile.doc
You can also simulate a hierarchy using a delimiter, but in reality there's just container plus blob.
If you keep the container or blob private, the user would either have to go through your web front end (or web service), or you'd have to provide them with a special URL with a Shared Access Signature appended, which is a time-limited URL.
I would recommend you to take a look at BlobShare Sample which is a simple file sharing application that demonstrates the storage services of the Windows Azure Platform, together with the authentication and authorization capabilities of Access Control Service (ACS). The full sample code is located at following link:
http://blobshare.codeplex.com/
You can use this sample code immediately, just by adding proper reference to your Windows Azure Account credentials. The best thing with this sample is that you can provide blob access directly through Access Control Services. You can also modify the code to add SAS support as well as blob download from public containers. Once you have it working and understood the concept you can tweak to make it the way you would want.
I have registered a Free-Trial account on Azure site.
In documentation I found:
http://msdn.microsoft.com/en-us/library/windowsazure/dd179428.aspx
that I need some Shared Keys to use APIs.
But where are they located on Azure Management Portal?
I could not find them at all.
Do I need a paid account for this?
If you open the Azure portal, click on the "Hosted Servcies, Storage Accounts & CDN" link on the lower left and then pick "Storage Accounts". Once you click on "New Storage Account" and create a storage account, you will see "Primary access key" and "Seconardy access key" on the right hand side if you select that storage account in the middle.
More info:
Creating a storage account: http://msdn.microsoft.com/en-us/library/windowsazure/gg433066.aspx
Viewing keys: http://msdn.microsoft.com/en-us/library/windowsazure/hh531566.aspx
Note, if you are meaning Shared Access Signatures for blob storage, look at: http://blog.smarx.com/posts/new-storage-feature-signed-access-signatures
Each storage account you create will have a primary and secondary key, which is ultimately used in the REST header for accessing tables, blobs, and queues. If you're using .NET, Java, PHP or any other language where you've found a library wrapping the REST API, you won't have to worry about constructing REST headers; instead, you'll just create a storage endpoint using your primary or secondary key.
Making this easier, the key can be stored in your configuration settings. Then, if you ever change your storage account (maybe one for development, one for production), it's a simple matter of changing your configuration settings instead of changing code.
FYI - these keys are created automatically when you create a new storage account. And... you may use either key, and invalidate / re-create either key at any time (which is great if, say, you share your secondary key with a 3rd-party service and then want to terminate your relationship with that service).
I'd suggest grabbing the Windows Azure Training Kit and trying out a few of the first exercises, as you'll see exactly how to reference a storage account this way.