I have been facing issues with understanding what would be the best way to store encrypted data in Azure Tables. The main objective here is to avoid someone with access to database to be able to read that data on the storage explorer.
One approach I have looked at is encrypting it on our server logic before saving it to the db, but the solution is causing a performance hit on the application.
Is there a way to achieve this directly on the Azure Tables? If it isn't, what else would be the best way to achieve this?
Data in Azure Storage is encrypted at rest by default. The scenario you're describing probably fits best with this option:
[...] create a storage account that relies on a key that is scoped to the account. When the account is first created, Microsoft uses the account key to encrypt the data in the account, and Microsoft manages the key. You can subsequently configure customer-managed keys for the account to take advantage of those benefits, including the ability to provide your own keys, update the key version, rotate the keys, and revoke access controls.
Source: Create an account that supports customer-managed keys for tables and queues
Related
Can data stored in Azure using BYOK (storage accounts, databases etc) be technically (rather than contractually) assured to be not to be access even from Microsoft? For example we assume the HSM key has been securely transferred to HSM backed Key Vault. How can application write to and read from storage using BYOK without Microsoft being able to peek in configuration, or in memory process, or while data is saved to storage?
In public preview now you can use Managed HSM (MHSM). You can provision an MHSM similar to a Key Vault (KV), but to activate and use it you need to set up 3 or more keys to download a security domain from the HSM. Microsoft has no access to decrypt the key - only a quorum of the 3 or more public keys you uploaded. While a bit specific to our testing environment, we have a script that shows how we create certificates and download the security domain using those public keys in order to test MHSM.
You can use the existing Key Vault SDKs and tools like the Azure CLI to access MHSM just like you would KV. For the Azure CLI you need to pass --hsm-name instead of --vault-name, but otherwise works the same for keys.
We are soon releasing another beta of the Azure SDKs for .NET, Java, JavaScript, and Python that support other algorithms supported by MHSM (AES-CBC, AES-CBC-PAD, and AES-GCM). Check out our blog for announcements.
I have sensitive data in my application, just because encryption is not available with the Azure Table Storage, I can't store the sensitive data into it.
As of 31 August 2017 the Azure storage service encrypts all data at rest using 256 bit AES for blobs, files, tables and queues as per this post Announcing Default Encryption for Azure Blobs, Files, Table and Queue Storage. This has also been applied to any storage accounts created before this date.
This is automatic and cannot be disabled but you can choose to use your own encryption keys instead of the Microsoft managed ones. It applies to both standard and premium performance tiers and Azure Resource Manager and classic deployment models.
Docs here Azure Storage Service Encryption for Data at Rest
If you have secure data then it's better to encrypt, you can encrypt with Azure Key Vault one more link
Basically, in your entity, you will mark properties with attribute
[EncryptProperty]
public string EncryptedProperty1 { get; set; }
The benefit would be that you will always communicate with encrypted data, even if somebody, somehow get access to storage he will not be able to extract data.
I guess that depends on how you're managing your data. If only parts of your data need to be encrypted and the rest can remain clear then you could potentially encrypt those sensitive fields in your app before storing them in Table Storage. You'd have to implement the encryption yourself, of course, but you should be able to find plenty of libraries to help with that.
If you had to encrypt everything before storing I don't think it would work well. Table Storage is key:value pairs and all your values would be encrypted, rendering Table Storage useless for any kind of lookup. You'd have to read in all your data, sort and index it in your app, then get on with your regular work.
I deploy Asp.Net Core web app and I need to build a storage for private keys of my clients (it is a lot of values). What should I use: Azure Key Vault or Data Protection APIs?
The second seems more easy to programming, however there is information from docs:
The ASP.NET Core data protection APIs are not primarily intended for
indefinite persistence of confidential payloads...
But I need to store keys long-term.
If you're using the keys to protect data for long term storage, I would advise you to use Azure Key vault.
Azure Key vault is a high availability service designed for storage of secrets and keys. Keys and secrets are automatically copied to Key vault instances in multiple regions and easily backed up securely using PowerShell cmdlets. You can store them in an HSM if you are dealing with highly sensitive data.
The Data Protection APIs are more designed to protect local or ephemeral data.
You should use Azure Key Vault to store your keys. The Data Protection API is always used in your application, for example it is used to encrypt and secure your session cookie.
But don't forget that you also do need to configure and store the data protection keys in a secure place. If you don't do it properly, then uses might be kicked out of your site when you redeploy.
See this document for more details:
Key storage providers in ASP.NET Core
If you do want to store the Data Protection Key ring in AKZ, then check out my implementation here:
Storing the ASP.NET Core Data Protection Key Ring in Azure Key Vault
I am developing a site that stores text in Azure Blob Storage. The text may be sensitive (not necessarily passwords, but personal information). I am trying to decide whether or not I should encrypt the text before I store it in Azure Blob Storage. My understanding is that this could mitigate a risk of exposing the data should the Azure key and account name get out and a malicious user download the blob. My questions are:
Are Azure Blobs already being encrypted when they land on disk at Microsoft? Is the account key used as an encryption key, or just an access token?
IF I were to do this in Azure Websites by using the .NET AES algorithm, where should I store the encryption key(s) or passphrase/salt used to generate a key? (ie is web.config an ok place for this?)
Blob content is not encrypted; that step would be completely up to you. Blob access is strictly controlled by access key (and there are two keys: primary and secondary, both working equally). Here are my thoughts on this:
If Storage access is exclusive to your app tier (that is, the key is never exposed outside of your app), risk is fairly low (vs. embedding the key in a desktop or mobile app, or using it with online storage browser services). Someone would need to steal the key from you somehow (like stealing source / config files). You mentioned using Websites, which doesn't provide RDP access, further protecting your running code.
If, somehow, your key were compromised, you can invalidate the key by generating a new one. This immediately cuts off access to anyone holding the old key. As a general pattern, when I use external tools (such as the Cerebrata tool), I always use my secondary key, reserving my primary key for my app. That way, I can always invalidate my secondary key as often as I like, preventing these tools from accessing my storage but not interfering with my running apps.
If you need to expose specific blobs to your customers, you have two ways to do it. First, you can download the blob to your web server, and then stream content down. Second: You can generate a Shared Access Signature (SAS) for the specific blob, and then give that resultant URI to the user (e.g. as the href of of an <a> tag). By using SAS, you permit access to a private blob for a given amount of time, like 10-20 minutes. Even if someone took an SAS URL and posted it on the Internet, it would only be valid for the time window you specified (it's hashed, preventing modification).
Consider multiple storage accounts for multiple apps (or even per app). This way, if there were a security breach, damage is limited to the specific compromised storage account.
EDIT April 2016
Azure Storage Service encryption for data at rest, just announced, is now in preview and available for any storage account created via the Azure Resource Manager (ARM). It is not available for "Classic" storage accounts (the rest of my answer, above, still applies). You can enable/disable encryption via the portal, for your storage account:
The service is available for blobs in both standard and premium storage accounts. More details are in this post.
David's answer is spot-on, but for people looking to actually implement the encryption the poster asked about, I've put together some samples and libraries at Azure Encryption Extensions.
I'm a bit confused about how to store keys (for data encryption) in Windows Azure.
According to the following two links (#1, #2), it is recommended to store the keys/key library in the Windows Azure Storage:
Storing your own key library within the Windows Azure Storage services is a good way to persist some secret information since you can rely on this data being secure in the multi-tenant environment and secured by your own storage keys.
But the "Security Best Practices For Developing Windows Azure Applications" (#3) recommends NOT to store any key related material in Windows Azure:
Also, developers should not upload
the key or any keying material to Windows Azure Storage, regardless of how careful they are about hiding it. If
any computer or storage services were compromised, it could lead to encryption keys being exposed.
What is the best approach to store keys for encryption in Windows Azure?
You'll see from my comment in that first link that I agree with your concerns. :)
Azure has no secure way of storing keys other than it's own Certificate Storage. Here is an article on using this method:
Field Note: Using Certificate-Based Encryption in Windows Azure Applications
You'll notice I've also commented on that article's shortcomings too, linking to this question:
Read azure ServiceConfiguration file's certificate section using c#
An example of using Azure's built in certificate storage to encrypt AES keys (avoiding the RSA restrictions on encrypted data length, while keeping the AES key secure) can be found in this project:
Codeplex: Azure Table Encryption via Attribute
The SymmetricKeyHelper class in the EncryptDecrypt project is of particular interest.
Kudos to #breischl for mentioning it, and for his contributions to the project.
The Azure Key vault service that has been released recently might be a perfect fit for the problem. This has been introduced so that keys can be managed in a central place and access can be easily controlled. It also supports HSM-backed service making it very secure.
Here is a artice on Getting Started with Azure Key Vault
For future Googlers - I've implemented the solution that Stuart Pegg describes above, but decoupled from Azure Tables.
See https://www.fasterweb.io/Blog/two-way-encryption-for-azure-web-roles for a writeup, or https://gist.github.com/strommen/20905504949072fe5e16 for just the code.
There’s always a risk. If someone gains access to your storage account using any means (such as using a tool), they may be able to find out your key. So in the end, it is needed to protect the storage account itself from accessed by unauthorized access.
For example, please do not allow a developer to access the production storage account. This includes don’t allow them to access the account using tools. Please protect the storage account key and do not leak any information in any application.
Only storage administrators (and developers who you 100% trust) can have full access to the production storage account. Then you’re safe to store the key in your storage account.
I know that this may be a bit late, but if anyone is looking for a quick and easy implementation of encryption for Azure Websites, I've created a (Azure.Security and the source code is currently on GitHub. The project is loosely based on the Codeplex: Azure Table Encryption via Attribute project but it is a lot more straightforward and easy to use. A blog post will follow shortly with instructions on how to set it up and use it.