I want to implement end to end encryption for my azure vm. According to the documentation encryption at host is the solution for data encryption at rest on a host machine. The other option is possibly Azure Disk Encryption.
The solutions are mutually exclusive:
Azure Disk Encryption cannot be enabled on disks that have encryption at host enabled.
The question is what are similarities and differences between both solutions and what are the arguments to use one instead of the other.
There are significant differences in the operation of both technologies. The two cannot both be used simultaneously on the same resources/VMs.
Azure Disk Encryption:
In more of less words, this is encryption at rest on the disks connected to your VM. OS and Data. It works with BitLocker on Windows and Linux machines and stores encryption keys and secrets within Azure Key Vault.
There are also restrictions on the VM SKUs that are supported. There are additional restrictions to review.
https://learn.microsoft.com/en-us/azure/virtual-machines/windows/disk-encryption-overview
https://learn.microsoft.com/en-us/azure/virtual-machines/linux/disk-encryption-overview
Encryption at host:
Encryption starts and happens on data processed by the host VM itself. The encrypted processed data is then sent back to the storage location – disk, table, blob – and then stored in its encrypted format. Therefore, the data processed and shared between the VM and its disks or other storage counterparts are encrypted, including in-transit.
https://learn.microsoft.com/en-us/azure/virtual-machines/disks-enable-host-based-encryption-portal
Note the list of supported VM sizes. There does remain a significant amount of limitations for host-based encryption.
Your subscription also needs to have this feature enabled by requesting it through the Azure Portal. Instructions are in the link provided under Prerequisites.
Related
My requirement is to encrypt newly added files in data lake storage container. For this I am creating an event grid that will trigger when a new file is added to a specified container. On the end side, I want to create a function that will encrypt the file that was added.
How to write code to create that function? or is there any alternate way?
Azure Storage automatically encrypts all data in a storage account at the service level using 256-bit AES encryption, one of the strongest block ciphers available, and is FIPS 140-2 compliant. Customers who require higher levels of assurance that their data is secure can also enable 256-bit AES encryption at the Azure Storage infrastructure level for double encryption. Double encryption of Azure Storage data protects against a scenario where one of the encryption algorithms or keys may be compromised. In this scenario, the additional layer of encryption continues to protect your data.
Infrastructure encryption can be enabled for the entire storage account, or for an encryption scope within an account. When infrastructure encryption is enabled for a storage account or an encryption scope, data is encrypted twice — once at the service level and once at the infrastructure level — with two different encryption algorithms and two different keys.
Service-level encryption supports the use of either Microsoft-managed keys or customer-managed keys with Azure Key Vault or Key Vault Managed Hardware Security Model (HSM) (preview). Infrastructure-level encryption relies on Microsoft-managed keys and always uses a separate key.
https://learn.microsoft.com/en-us/azure/storage/common/infrastructure-encryption-enable?tabs=portal
My issue
I try to understand how VM Disk Encryption work on Azure VM.
I have a Key Vault and a Disk Encryption set configured
What I did - 1
I create a VM with a disk selecting Encryption type = Encryption at rest with a custom managed key.
I select my Key Encryption set.
Portal confirms by displaying SSE with CMK in VM/Disks.
But:
Get-AzVMDiskEncryptionStatus says OS volume and DataVolumes are NotEncrypted.
What I did - 2
I create another VM, but this time I go to VM/Disks/Additional Settings and select Disk to encrypt.
This time the only option is a Key Vault, not a Disk Encryption Set.
I select OS and Data Disks.
Portal says:
This time, Get-AzVMDiskEncryptionStatus says:
My questions
What is the difference between the two methods?
Why Get-AzVMDiskEncryptionStatus return so un expected results?
To answer your questions:
There are Server-side encryption and Azure disk encryption. The first one of what you did is the Server-side encryption and the second one is Azure Disk encryption.
Most Azure managed disks are encrypted with Azure Storage encryption, which uses server-side encryption (SSE) to protect your data. Azure Storage Encryption automatically encrypts your data stored on Azure managed disks (OS and data disks) at rest by default when persisting it to the cloud. Disks with encryption at host enabled, however, are not encrypted through Azure Storage. When you enable encryption at the host, that encryption starts on the VM host itself, the Azure server that your VM is allocated to. After enabling encryption at the host, all this data is encrypted at rest and flows encrypted to the Storage service, where it is persisted. Essentially, encryption at the host encrypts your data from end-to-end.
Azure Disk Encryption leverages either the DM-Crypt feature of Linux or the BitLocker feature of Windows to encrypt managed disks with customer-managed keys within the guest VM. Server-side encryption with customer-managed keys improves on ADE by enabling you to use any OS types and images for your VMs by encrypting data in the Storage service.
For more information, read here.
It's expected. The Get-AzureRmVMDiskEncryptionStatus cmdlet gets the encryption status of the virtual machine. It displays the encryption status of the operating system and data volumes. This does not get server-side encrytion status.
Can data stored in Azure using BYOK (storage accounts, databases etc) be technically (rather than contractually) assured to be not to be access even from Microsoft? For example we assume the HSM key has been securely transferred to HSM backed Key Vault. How can application write to and read from storage using BYOK without Microsoft being able to peek in configuration, or in memory process, or while data is saved to storage?
In public preview now you can use Managed HSM (MHSM). You can provision an MHSM similar to a Key Vault (KV), but to activate and use it you need to set up 3 or more keys to download a security domain from the HSM. Microsoft has no access to decrypt the key - only a quorum of the 3 or more public keys you uploaded. While a bit specific to our testing environment, we have a script that shows how we create certificates and download the security domain using those public keys in order to test MHSM.
You can use the existing Key Vault SDKs and tools like the Azure CLI to access MHSM just like you would KV. For the Azure CLI you need to pass --hsm-name instead of --vault-name, but otherwise works the same for keys.
We are soon releasing another beta of the Azure SDKs for .NET, Java, JavaScript, and Python that support other algorithms supported by MHSM (AES-CBC, AES-CBC-PAD, and AES-GCM). Check out our blog for announcements.
We have migrated our Azure Classic storage accounts to Azure Resource Manager storage accounts. We migrated just under a week ago and data which wasn't encrypted before is still not encrypted.
We were under the impression that after migrating there would be a background task which would go through and encrypt any data in the storage account which was not already encrypted.
Is this true?
Yes, it is true. Data stored in the storage account prior to enabling the Storage Service Encryption will retroactively get encrypted. The data may not be immediately encrypted but the system background encryption process will encrypt the data, and it may take months for the process to encrypt the data.
I am developing a site that stores text in Azure Blob Storage. The text may be sensitive (not necessarily passwords, but personal information). I am trying to decide whether or not I should encrypt the text before I store it in Azure Blob Storage. My understanding is that this could mitigate a risk of exposing the data should the Azure key and account name get out and a malicious user download the blob. My questions are:
Are Azure Blobs already being encrypted when they land on disk at Microsoft? Is the account key used as an encryption key, or just an access token?
IF I were to do this in Azure Websites by using the .NET AES algorithm, where should I store the encryption key(s) or passphrase/salt used to generate a key? (ie is web.config an ok place for this?)
Blob content is not encrypted; that step would be completely up to you. Blob access is strictly controlled by access key (and there are two keys: primary and secondary, both working equally). Here are my thoughts on this:
If Storage access is exclusive to your app tier (that is, the key is never exposed outside of your app), risk is fairly low (vs. embedding the key in a desktop or mobile app, or using it with online storage browser services). Someone would need to steal the key from you somehow (like stealing source / config files). You mentioned using Websites, which doesn't provide RDP access, further protecting your running code.
If, somehow, your key were compromised, you can invalidate the key by generating a new one. This immediately cuts off access to anyone holding the old key. As a general pattern, when I use external tools (such as the Cerebrata tool), I always use my secondary key, reserving my primary key for my app. That way, I can always invalidate my secondary key as often as I like, preventing these tools from accessing my storage but not interfering with my running apps.
If you need to expose specific blobs to your customers, you have two ways to do it. First, you can download the blob to your web server, and then stream content down. Second: You can generate a Shared Access Signature (SAS) for the specific blob, and then give that resultant URI to the user (e.g. as the href of of an <a> tag). By using SAS, you permit access to a private blob for a given amount of time, like 10-20 minutes. Even if someone took an SAS URL and posted it on the Internet, it would only be valid for the time window you specified (it's hashed, preventing modification).
Consider multiple storage accounts for multiple apps (or even per app). This way, if there were a security breach, damage is limited to the specific compromised storage account.
EDIT April 2016
Azure Storage Service encryption for data at rest, just announced, is now in preview and available for any storage account created via the Azure Resource Manager (ARM). It is not available for "Classic" storage accounts (the rest of my answer, above, still applies). You can enable/disable encryption via the portal, for your storage account:
The service is available for blobs in both standard and premium storage accounts. More details are in this post.
David's answer is spot-on, but for people looking to actually implement the encryption the poster asked about, I've put together some samples and libraries at Azure Encryption Extensions.