I need to build a web app that accesses some encrypted columns on a DB. All must be hosted in the client's azure account. I have searched for a couple of days and read a lot of tutorials but I can't find an answer to my problem.
I have mainly followed these:
https://learn.microsoft.com/en-us/azure/sql-database/sql-database-always-encrypted
http://www.bradleyschacht.com/always-encrypted-with-azure-key-vault/
I was able to run a web app on my machine with the certificate generated by SSMS encryption wizard and a SQL DB hosted on azure. I couldn't do it with an azure vault key.
Now I need to publish my web app on azure but I'm unable to access/modify the DB data. I need to either use the certificate from my machine or use the azure vault. Can anyone explain to me how it's done?
I tried to export the certificate to the azure vault, but I don't know how to "reference" it
I tried to create a new table on the db and encrypting it with a vault key, but I get:
Failed to decrypt a column encryption key. Invalid key store provider
name: 'AZURE_KEY_VAULT'. A key store provider name must denote either
a system key store provider or a registered custom key store provider.
Valid system key store provider names are: 'MSSQL_CERTIFICATE_STORE',
'MSSQL_CNG_STORE', 'MSSQL_CSP_PROVIDER'. Valid (currently registered)
custom key store provider names are: . Please verify key store
provider information in column master key definitions in the database,
and verify all custom key store providers used in your application are
registered properly."
I read somewhere that I need to give permission in the AD to my application, but I don't have permissions from my client (the owner of the Azure subscription) to do that.
I read also that a stored procedure must be used to read and write to the DB. Is this true?
Thanks in advance for any help.
I need to either use the certificate from my machine or use the azure
vault. Can anyone explain to me how it's done?
It depends on your use-case. Actually Selecting Keystore Provider for your Column Master key is depends on which driver and version you are using. There are two high-level categories of key stores : Read here
Local
Centralized Key Store
Local
If you planning to deploy your App in On-Prem/VM, then you can generate our own Certificate and keep the certificate within your Local VM.
Centralized Key Store
If you planning to deploy your App in azure web APP/Cloud then you should keep your Key Store in a centralized Secure Vault which may be here as Azure Key Vault
As a best practice, you should not store the provider in the Local machine, Which would be a problem if you VM is compromised then your DB certificate also be compromised.
I tried to export the certificate to the azure vault, but I don't know
how to "reference" it
CREATE COLUMN MASTER KEY [TESTMASTERKEY]
WITH
(
KEY_STORE_PROVIDER_NAME = N'AZURE_KEY_VAULT',
KEY_PATH = N'' --Paste your Key Identifier
)
GO
I tried to create a new table on the DB and encrypting it with a vault
key, but I get:
Always try to download the latest SSMS version.
Assume you are using Azure SQLDB. Always encryption will work only on SQL Server
2016 and above in on-prem and all versions of Azure SQLDB
Set the connection string to Column Encryption Setting=enabled
The behavior you describe is a bug in CTP 3.0 and SSMS October update. The issue, as you surmised, is that the Azure Key Vault provider is not registered if you open the Query Editor window opening the Always Encrypted wizard first. We’ve already fixed this for the next update of SSMS! In the meantime, the workaround is to open the Always Encrypted wizard (you can close it/cancel immediately after opening) which will cause the Azure Key Vault provider to get registered.
This bug manifests itself only through this specific case (using the Query Editor before the wizard), and won’t at all impact your ability to use the Always Encrypted wizard or use the Azure Key Vault provider with any of your client applications.
So try to download the latest SSMS version.
I read somewhere that I need to give permission in the AD to my
application, but I don't have permissions from my client (the owner of
the Azure subscription) to do that.
This is mainly for the Client side. You need to register your app in order to get the client id and client secret for your client-side application to talk with encrypted data in DB. Read here for how to register your client app. Unless you register your app, you couldn't able to connect from any client-side(Except SSMS). You need to contact the subscription owner to register the app.
I read also that a stored procedure must be used to read and write to
the DB. Is this true?
Depends on your Encryption Type. There are two types of Encryption Read here about it
Deterministic
Randomized
Each having its own pro and cons.
Deterministic encryption always generates the same encrypted value for any given plaintext value. Using deterministic encryption allows point lookups, equality joins, grouping and indexing on encrypted columns. However, but may also allow unauthorized users to guess information about encrypted values by examining patterns in the encrypted column, especially if there is a small set of possible encrypted values, such as True/False, or North/South/East/West region. Deterministic encryption must use a column collation with a binary2 sort order for character columns.
Randomized encryption uses a method that encrypts data in a less predictable manner. Randomized encryption is more secure, but prevents searching, grouping, indexing, and joining on encrypted columns.
Full explanation of all aspects related to this topic here: https://www.codeproject.com/Articles/5355073/Full-Tutorial-on-using-Always-Encrypted-with-Azure
I tried to cover in the article both legacy projects and new approaches and also transition phase.
Related
we used Encrypt and decrypt blobs using Azure Key Vault to protect our files from dev ops person or any unwanted access to files.
i have created the RSA key like below
now i have few doubts
#1 : if i set expiration date to this key,( let's say 2 year from today's date ) will it effect my encryption? -
for example, after 2 year, i will create new RSA key and old files which is already encrypted wont be able to decrypt ? if answer is no - how the version changed will make sure decryption keep working?
#2 : which RSA key size i have to use ? what is best as per industry standards?
#3 : in blob files, have metadata properties added by SDK : "encryptiondata" - what is that, and it include "EncryptedKey" also, what's that use?, seems like SDK is doing behind the process, when we set "BlobEncryptionPolicy"
#4 : when we set KEY to azure key vault - is it private key or public key? will we're able to see it's content ?
let's say someone got to know the RSA key from key vault in plain text..he/she will download encrypted files directly from blob and use that key in separate program and unlock/decrypt it?
Thanks,
#1 In Azure Keyvault the encryption keys don't have an expiry by default. But it is a good practice to set one. And then rotate the keys.
Rotation would involve Generate new key(s),
Re-encrypt all data that was encrypted using the old key, using new key(s)
Delete old encrypted data and old encrypted key.
Azure supports three models with respect to Data Encryption.
1)Server-side encryption using Service-Managed keys
2)Server-side encryption using customer-managed keys in Azure Key Vault
3) Server-side encryption using customer-managed keys on customer-controlled hardware
You can read more about that here. https://learn.microsoft.com/en-us/azure/security/fundamentals/encryption-models
and choose the option that you need.
If you need to bring your own keys in azure storage - https://learn.microsoft.com/en-us/azure/storage/common/customer-managed-keys-overview
Rotation process can be automated with Events, event grid, functions.
For example A secret near expiry gets triggered when the secret is near its expiry date and that is captured in an event grid and the necessary action is taken via an azure function who's trigger is that event grid mapping for this event.
Azure runbook based automation options are also possible.
#2 Bigger keysize is tougher or takes time to crack is what I know and from what I read 2048 or 4096 should be good. But again there are schools of thought on not using standard keysize etc. I guess you can consult a security/cryptography expert for the specifics.
#3 those properties refer to the default encryption at rest done in azure storage. Refer the data encryption models available for azure storage.
#4 In Azure keyvault a Cryptographic key is represented as JWK (JSON Web Key)
for Example a .pfx certificate file that contains a pair of public & private keys.
The API call to GetKeyAsync doesn't return private key data.This is why the DecryptAsync wrapper method does use the Key Vault API for decryption.
I'm try to create a Column Encryption in Azure(via Always Encrypted Wizard).I have one big doubt in these Scenario.
There are several other ways to store the CMK like Windows Store,Azure Key Vault.
In Windows Store there are two options User/LocalMachine,Where in User Options the CMK will be work for that particular Local Machine and that Particular User alone.(able to decrypt Using Column Encryption Setting=Enabled )
Where as Local Machine option can be work for all the Users in the Same Machine.(able to decrypt using Column Encryption Setting=Enabled)
My Doubt is ,When coming to Azure Key Vault for storing CMK who can able to decrypt the Encrypted Column ?
My understanding is
The User who have the access(Under Access Policies) to Azure Key Vault can only be able to decrypt the Encrypted Column ? Is that right ?
Please anyone guide me
Thanks in advance,
Jay
My understanding is The User who have the access(Under Access
Policies) to Azure Key Vault can only be able to decrypt the Encrypted
Column ? Is that right ?
Yes that is correct. You would need the following permissions : create,get,wrapKey,unwrapKey,sign,verify,list
Here are a couple of articles that will walk you through basic Always encrypted workflow using AKV
Article 1
Article 2
I want to increase my safety of my web app in case of an attack.
The following components are present in my system:
Azure Web App
Azure Blob Storage
Azure SQL Azure
Azure KeyVault
Now there is the scenario that the app encrypts and stores uploaded documents.
This works as described:
1) User Uploads doc to the web app
2) random encryption key is generated
3) random encryption key is stored to the azure key vault
4) sql azure stores the blob url and the key url
Now my question is:
How is using the key vault safer in case of hacking the web app instance? I mean there is the client id and client secret in the app.config to access the keyvault, we need it to read and write keys. So if i use key vault or not does not increase safety in terms of hacking the web app, right?
The Key Vault is an API wrapped around an HSM. What makes the Key Vault or HSM Secure is that the keys can not be extracted from them once imported / created. Also, the crypto (encrypt / decrypt in your case) operations happen inside the vault so the keys are never exposed, even in memory.
If someone was able to hack your web application and get the credentials to your key vault they could use the vault to decrypt the data. So, in this case you could regenerate the credentials for the Key Vault and still continue to use the same keys that are in the vault - because they were never exposed. Meaning any data that is encrypted that the attacker didn't already decrypt is still safe because the keys were never exposed.
Typically HSMs aren't designed to store a large number of keys in only a few really important keys. You might want to consider using a key wrapping solution where you have one key in the vault.
You probably want to encrypt the client id and client secret in your config and decrypt them at runtime - this adds another layer of security. Now the attacker either needs to read the keys out of your application memory while it is running on your Cloud Service / VM (not an easy task). Or the attacker would need to obtain the config file and the private key of the certificate used to encrypt your config values (easier than reading memory, but still requires a lot of access to your system).
So if i use key vault or not does not increase safety in terms of
hacking the web app, right?
It all depends at what level they were able to hack the site. In the case you describe, if they obtained your source code then - yes, its game over. But it doesn't have to be that way. It truly comes down to your configuration.
However, most of the time, developers forget that security is a layered approach. When you're talking about encryption of data and related subjects, they are generally a last line of defense. So if a malicious actors has acquired access to the encrypted sensitive data they have breached other vulnerable areas.
The problem is not Key Vaults but your solution of using client secret. Client secret is a constant string which is not considered safe. You can use certificate and thumbprint as a "client secret". Your application needs to read the .pfx file which is stored in web app, then decrypt to grab thumbprint. Once thumbprint is retrieved successfully then you Key Vault secret is retrievable. Moreover, in Key Vault you are given the ability to use your own certificate rather than just a masked string in Secret. This is so-called "nested encryption".
The hacker if getting access to your app.config, he get nothing than the path of .pfx file which he does not know where to store, even how it looks like. Generating the same pfx file becomes impossible. If he could he would break the entirely crypto world.
I am building an application that will run on Google App Engine (GAE). It will need access to data stored by the user in other systems (e.g. the user's Nest thermostat, Yahoo mail). The application running on GAE will allow the user to provide credentials for the other system. The application will store these credentials in Google Cloud (Datastore) for later use by an application running on Google Compute Engine on the users behalf. The application will also allow OAuth to allow the user to allow the application access the external system in the user's behalf. The application will need to store user credentials (username and passwords) or OAuth access tokens in the Google Cloud.
The application will need to encrypt the secrets before they are stored and be able to unencrypt the data to send it to the external systems. That is, the system will need to use symmetric encryption and therefor need to securely manage keys.
How can the application store these secrets in the Google Cloud Datastore (Datastore) securely? I think I am looking for something like the AWS CloudHSM for Google. That is, I would like to store each secret with a seed and key id and use the key id to get the key from a key management system. This implementation would also allow for key rotation and other standard security practices.
I think I am looking for a Google Cloud service or Google API that provides secrets management and only allows an app with the proper Google app identifier to access the secrets.
Is there a service within Google Cloud or Google APIs that will manage secrets? Is there another architecture that I should be considering?
By the way, the application uses Google Identity Toolkit (GitKit) to authenticate and authorize users to use the GAE hosted application. The application allows users to create accounts using either federate identities or username and passwords via GitKit.
Thanks,
chris
In the meantime, Google also added a Key Management Service: https://cloud.google.com/kms/
You could e.g. use it to encrypt your data before storing it in a database. Or, use KMS to encrypt an AES key to encrypt your data, and possibly keep a backup of your AES key somewhere in case you lose access to KMS.
App Identity Service might be what you are looking for https://cloud.google.com/appengine/docs/java/appidentity/#Java_Asserting_identity_to_other_systems
It lets you sign content with an application-specific private key, and provides rotating certificates to validate signed content.
So as far as I can tell the answer is that you can't. What you are looking for is an equivalent to KMS. That service let's you create and manage keys and do a bunch of your own crypto stuff. It's really cool and it will allow you to quickly do incredibly strong crypto with just a few simple lines of code. Azure has a similar service called KeyVault. It lacks automated key generation and rotation as far as I can tell, but other than that it's good. At the time of this response there was not an equivalent service for Google. They have an internal KMS which they used for crypto operations and you can provide your own keys, but that's pretty much it. Not quite the same thing that you get on KeyVault, and nothing like KMS.
That said there is hope. You can do one of two things:
Create a VPC and use an HSM from somewhere else. You could use RackSpace, or you could simply use AWS KMS. That sounds crazy but it's actually a good idea and the extra management is worth it. In general the most secure solution separate the keys from the encrypted data, particularly at rest. That means that keys in one data center and encrypted data stored in another data center is the most secure solution. That sounds like hard stuff, but thankfully I've made an opensource project which makes it very easy for you called KeyStor. With KeyStor you can get a data center that deals with encryption services set up in a day, no problem, and you can use AWS very cost effectively.
Set up your own cypto service, skip the HSM integration and simply be careful about who has access to the machines that maintain your keys. You can do this with KeyStor as well, and if KeyStor doesn't quite do what you want, that's why it's open-source. Take the code and build what you need to build.
You could store secrets in storage (e.g., in Datastore, Google Cloud Storage, or another storage system of your choice) and encrypt those with a key from Google's Cloud KMS.
Here's some documentation from Google on secret management, and here's a codelab on specifically encrypting data in Google Cloud Storage at the application layer using Cloud KMS.
For the Google Cloud managed service that provides the API for secure storage of secrets, see Google Cloud Secret Manager for more details.
Secret Manager is a secure and convenient storage system for API keys, passwords, certificates, and other sensitive data. Secret Manager provides a central place and single source of truth to manage, access, and audit secrets across Google Cloud.
Consider the following image that shows the encryption hierarchy used in SQL Server. Please note the first blue block, that says the SMK is encrypted using DPAPI. The DPAPI uses a currently logged-in user credentials (+ more) to encrypt data, so it's machine-specific. This means that SMK (as well as DMK and any derived password) will be machine-specific (actually it's generated by SQL Server's setup). OTOH, I can create/backup an X.509 certificate in SQL Server (using CREATE CERTIFICATE, BACKUP CERTIFICATE and so on).
The scenario/question:
I'm developing a Web App that needs to encrypt and store CC information in a database column. I need to access those data, later on another machine so the db backup should be actually readable when restored on another machine (albeit, for someone who has got access to the above-mentioned certificate).
I'm wondering how am I supposed to restore a backup on another machine when the SMK is specific to the current instance of SQL Server? What should I do to access those encrypted data once they are restored on another machine?
UPDATE: Correct me if I am wrong!
We could use the BACKUP SERVICE MASTER KEY TO FILE command to back the currently used SMK. This key, however, can be restored on any other SQLServer (on/out of the same machine) using the RESTORE SERVICE MASTER KEY FROM FILE command. When the SMK is restored, it's being encrypted once again using DPAPI so that the key itself can be stored somewhere on the machine.
Any help would be highly appreciated,
The diagram shows that a certificate can be protected either by the DMK or via a password. If you protect it with just a password, it should be portable.