Consider the following image that shows the encryption hierarchy used in SQL Server. Please note the first blue block, that says the SMK is encrypted using DPAPI. The DPAPI uses a currently logged-in user credentials (+ more) to encrypt data, so it's machine-specific. This means that SMK (as well as DMK and any derived password) will be machine-specific (actually it's generated by SQL Server's setup). OTOH, I can create/backup an X.509 certificate in SQL Server (using CREATE CERTIFICATE, BACKUP CERTIFICATE and so on).
The scenario/question:
I'm developing a Web App that needs to encrypt and store CC information in a database column. I need to access those data, later on another machine so the db backup should be actually readable when restored on another machine (albeit, for someone who has got access to the above-mentioned certificate).
I'm wondering how am I supposed to restore a backup on another machine when the SMK is specific to the current instance of SQL Server? What should I do to access those encrypted data once they are restored on another machine?
UPDATE: Correct me if I am wrong!
We could use the BACKUP SERVICE MASTER KEY TO FILE command to back the currently used SMK. This key, however, can be restored on any other SQLServer (on/out of the same machine) using the RESTORE SERVICE MASTER KEY FROM FILE command. When the SMK is restored, it's being encrypted once again using DPAPI so that the key itself can be stored somewhere on the machine.
Any help would be highly appreciated,
The diagram shows that a certificate can be protected either by the DMK or via a password. If you protect it with just a password, it should be portable.
Related
I need to build a web app that accesses some encrypted columns on a DB. All must be hosted in the client's azure account. I have searched for a couple of days and read a lot of tutorials but I can't find an answer to my problem.
I have mainly followed these:
https://learn.microsoft.com/en-us/azure/sql-database/sql-database-always-encrypted
http://www.bradleyschacht.com/always-encrypted-with-azure-key-vault/
I was able to run a web app on my machine with the certificate generated by SSMS encryption wizard and a SQL DB hosted on azure. I couldn't do it with an azure vault key.
Now I need to publish my web app on azure but I'm unable to access/modify the DB data. I need to either use the certificate from my machine or use the azure vault. Can anyone explain to me how it's done?
I tried to export the certificate to the azure vault, but I don't know how to "reference" it
I tried to create a new table on the db and encrypting it with a vault key, but I get:
Failed to decrypt a column encryption key. Invalid key store provider
name: 'AZURE_KEY_VAULT'. A key store provider name must denote either
a system key store provider or a registered custom key store provider.
Valid system key store provider names are: 'MSSQL_CERTIFICATE_STORE',
'MSSQL_CNG_STORE', 'MSSQL_CSP_PROVIDER'. Valid (currently registered)
custom key store provider names are: . Please verify key store
provider information in column master key definitions in the database,
and verify all custom key store providers used in your application are
registered properly."
I read somewhere that I need to give permission in the AD to my application, but I don't have permissions from my client (the owner of the Azure subscription) to do that.
I read also that a stored procedure must be used to read and write to the DB. Is this true?
Thanks in advance for any help.
I need to either use the certificate from my machine or use the azure
vault. Can anyone explain to me how it's done?
It depends on your use-case. Actually Selecting Keystore Provider for your Column Master key is depends on which driver and version you are using. There are two high-level categories of key stores : Read here
Local
Centralized Key Store
Local
If you planning to deploy your App in On-Prem/VM, then you can generate our own Certificate and keep the certificate within your Local VM.
Centralized Key Store
If you planning to deploy your App in azure web APP/Cloud then you should keep your Key Store in a centralized Secure Vault which may be here as Azure Key Vault
As a best practice, you should not store the provider in the Local machine, Which would be a problem if you VM is compromised then your DB certificate also be compromised.
I tried to export the certificate to the azure vault, but I don't know
how to "reference" it
CREATE COLUMN MASTER KEY [TESTMASTERKEY]
WITH
(
KEY_STORE_PROVIDER_NAME = N'AZURE_KEY_VAULT',
KEY_PATH = N'' --Paste your Key Identifier
)
GO
I tried to create a new table on the DB and encrypting it with a vault
key, but I get:
Always try to download the latest SSMS version.
Assume you are using Azure SQLDB. Always encryption will work only on SQL Server
2016 and above in on-prem and all versions of Azure SQLDB
Set the connection string to Column Encryption Setting=enabled
The behavior you describe is a bug in CTP 3.0 and SSMS October update. The issue, as you surmised, is that the Azure Key Vault provider is not registered if you open the Query Editor window opening the Always Encrypted wizard first. We’ve already fixed this for the next update of SSMS! In the meantime, the workaround is to open the Always Encrypted wizard (you can close it/cancel immediately after opening) which will cause the Azure Key Vault provider to get registered.
This bug manifests itself only through this specific case (using the Query Editor before the wizard), and won’t at all impact your ability to use the Always Encrypted wizard or use the Azure Key Vault provider with any of your client applications.
So try to download the latest SSMS version.
I read somewhere that I need to give permission in the AD to my
application, but I don't have permissions from my client (the owner of
the Azure subscription) to do that.
This is mainly for the Client side. You need to register your app in order to get the client id and client secret for your client-side application to talk with encrypted data in DB. Read here for how to register your client app. Unless you register your app, you couldn't able to connect from any client-side(Except SSMS). You need to contact the subscription owner to register the app.
I read also that a stored procedure must be used to read and write to
the DB. Is this true?
Depends on your Encryption Type. There are two types of Encryption Read here about it
Deterministic
Randomized
Each having its own pro and cons.
Deterministic encryption always generates the same encrypted value for any given plaintext value. Using deterministic encryption allows point lookups, equality joins, grouping and indexing on encrypted columns. However, but may also allow unauthorized users to guess information about encrypted values by examining patterns in the encrypted column, especially if there is a small set of possible encrypted values, such as True/False, or North/South/East/West region. Deterministic encryption must use a column collation with a binary2 sort order for character columns.
Randomized encryption uses a method that encrypts data in a less predictable manner. Randomized encryption is more secure, but prevents searching, grouping, indexing, and joining on encrypted columns.
Full explanation of all aspects related to this topic here: https://www.codeproject.com/Articles/5355073/Full-Tutorial-on-using-Always-Encrypted-with-Azure
I tried to cover in the article both legacy projects and new approaches and also transition phase.
I was doing some demo in SQL Server 2016 for topic Always encrypted. Got few doubts. Below are the steps followed:
In Database server (hosted in Microsoft Azure VM):
In table MyTable, Created the Column Encryption Key (CEK) and Master Encryption Key (CMK)
Select * from MyTable, shows encrypted data.(both from App and DB server)
Exported the certificate from Database Server
Imported the certificate in App Server (my Local machine)
Added Column Encryption Setting=Enabled to the connection string of my application.
It is working fine, now it shows the plain text data as expected.
Doubt:
In Database Server (in MS Azure VM), If a SysAdmin login (SQL Authentication) connects to SSMS with additional parameter Column Encryption Setting=Enabled, It is shows plain text data (expecting encrypted data). My understanding is, no one other then application users should see the plain text data). Can anyone please clarify?
In step 3 you mention that you export the certificate from the Database Server, to ensure maximum security, never store your certificate on the Database Server. The server does not need to have access to the certificate.
If a SysAdmin login (SQL Authentication) connects to SSMS with
additional parameter Column Encryption Setting=Enabled, It is shows
plain text data (expecting encrypted data). My understanding is, no
one other then application users should see the plain text data). Can
anyone please clarify?
If the SysAdmin is connecting to SSMS from a client machine that has the certificate and if the SysAdmin has permission to access the certificate, then they will see the plain text data.
Roughly speaking, Always Encrypted provides the following security guarantee, Plaintext data will only be visible to entities that have access to the ColumnMasterKey (Certificate)
To elaborate, Consider the following scenario.
Consider two machines:
MachineA: Machine on which SQL Server is running
MachineT: Client Machine.
Consider two users
UserA (this can technically be a group of users, but I will be considering a scenario with single user for simplicity): Who is an Administrator on MachineA, managing SQL server and is SysAdmin on SQL server. However, userA does not have any kind of access to MachineT and UserA should not be able to decrypt any encrypted data stored in SQL Server on Machine A (Encrypted data, in the context of this answer is data that is encrypted using Always Encrypted feature of SQL Server).
UserT (this can technically be a group of users, but I will be considering a scenario with single user for simplicity): Is a trusted user, has access to MachineT, has access to all data in database db which is hosted in SQL Server on MachineA. Also, since userT is trusted, he/she should be able to decrypt the encrypted data.
Consider SQL Server running on MachineA has database db and table t.
Our goal is to secure a column belonging to table t, say ssnCol, such that only userT should be able to see the ssnCol in plaintext.
The goal described above can be achieved using the following steps.
UserT logs into MachineT.
UserT opens SSMS in MachineT.
UserT connects to SQL Server on MachineA
UserT encrypts ssnCol in table t using the steps mentioned in the Encrypt columns (configure Always Encrypted) section of this article
After this step, the column ssnCol would be encrypted.
When userT encrypts ssnCol in the manner described above, two keys are generated
CMK: CMK aka column master key is the key that is used to encrypt CEK/s. This key is stored in the windows certificate store of MachineT.
CEK: CEK aka column encryption key is the key that is used to encrypt ssnCol, this key is stored in encrypted form in SQL Server on MachineA and is not persisted anywhere in plaintext.
Hence, In order to decrypt ssnCol, CEK is required, however, in order to decrypt CEK, CMK is required.
Since CMK is in the Windows certificate store of machineT, only userT can access the CMK, decrypt the CEK and decrypt ssnCol.
userA is an administrator on machineA and also a SysAdmin on SQL Server, but, since he/she does not have access to the CMK, userA can not access ssnCol in plaintext. You can verify this by, using SSMS from MachineA, logging in as userA and querying ssnCol
If you have additional questions please put them in the comments section and I can answer them.
One additional and very important consideration:
The primary goals of Always Encrypted is to protect your data from malware running on the machine hosting SQL Server and from malicious high privilege users on the machine hosting SQL Server (DBAs, sys admins). If these are the attack vectors you want to address in you application, you should never provision keys for Always Encrypted on a machine hosting a SQL Server instance that contains a database with columns you want to protect. If you run a key provisioning tool, e.g. SSMS or PowerShell, on a machine hosting your instance, and the machine is compromised, an attacker can steal your keys, e.g. by scraping SSMS memory. And, of course, if you generate a certificate and put in the certificate store on the server machine, it is even easier for an attacker to get it.
Please, refer to https://msdn.microsoft.com/en-us/library/mt708953.aspx#SecurityForKeyManagement for more details and useful guidelines.
I would like to use PouchDB in a web app desktop client. I work in an environment where the computer user is generic and different persons use the same computer account. However, using my app they must log in with individual user names granting them their corresponding privileges. The system works offline, with period replication to the server.
Browsing through the documentation of PouchDB and searching the Internet I come to understand that there is no access restriction to a local PouchDB. Anyone who has access to the client/browser has in principle access to the cached data. Also implementing any sort of user access control in my web app seems to be kind of pointless. The code could simply be altered to allow access.
I came to the following possible solution and would like to know if that could work:
First contact with the central server
App sends user credentials to the server. The server encrypts a special databaseKey with the user credentials and sends this encryptedDatabaseKey back to the client app. The client app stores this encryptedDatabaseKey in localStorage, decrypts the contained databaseKey, creates and encrypts the local PouchDB using this databaseKey (e. g. crypto-pouch).
Offline usage
User logs into the app, his credentials are used to decrypt the encryptedDatabaseKey in localStorage, only then has he access to the stored data. If someone alters the code of the app he still cannot gain access to the encrypted PouchDB.
I see the following advantages:
- Without correct credentials there is no access to the local data
- Multiple users can have access to same local PouchDB since the databaseKey is identical.
- The databaseKey could even be changed regularly (app compares during a connection to the server the local encryptedDatabaseKey and the one received from the server, if they differ the app decrypts the database using the old key and encrypts it with the new)
Does this seem like a viable solution? Are there any other/better methods of securing a local PouchDB?
crypto-pouch is indeed the best method to encrypt a local PouchDB. However, I think where you say
Offline usage User logs into the app, his credentials are used to decrypt the encryptedDatabaseKey in localStorage, only then has he access to the stored data
I think it's pointless to decrypt the key and use that to decrypt the database; you might as well just as the user to create and memorize a password? Then you can use that as the key to the crypto-pouch.
I am using Microsoft Access 2010 with unbound forms. No linked tables allowed, otherwise the connections string is stored in the table definitions. So it follows that we will use a query definition with no name to access SQL SERVER. This is recommended by Microsoft. We need to get the connection string from somewhere though. So it is recommended to return it from a method with a obfuscated name. It is recommended not to embed the connection string in plain text in the application source. So we use encryption.
A good way of doing this is to require the applications administrator to define the connection string at the first run of the application and according to this msdn article
...encrypt its value via DPAPI with a user-specific key of the account under which the application runs, and save the encrypted value in the Windows registry.
The accde launches from the logged on windows user account, after which the apps admin can login and setup the connection to the database, following the recommendations above.
The weakest link i now seem to have is the windows user account. It seems that anyone logged in to that account could decrypt the connectionstring if they knew the implementation of the security scheme. Which means that the system still is not secure enough.
I could create a new windows user, but that would mean that the password for that user must be kept safe, which means we are back at square 1, securing the password that is used to access some secret information.
There must be an easier way, any ideas?
Is there a reason you need to persist the connection string from session to session? Could you instead build a log-in form in your application where you accept the user's credentials, server instance and database name that they will be connecting to and keep this information in memory while the application is running?
This might provide more flexibility in that the administrator could decide to move the database to a new server and wouldn't have to worry about decrypting the connection string to change it and re-encrypt it. It would also allow for multiple databases to be defined - I'm thinking of a situation where you would have a QA server defined for testing changes before rolling out to the production server.
My company is going to be storing sensitive data for our customers, and will be encrypting data using one of the managed .NET encryption algorithm classes. Most of the work is done, but we haven't figured out how/where to store the key. I've done some light searching and reading, and it seems like a hardware solution might be the most secure. Does anyone have any recommendations on a key storage solution or method?
Thanks for your replies, everyone.
spoulson, the issue is actually both the "scopes" that you mentioned. I suppose I should have been clearer.
The data itself, as well as the logic that encrypts it and decrypts it is abstracted away into an ASP.NET profile provider. This profile provider allows both encrypted profile properties as well as plain text ones. The encrypted property values are stored in exactly the same way the plain text ones are - with the obvious exception that they've been encrypted.
That said, the key will need to be able to be summoned for one of three reasons:
The authorized web application, running on an authorized server, needs to encrypt data.
Same as #1, but for decrypting the data.
Authorized members of our business team need to view the encrypted data.
The way I'm imagining it is that nobody would ever actually know the key - there would be a piece of software controlling the actual encrypting and decrypting of data. That said, the key still needs to come from somewhere.
Full disclosure - if you couldn't already tell, I've never done anything like this before, so if I'm completely off base in my perception of how this should work, by all means, let me know.
There only two real solutions for (the technical aspect of) this problem.
Assuming it's only the application itself that needs access the key...
Hardware Security Module (HSM) - usually pretty expensive, and not simple to implement. Can be dedicated appliance (e.g. nCipher) or specific token (e.g. Alladin eToken). And then you still have to define how to handle that hardware...
DPAPI (Windows Data Protection API). There are classes for this in System.Security.Cryptography (ProtectedMemory, ProtectedStorage, etc). This hands off key management to the OS - and it handles it well. Used in "USER_MODE", DPAPI will lock decryption of the key to the single user that encrypted it.
(Without getting too detailed, the user's password is part of the encryption/decryption scheme - and no, changing the password does not foul it up.)
ADDED: Best to use DPAPI for protecting your master key, and not encrypting your application's data directly. And don't forget to set strong ACLs on your encrypted key...
In response to #3 of this answer from the OP
One way for authorized members to be able to view the encrypted data, but without them actually knowing the key would be to use key escrow (rsa labs) (wikipedia)
In summary the key is broken up into seperate parts and given to 'trustees'. Due to the nature of private keys each segment is useless to by its self. Yet if data is needed to be decrypted then the 'trustees' can assemble thier segments into the whole key.
We have the same problem, and have been through the same process.
We need to have a process start up on one computer (client) which then logs in to a second computer (database server).
We currently believe that the best practice would be:
Operator manually starts the process on client PC.
Client PC prompts operator for his personal login credentials.
Operator enters his credentials.
Client PC uses these to login to the database server.
Client PC requests its own login credentials from database server.
Database server checks that operator's login credentials are authorised to get the client process' credentials and returns them to the client PC.
Client PC logs out of datbase server.
Client PC logs back into database server using its own credentials.
Effectively, the operator's login password is the key, but it isn't stored anywhere.
Microsoft Rights Management Server (RMS) has a similar problem. It just solves it by encrypting its configuration with a master password. ...A password on a password, if you will.
Your best bet is to physically secure the hardware the key is on. Also, don't ever write it to disk - find some way to prevent that section of memory from being paged to disk. When encrypting/decrypting the key needs to be loaded into memory, and with unsecure hardware there's always this venue of attack.
There are, like you said, hardware encryption devices but they don't scale - all encryption/decryption passes through the chip.
I think I misunderstood your question. What you're asking for is not in scope of how the application handles its key storage, but rather how your company will store it.
In that case, you have two obvious choices:
Physical: Write to USB drive, burn to CD, etc. Store in physically secure location. But you run into the recursive problem: where do you store the key to the vault? Typically, you delegate 2 or more people (or a team) to hold the keys.
Software: Cyber-Ark Private Ark is what my company uses to store its secret digital information. We store all our admin passwords, license keys, private keys, etc. It works by running a Windows "vault" server that is not joined to a domain, firewalls all ports except its own, and stores all its data encrypted on disk. Users access through a web interface that first authenticates the user, then securely communicates with the vault server via explorer-like interface. All changes and versions are logged. But, this also has the same recursive problem... a master admin access CD. This is stored in our physical vault with limited access.
Use a hard-coded key to encrypt the generated key before writing it out. Then you can write it anywhere.
Yes you can find the hard-coded key, but so long as you're assuming it's OK to store a symmetric key anywhere, it's not less secure.
Depending on your application you could use the Diffie-Hellman method for two parties to securely agree on a symmetric key.
After an initial, secure exchange, the key is agreed upon and the rest of the session (or a new session) can use this new symmetric key.
You can encrypt the symmetric key using another symmetric key that is derived from a password using something like PBKDF2.
Have the user present a password, generate a new key used to encrypt the data, generate another key using the password, then encrypt and store the data encryption key.
It isn't as secure as using a hardware token, but it might still be good enough and is pretty easy to use.