People can register for a newsletter on my website.
Their email address and other personal information will be encrypted in the database.
Now for that registration form I'd like to check if the newly inputted email address is already in my database, but as all emails are encrypted, I can't really check unless I loop through every possible email address, decrypt it and see if it matches the newly inputted email? Is that correct? Or is there a more efficient way to do this?
If you use the same encryption key all the time, you can simply search for the encrypted version. If you do not, you can still store a hash of the addresses additionally and compare against that.
Just a simple thought: create a table with a md5 hash of the mail and compare this in your SQL WHERE clause? Not sure if this is very effective..
Related
I'm trying to authenticate a user after registration. What's the correct or standard way to go about it?
Using this method as the way to implement it, in step 3, how can I generate the random hash to send to the users email? I see two different options:
crypto
JWT token
I'm currently using JWT for login, so would it make sense to use the same token for user verification? Why or why not, and if not, what's the correct way?
The answer to your question of whether you should use a crypto hash or a token is neither.
The hash you are generating to use as a verification method does not need to be cryptographically secure, it only needs to be a unique verification hash that is not easy to guess.
In the past I have used a v4 UUID with the UUID lib and it works just fine. You could also base64 some known piece of information about the user, like their id or email concatenated with something random, like the time in mircoseconds and a random hex string with substantial length, but honestly the time it takes to build out something like that is wasted when UUID v4 works just fine.
Your hash also doesn't need to be unique (different for each user, yes, but avoid all potential collisions? No) - hitting an endpoint with only the hash is not a great idea. The endpoint should also take an identifier for your user combined with the verification hash. This way, you don't need to worry about the hash being unique in your datastore. Find user by ID, check that verification hashes match, verify. I would only suggest that you obfuscate the user's know information in a way that you can decode on your end (ex: base64 encode their user ID + email + some const string you use).
[EDIT]
Verifying or validating a user is really just asking them to prove that the email address (or phone number) they entered does in fact exist and that it belongs to the user. This is an attempt to make sure the user didn't enter the information incorrectly or that the registration is spam. For this we don't need cryptographic authentication, a simple shared secret is more than enough.
When you store your user's registration data, you generate the shared secret you will use to verify the account. This can be anything that is (relatively) unique and contains enough length and entropy that it is not easy to be guessed. We aren't encoding or encrypting information that will be unpacked later, we are doing a literal string comparison to make sure the secret we provided to the user was echoed back to us intact. This is why a simple one-way hash is OK to use. I suggested a UUID v4 because the components of this hash are generated from random information (other versions of UUID make use of the machine's MAC or the time or other known pieces of information). You can use any method you like as long as it can't be easily decoded or guessed.
After you generate the verification hash you send it to the user (in a nicely formatted URL that they only need to click) in order for them to complete their account registration. URL guidelines are totally up to you, but here are some suggestions:
BAD
/verify/<verification hash>
or
/verify?hash=<verification hash>
With only the verification hash in the URL, you are relying on this value to be globally unique in your datastore. If you can reliably generate unique values that never contain collisions, then it would be OK, but why would you want to worry about that? Don't rely on the verification hash by itself.
GOOD
/users/<id>/verify/<verification hash>
or
/users/<id>?action=verify&hash=<verification hash>
Out of these two examples you can see that the point is to provide two pieces of data, 1. is a way to identify the user, and 2. the verification hash you are checking.
In this process you start by finding the user in your datastore by ID, and then literally compare the secret you generated and stored against the value given in the URL. If the user is found and the verification hashes match, set their account to Active and you're good to go. If the user is found but the hashes don't match... either you provided a malformed URL or someone is trying to brute force your verification. What you do here is up to you, but to be safe you might regenerate the hash and send out a new email and try the process again. This leads very quickly into a black hole about how to prevent spam and misuse of your system, which is a different conversation.
The above URL schemas really only work if your user IDs are safe for public display. As a general rule you should never use your datastore IDs in a URL, especially if they are sequential INTs. There are many options for IDs that you would use in a URL like UUID v1 or HashIDs or any implementation of a short ID.
ALSO
A good way to see how this is done in the wild is to look at the emails you have received from other systems asking you to verify your own email address. Many may use the format:
/account/verify/<very long hash>
In this instance, the "very long hash" is usually generated by a library that either creates a datastore table just for the purpose of account verification (and the hash is stored in that table) or is decoded to reveal a user identifier as well as some sort of verification hash. This string is encoded in a way that is not easily reversible so it can not be guessed or brute forced. This is typically done by encoding the components with some sort of unique salt value for each string.
NOTE - while this method may be the most "secure", I only mention this because it is based on the typical methods used by third-party libs which do not make assumptions about your user data model. You can implement this style if you want, but it would be more work. My answer is focused your intent to do basic verification based on data in your user model.
BONUS
Many verification systems are also time constricted so that the verification URL expires after some period of time. This is easily able to be set up by also storing a future timestamp with your user data that is checked when the verification endpoint is hit and the user is found. What to do when an expired link is clicked is up to you, but the main benefit is to help you more easily clean up dead registrations that you know cannot be verified.
I am making a social media type website, and I store user details such as emails, names and other personal details.
I will be encrypting the personal details using an Encrypt-then-MAC concept. When the user registers, a cryptographically secure string will be made to use as the private encryption key. When the user selects a password, the encryption key will be encrypted using the password.
The password will NOT be stored in the database, but will be the private key to decrypt the encryption key used to encrypt the personal details. The only person who knows the password is the user. My question is: how can I store the encryption key once decrypted?
I have thought of having a table with one column for IP and another column for the encryption key, but some people close the browser window without logging out, therefore there would not a possible way to remove the entry from the database when they have finished their session on the website.
Another way would be to store it in a cookie, but that could be intercepted when sent back to the server. I would like to know if there is a secure, nearly foolproof way to store the encryption key, client side or server side.
Thanks in advance.
EDIT:
In reply to TheGreatContini's answer -
The idea of a "zero-knowledge web application" (in your blog) is a good one, however, for zero-knowledge, even the key cannot be stored in the database, this complicates things a bit, as you would then have to use the user's password as the key. Using the password isn't as secure, as it is a bit harder to verify the password to prevent data which has been "decypted with the wrong key" from passing. There is the concept of Encrypt-then-MAC but that only verifies if the data is legit, and will assume that a hacker has messed with some data and data cannot be trusted, however, as you cannot actually verify the password (the hash would not be stored as it is "zero-knowledge"), so the password may just be wrong.
Not sure I have the answer, but a few considerations:
(1) Sessions need to be timed out. Perhaps you can do this by periodically running batch jobs that scan the database looking for sessions that have lacked activity. This requires storing in the db the date of the last action from the user.
(2) Generally keys are higher value than the content they protect because the keys have a longer lifetime than the individual data elements that the protect (because the data may change or additional data may be added). Rather than storing the key in the db, you can store the decrypted contents in the database for the length of the session. Of course, this is provided that you did (1).
Perhaps I am not adding much beyond what you already know, however may be worth considering a blog I wrote exactly about this topic. The low level details start in the section "A second line of defence for all the sensitive data." Prior to that it mainly motivates the concept. Glad to see somebody actually doing something like this.
Just thinking about my registration flow for my Nodejs/MongoDB app and wondered whether I really need to check if email addresses are already in the database when users register.
I make my users verify their email address by clicking on an email that I will send to them. They must do this before logging in at all. Once they do click on this link then I will mark that email as verified in the database. With this in mind, I'm thinking that it doesn't really matter if someone registers with the same email address (for whatever reason) as it won't be marked as verified unless they can access the email account.
I'm wondering however if it will impact on performance when it comes to searching for users in the database via email address. For example, users login using email address. Would it be advisable in this case to create an index on both email and verified?
Have I overlooked something that could potentially be a security flaw?
Performance-wise it will probably not matter a great deal (you probably already have an index on the email field), but I don't think you should allow it anyway.
For one, it has the potential to add junk to your database. Secondly, it would allow existing users to create—by mistake or not—a second account with the same e-mail address, which could cause all sorts of issues.
You don't give a specific reason for wanting to allow duplicate e-mail addresses, but a duplicate check isn't really all that time-consuming to implement (you could also consider a unique index on email so the database will throw an error when someone is trying to reregister an existing address).
When changing the password-hashing algorithm for an application, how should the system migrate the values already saved in the database? I am well aware of the fact that I can't migrate them in their hashed form but that I need to have the input data in order to calculate the new hash.
There are two situations in which I have access to the input data:
During login
When the user changes her password in her profile settings
obviously only during one of these I am able to save the new hash to the database to migrate the password.
Although all of my colleagues are voting for method one my gut tells me to not do that. Is there a recommended way?
I see no reason not to do this on logon. Is there a reason you don't want to do #1? You validate against the new hash, if that fails, validation against the old hash algorithm. If that works, I'd then write the new hash over the old one. This means that your passwords will be converted faster, since users probably logon more than they go to change their password. Unless you force people to, I doubt most will change it on their own.
Here's an alternative solution if you don't won't to touch the old authentication code (ie. switching to a new framework) or just want to rid yourself of the old password fields:
Backup the existing table of passwords and then delete all the existing entries in the passwords column in this table (and update the column type if necessary of course) so that it is ready to receive fresh passwords with the new encryption.
The next time users try to log in, check the passwords table and if the user exists with no password, then prompt them with "We have implemented a system wide upgrade and all accounts will need to be re-verified from email. We have sent you an email, please use the email to complete your account upgrade. We apologize for the inconvenience."
The users will go to their email and click a link that may say something like "Re-confirm my account". They will be taken to a page that requires some secure token parameter, received from the link given in the email. This page will now ask them to enter their username and password (more importantly password) to complete the upgrade. You can require they enter the password twice, to guard from typos. Technically, you are creating their password here. Just simply ask for it in 2 inputs labeled "password" and "confirm password".
There are of course both pro's and con's to this solution as well, in comparison to the others. The good thing is you don't have to add old hashing code in your new environment and have it sit there until the one day all your users have finally logged in again. But this solution comes with the price of writing extra code as well (code to send emails/token and so on). You'll have to compare that work to the work involved with your proposed solution of intercepting the form input coming in, checking against old hashing, and then passing onto new authentication code. Just another idea for you.
Look at this IT scenario: Company A Took over Company B with similar business model, All the customers need to be merged into one bigger system owned by Company A, while decommissioning user system in company B which has different password hashing algorithm,
The best Implementation to getting this done is to Force password Change for all migrated users via their registered email address.
It's hard to get specific advice without knowing specifics about the problem. I'm going to assume that the reason you want to change your password storage strategy is because your new strategy is going to be significantly more secure than your existing strategy.
If that's the case, then what's the possible advantage to waiting? The idea is to mitigate the existing risk. Users, realistically, very rarely change their passwords. If you want to migrate to a new storage strategy, you should probably do it at login or you're just going to have a big database full of passwords with dubious security.
What is your opinion on sending the username and password to their email address when they register on our website..this way if they forget the password in the future, they can look it up in their email...also we wont have to implent the forget/reset password scenario (we are close to release)..
is this approach safe enough?
My second question is that basically on our site, the user fills out certain forms and enter some information like their name, address, phone number, income information and such personal information..at the end, when they submit the application, we are thinking of emailing them a summary of all this information like their name, address etc so that they have it for their records..
is this ok..safe enough..what are the concerns
Never send a password or other sensitive information in the clear. That includes e-mail. You should also be storing as little of this as possible in a recoverable format. Unencrypted communication, especially e-mail, is easily tampered with, and you don't want the wrong people getting at passwords.
If possible:
Store your passwords in a salted hash, so the original text is unrecoverable, and thus unbreakable by anything short of a brute force attack. If the user forgets his/her password, make them reset it and send a temporary password (which they are required to change upon login) or a confirmation link (which, again, prompts for a new password) via e-mail.
Never send anything sensitive via e-mail; if the user needs information, make them go to your site to get it. You are using HTTPS, right?
People often share passwords across sites. So you should assume the same password works for the customer's online banking, and you should never send it by e-mail or provide a way for (someone pretending to be) the customer to retrieve it.
It's fine to send them a confirmation e-mail with their username - this is useful.
Remember, if you e-mail them their password they're likely to forget about that e-mail, or just delete it. So you need another password reset mechanism anyway.
The best way to handle the "forgotten password" case is for the user to request you to e-mail the user a link; when they click the link you allow them to type in a new password.
Regarding personal information (address, income etc): why would anyone want this mailed to them? They already know it! You're just sending private data unencrypted over the internet for no reason.
My rule of thumb would be - if you're OK writing it on a postcard and sending it through the mail, then it's OK for standard Email. I don't think income information would fall in that category for most people.
As for passwords, if they can't remember them in the first place, they won't be able to find the Email you sent them with the password in it, and it's an admission of storing it in the clear. I would avoid it and give them the means to reset - they will need that anyway.
The concern is definitely in the sending of the email with the password. If it is not properly encrypted, someone could potentially sniff the packets from the email being sent and recover the password. Also, the person could potentially have a hijacked email account. If it's not a big deal if someone steals the password then you may not have to worry, but otherwise I would NOT send any unencrypted passwords via email.
Edit: To address your second question, I wouldn't even email that. I would instead send a link so that they can easily see their profile/information when they log in.
I tell people to think of email like a postcard -- an employee of any company that handles it between the sender and the recipient can read it.
I agree with the top answer and have this to add: every time I receive a signup confirmation email that contains my password I delete the email and strongly consider never using that web service again. To me, it indicates a lack of security & privacy consciousness.
When you are sending any information via email, it won't be secure. There are too many ways someone can get it. It would be child's play for a skilled hacker looking to steal your information.
Refrain from sending any personal information like passwords and income information via email as it can become VERY EMBARRASSING for you and your organization if such information was leaked or stolen. Think about security seriously. It just takes that one incident for all the bricks to fall.
As for password retrieval, thoroughly read Forgot Password Best Practices.
The bottom line is that an application
following best practices should allow
a user to reset his own password.
Personal security questions should be
used. The application should not send
email, display passwords, nor set any
temporary passwords.
EDIT: Updated Link...
Most company simply do not include Username password combination due to the security of the external email client. Any numbers of users could brute force or guess the password to the email account of another users which would allow the hacker to view the email of your site. Then the hacker could wreak havoc on your site as well
I'd say providing a forgotten password function will still be vital as not everybody will be guaranteed to keep all there emails (or even be able find them later on)...
I have three rules concerning passwords:
Don’t store passwords in plain text in the database
Why should people trust you with that kind of information? You may only have good intentions, but big companies have failed before, so you're at risk too.
Don’t use password reminders
Password reminders are not worth it. They are easy to guess from people in your entourage and you often forget them. There are better ways to reset a password.
Always offer to send a new password by email
This is the most secure way of retrieving passwords. You should force the user to change the password once logged in with the new password.
As mentioned in comments, you might want to look at OpenID. The most secure way to manage passwords is to eliminate them.
I build an Web Application to send sensitive information by email. It's not UI perfect but it's really secure and working very fine.
There an outlook plugin, API to connect external website and the WebSite.
The concept is the message received in your mailbox are not in clear text. It's an HTML email with a link. You need to click the link to access the content of the email. When it's access one time, the message are destroy.
The message are stock in a crypted database on our side. You can configure a password that are know only by the two part to open the message online, or receive an password (Random 6 number) by SMS.
It's very simple to implement by API.
There is a sample
// https://www.secure-exchanges.com/API.aspx
List<string> files = new List<string>();
files.Add(originalFilePath);
string input = $"{body}";
string inputSubject = $"Your {subject}";
SendMessageAnswer answer = MessageHelper.EncryptMessage(new EncryptMessageArgs(GlobalSettings.bindingSecure, GlobalSettings.serial, GlobalSettings.ApiUser, GlobalSettings.ApiPassword, input, inputSubject + " - to open", recipient1, "", password, null, SecureExchangesSDK.SecureExchanges.SendMethodEnum.onlyEmail, false, true, true, "fr-CA", 1, 5)
{
FilesPath = files
});
if (answer == null || answer.Status != 200)
{
throw new Exception($"Impossible d'envoyé un message : {methodName}");
}