I run a service that integrates with a few other cloud platforms via their apis. In order to do this, we have to store the login credentials for OTHER sites in our database. Obviously security is a bit of a risk here.
So far, we have been storing the passwords using AES encryption and a salted version of the user's password(for our site) as the cipher. When a user requests something from the api, they must input their password. The password checked for validity against the sha hash that we store, and once confirmed, is used to decrypt the password.
The problem is, we would like to start offering a service that retrieves data from the apis we interact with at scheduled intervals(outside the scope of synchronous user requests.). If we do this, our current security structure will no longer be viable.
My question is, are there any ways to allow for this type of api interaction without storing recoverable versions the passwords in our database? If not, what are my options for securely storing passwords?
we would like to start offering a service that retrieves data from the apis we interact with at scheduled intervals(outside the scope of synchronous user requests.).
This is what the OAuth protocol is designed for. The OAuth 2.0 code grant gives a client application an access token and a refresh token. The refresh token allows the application to get an access token even when the user is not there to authorize the request.
Related
I am building an intranet web application consisting of an Angular frontend and a Node.JS backend. The application needs to use the corporate Active Directory for authentication and authorization.
I'm considering how to best implement this in a secure way. I am planning to use the Active Directory node module for actually communicating with the AD to authenticate when the user logs in, and to check security group membership for certain restricted actions, etc.
However, I am not quite sure what is the best way to authorize my backend endpoints. The AD module does not offer any token/ticket, even though I suppose Kerberos is used for the actual authentication process. In other authenticated apps I've developed I've generated a jsonwebtoken when the user logs in, and then passed and verified that token in each backend route, is that a good idea also when authenticating against AD?
EDIT: Second part of question spawned to separate thread: Best practices for server-side handling of JWT tokens
Also, I have a more general concern, regarding what the best practice is for actually verifying tokens. Suppose that the "secret" used for JWT generation is compromised (in my scenario many people may have access to the source code of the system, but not to the system itself). Am I right in believing that a malicious user could then, with only this information, generate a token on behalf of any given user, and without ever authenticating with AD use that token in my API requests? A token is typically generated using jwt.sign(payload, secretOrPrivateKey, options).
Alternatively, suppose a malicious user could get hold of an actual token (before it has expired). To me it seems like instead of having to know a user's username and password, the security is now reduced to having to know the username and the JWT secret. Is this a valid concern and what should I do to prevent this?
My best hope so far is using a server side session to store information about the current user after logging in, so that even if a token is maliciously generated and used when accessing backend endpoints, it would fail unless the user has actually gone through the login route, authenticated with AD and stored some information in the session as a result of this.
I also considered actually authenticating with AD in each API endpoint, but that would require the AD username/password to be sent in every request, which in turn would require that sensitive information would have to be stored in the client's sessionstorage or localstorage, which is most likely a bad idea.
So, questions:
1) Is it reasonable to combine AD authorization with JWT as bearer token or what is the preferred way to build a secure backend + frontend utilizing AD for authentication?
2) If JWT is a good idea, what is the best practice for securing endpoints using JWT? Is using a server side session reasonable?
Interestingly enough I have found tons of examples on how to best implement token based authentication (in general, or with NodeJS specifically), but many of them seem flawed in one way or another.
1) Is it reasonable to combine AD authorization with JWT as bearer
token or what is the preferred way to build a secure backend +
frontend utilizing AD for authentication?
It is reasonable, but if you are already using Kerberos and AD to initially authenticate the user, you might consider using s4u2proxy constrained delegation which allows the service to present the user's service ticket to the KDC and acquire (subject to authorisation checks) a ticket for a backend service (and repeat for as many services are necessary).
If you have a lot of backend services that need to be contacted, a single JWT bearing all the authorization claims needed for all the services to enforce authorization policy may be a better option.
2) If JWT is a good idea, what is the best practice for securing
endpoints using JWT? Is using a server side session reasonable?
General key security practices apply:
Never store keys in the clear in non-volatile storage, anywhere.
Ideally do not store encrypted keys in attached storage on the server where, if the server is compromised, they would be subject to offline attack. Make them available to the host only at server startup.
Ensure key material resides in secure memory so that it cannot be swapped to disk (and/or use encrypted swap).
Use public key algorithms so that no secret key need exist on multiple hosts.
Consider using a hardware security module (HSM).
I am setting up a website to use Google's OAuth2 interface for user authentication. The website will store private data associated with each user - which I'm planning to encrypt.
If I implemented my own authentication method for the website, I could easily derive a key from the user's credentials (which include the user's password), allowing the data for each user to be strongly protected. But with OAuth2, I believe I can only receive an access token, granting that user permission for a period of time - the problem is that the access token value will change over time.
Is there a way that OAuth2 can provide me with an immutable secret tied to the user which I can use to derive a secure key from? Or is there some other method of creating a secure persistent secret using OAuth2?
--- Edit ---
In response to the questions and comments, here are some thoughts to consider:
All user information should always be protected with strong encryption and user authentication - the reason we read so many news articles about website & database hacks is because developers say "do we really need to secure that" and then respond with "No - because no-one but us will be able to access the database, security is hard, etc". The hacker downloads the database et violá. Credit cards, email addresses, phone numbers, passwords, you name it, then become compromised.
There are only two real secrets - one is a password stored in someone's head, the other is a strong random value that only the authorised user has access to (like a physical token). If you think a secure key can be derived from an email address alone, or that a secret needs to be stored in a database, you don't really understand security.
I guess what I was trying to discover was whether an OAuth provider can provide to the OAuth client an immutable value securely linked to both user and client - effectively, this would be a key that could only be unlocked by the OAuth provider using a combination of the user's secret (their authentication password) and the client's secret (used in the OAuth protocol). The client could then use this value to provide a reasonable level of security for the user's data.
Of course this implementation is not perfect from abuse, but implemented correctly, could provide a reasonable way to secure data whilst still using the good practices of the OAuth scheme.
The point of the token is that you can then use the token to obtain information from Google about the user. During the initial authentication, you will tell the user, and google, that you want to access certain information about the user:
https://developers.google.com/+/api/oauth
Assuming that the user allows you to access their information, such as their email address, you can then get their email address from google. Once you have their email address, you can generate a secret key for their user, store this in your user table, and use it to encrypt their data. Then, when they login again, you can lookup their email address and find their key.
Is there a specific need for the immutable information be 'secret'? Or is it just a key to identify a user?
If the information that you're storing is truly private, and you want to make it so that you can't access your user's data, then all you have to do is store the encrypted blob for your users. Once the user had downloaded their data, they can use their key to decrypt the data client-side.
My first question would be: Why do you want to derive your encryption keys from some tokens?
The tokens and your encryption keys could remain independent and can be associated to a user identified by a unique id. User authentication can be done by whatever way you need either via credentials or open ID authentication or something else. But, once a user is authenticated, your decryption APIs can fetch the decryption key associated with the authenticated user and do whatever decryption it has to.
This way you can potentially allow users to tie multiple open ID accounts with the same user similar to what Stackoverflow does. I can associate my yahoo, facebook and google accounts with my Stackoverflow user and can sign in with any of those providers. I can disassociate those accounts any time I want. But that does not affect my Stackoverflow profile and data.
So, it is not a good idea to derive your keys from something that is not constant and keeps changing. Instead, keep them separate.
If I implemented my own authentication method for the website, I could easily derive a key from the user's credentials
This schema has a terrible weakness - if a user forgets /resets its credentials, the key is lost forever
Is there a way that OAuth2 can provide me with an immutable secret tied to the user which I can use to derive a secure key from?
OAuth2 is an authorization protocol. It is not intended to give you any user secrets.
the Google's OAuth2 should provide a user info service returning username (email) and some id (sub).
However these are identity information, not any secrets. And mixing user credentials is imho bad idea (as already mentioned), using an external IdP (google) you will have no access to credentials tat all.
And now what?
My suggestion:
If you really want to encrypt user's data using a user-provided secret, let the user provide the secret or encrypt the data encryption key by the user itself (or using user's public key?). The user must be aware that if this secret is lost, the data will be unaccessible. It is quiet uncomfortable in long run. Some DMS systems use this approach to encrypt the stored sensitive documents.
if you want to encrypt data in rest (on the server, database), you may have an application specific key, imho best stored somewhere else (key vault, key management service,..). Indeed then there is a different risk profile (you have to protect the key,..) but it's much more convient for users. This is usually good enough along other security measures (even big enterprises don't ask for a separate password to encrypt your credit card number or email)
What you need is constant secure (random) key for each user that you could get from authentication service that gives OAuth2 endpoint (in this case - Google).
OAuth2 protocol itself does not provide such value - Authentication server uses generated values that are not constant. But OAuth2 does not prohibit from giving this value from Resource server (together with user id, email etc). So basically OAuth2 lets you secure data in the way you want, but Google, which you currently use, does not give this type of constant random value.
Also note, that this would not work if you would let user relate few accounts, like Google and Facebook, as they would give different random keys.
If you derive secret from credentials, this would also mean that resetting password would reset user account.
Furthermore, if you encrypt data like emails in this way, it becomes impossible to decrypt them without currently signed-in user. So emailing newsletter becomes practically impossible. You cannot query the data in SQL also.
I could only suggest some countermeasures:
Do not store sensitive data at all, or store it hashed. Passwords must be hashed, not encrypted. Do not store CC numbers, store tokens that represent them.
Use encryption with key, stored in another data source. This adds at least some security - attacker must get not only DB copy, but also encryption key.
As data is encrypted, storing it in database is no longer necessary. You can store encrypted data in files or some other source, where it is safer than in DB (no risk of SQL injections etc)
I've been having this same issue. So far, I can't find a secure way around it.
Basically, we need a per-site randomly generated secret provided only with implicit flow that can be used to derive credentials to access systems and decrypt data.
Because I want to protect the data from myself, I could write the client to salt/hash the secret in two ways, one way to retrieve the data and another to decrypt it.
Alas, this is not the case.
I could derive credentials from things in the basic scope of the oAuth and that would protect the data against me, but that leaves the user wide open for cross-site vulnerabilities, and besides, personally identifiable information makes for a poor secret.
The best I got is to use implicit flow oAuth2 to acquire the user's email address, randomly generate a client side secret, and force the user to email themselves the secret (as a recovery key), then store the secret in localStorage. Salt/Hash the secret + oauth scope variable to derive the credentials client side (so the user must be logged in) needed to access, encrypt and decrypt data.
If the user ever clears their localStorage, they need to go click the link in the recovery email, which places the secret back into localStorage.
This places the scope of vulnerability back on the client, but is resistant to public machines (would have to know who was logged in last, and get access to the localStorage token), allows for recovery, and weakly requires the user to be logged in. Still vulnerable to plugin injection attacks and physical access + knowing the user.
Update: I have decided to use some oAuth extensions (hello.js, folder APIs) to store the keys in the user account as files. It requires some permissions and some APIs to implement, but appears to be viable.
Imagine this situation: your users give you their credentials (username/password) to access a third party service. So you have to produce those credentials when connecting to the service, you cannot just store a salted hash.
The environment is Grails, with psql as DB. From the programmer point of view, ideally the user/password would still be part of the domain objects (so they are easy to use).
What would be the best practice to securely store them?
*(I'm not a security or crypto expert; this is my understanding based on my reading and research, but is very far from authoritative advice. Get the advice of web-app security professionals and get a proper security audit.)*
The best you can really do is have your app unable to decrypt them when the user isn't actively logged in.
Encrypt the credentials with a key based partially on the user's raw, unhashed password. You never store the user's password to log into your service on your systems, of course, so you only have access to it for a brief moment during authentication (and only then because the web hasn't caught up with the mid-90's and adopted sane challenge-response authentication schemes). You can, at the moment of user log-in, decrypt the saved credentials for the 3rd party services and store them in the volatile server-side session for the user.
For the encryption key you might hash the username and user raw password with a large-ish salt value you generate randomly for each (user,3rd-party-credential) pair and store alongside the encrypted credentials. The salt should be different to their salt used for their stored hashed password.
This is far from ideal and has all sorts of problems - but the credentials won't be accessible after the user's session expires or they log our and you purge their session.
It also means your app cannot act on their behalf when they aren't actively logged in, a limitation that may be a showstopper for you depending on your requirements.
A weaker option is to have a key for all user credentials that's manually entered by the sysadmin when the app is re-started. This key has to be stored in memory, but it's at least not sitting on the disk or in the database, so someone stealing a dump of your database will have a much harder time decrypting the stored credentials.
Neither option will help you if the attacker finds a way to trick your app into revealing those domain objects after decryption - or getting it to let them impersonate that user, getting it to perform actions on the 3rd party service on behalf of another user, etc. It'll at least protect against theft of database dumps and similar attacks, though.
One further recommendation: Rather than using pgcrypto to the crypto in the DB, do it on the application side. This means the DB never sees the key material required to decrypt the data; it can never be leaked into database logs, sniffed out of pg_stat_activity, etc.
Am I correct that OAuth 1.0a credentials need to be stored in plaintext (or in a way that can be retrieved as plaintext) on the server, at least when doing 2-legged authentication? Isn't this much less secure than using a username and salted+hashed password, assuming you're using HTTPS or other TLS? Is there a way to store those credentials in such a way that a security breach doesn't require every single one to be revoked?
In more detail: I'm implementing an API and want to secure it with OAuth 1.0a. There will possibly be many different API clients in the future, but the only one so far has no need for sensitive user data, so I'm planning to use "2-legged" OAuth.
As I understand it, this means I generate a consumer key and a shared secret for each API client. On every API request, the client provides both the consumer key, and a signature generated with the shared secret. The secret itself is not sent over the wire, and I definitely understand why this is better than sending a username and password directly.
However, as I understand it, both the consumer and the provider must explicitly store both the consumer key and the shared secret (correct me if I'm wrong), and this seems like a major security risk. If an attacker breached the provider's data store containing the consumer keys and shared secrets, every single API client would be compromised and the only way to re-secure the system would be to revoke every single key. This is in contrast to passwords, which are (ideally) never stored in a reversible fashion on the server. If you're salting and hashing your passwords, then an attacker would not be able to break into any accounts just by compromising your database.
All the research I've done seems to just gloss over this problem by saying "secure the credentials as you would with any sensitive data", but that seems ridiculous. Breaches do occur, and while they may expose sensitive data they shouldn't allow the attacker to impersonate the user, right?
You are correct. oAuth allows you however to login on the behalf of a user, so the target server (the one you access data from) needs to trust the token you present.
Password hashes are good when you are the receiver of the secret as keyed-in by the user (which, by the way, is what effectively what happens when oAuth presents the login/acceptance window to the user to generate afterwards the token). This is where the "plaintext" part happens (the user inputs his password in plaintext).
You need to have an equivalent mechanism so that the server recognizes you; what oAuth offers is the capacity to present something else than a password - a limited authorization form the use to login on his behalf. If this leaks then you need to invalidate it.
You could store these secrets in more or less elaborated ways, at the end of the day you still need to present the "plaintext" version t the server (that server, however, may use a hash to store it for checking purposes, as it just needs to verify that what you present in plain text, when hashed, corresponds to the hash they store)
I'm currently developing an API for an online service.
I would like to give access for mobile and web developers to create their applications.
Developers will have the usual number reqs/minute limits for their applications.
What are the best practices for authenticating applications?
For web applications it's easy. We provide token, token is valid for a domain so even if somebody will try to use anywhere else it will fail.
How to do that for mobile applications?
We can provide token. Such token needs to be distributed with application on the device
and means that somebody will sniff that token he can write another application that will use the same token. This will mean that original user will have to revoke old token, create a new one and release new version (that his users will have to download again).
Do you know any solution for that?
I'm not sure that your developers would be able to securely do this without having some form of communication with their own host and some form of user account on their system. As you said, if you included a long-lived token in an app, no matter what obfuscation is done it could eventually be discovered by reverse engineering techniques.
There are 2 options that I can see:
1. Short lived token
In this scheme the mobile application contacts the developer's system to receive an short auth token.
During enrollment and periodically thereafter, developers generate a public-private keypair and give you the public key.
Each auth token would need to include an unencrypted "developer key ID" of some sort and an encrypted bit of data including the token's issue data and a salt of pseudo-random data. The developer's host would encrypt the data using a private key in a public-private keypair. This keeps the secret in a controlled and secure space. The encrypted data needs to include the salt in order to prevent known-plaintext attacks on your developers' keys.
The app sends the token to you, you can determine it's legitimacy by:
Use the unencrypted developer key ID to determine which key to use in decrypting the encrypted string.
Has the developer key ID been revoked or expired? (due to key compromise, dev API subscription expiration or abuse, etc). If it was revoked, deny access.
Does the encrypted data in the token decrypt correctly? If not, deny access.
Has the token expired? (based on the encrypted token date) If so, tell the client to get a new token from the dev server. Their software should do this before contacting your API, but you have to check just in case. I'd suggest that tokens be allowed to live for a relatively short time since copying a token between apps is a weakness.
Allow access
You could also use symmetric encryption instead of public-private key encryption, but then you and the dev both know the secret. It'd be more secure if only the dev knows it.
2. Pass API calls through dev host
It'd be possible for mobile applications to talk to their developer's host instead of your host for calls to the API. When the dev host receives one of the calls, it simply passes the call through to your API and adds their secret token.