I'm looking for some guidance/opinion on best practice for both authenticating and signing (integrity checking) an HTTP request. I'm leaving this fairly open for language and technology used, but the basic parameters and requirements are as follows:
Need both authentication and integrity checking.
The request contains sensitive data and will be passed over HTTPS, but I cannot assume that the request cannot be sniffed by an attacker, either before it is HTTPS encrypted (at the client end by some illicit piece of installed software) or at the server end, e.g. by a sniffer placed between the HTTPS endpoint and the actual server. The HTTPS encryption/decryption handling has been passed off to an external load balancer so there is a small window for an attacker to insert a sniffer between the load balancer and the server.
I need to make sure that a transaction cannot be faked.
I need to make sure that a transaction cannot be replayed.
Requests can be GET or POST, and so any extra data needs to stay within the maximum size limit of a GET request.
Things that we have considered:
Username/password on each request. This guarantees authentication I guess, but if an attacker can sniff the username/password pair then it all fails.
Private key signing of each request. The server will have the public key, only the client will have the private key. This guarantees integrity because only the client could have generated the signature, but the server can check the signature. It generally guarantees authentication as well because the private key is not part of the request data and cannot be sniffed. However by itself it does not stop the transaction being replayed, because 2 transactions with the same data will have the same signature.
Using a cryptographic client nonce ( https://en.wikipedia.org/wiki/Cryptographic_nonce ) as part of the request data and including that in the data to be signed. We have had issues with these especially when they are generated on the client side because if they aren't sufficiently random then an attacker can find out how they are generated then the attacker can generate a sequence of nonces of their own, in advance of the client generating the same sequence, which can lead to a denial of service attack because the client will be trying to re-use a nonce that has already been used by the attacker. Generating nonces on the server side has been considered but it's an extra transaction and potentially a performance issue.
Including a date/time in the request data, however this can cause issues where the client's clock and the server's clock drift out of sync.
In case some admin decides to flag this as a duplicate, here are other Qs that I have considered that don't quite address the full scope of this issue:
What is the current standard for authenticating Http requests (REST, Xml over Http)?
Authenticity and Integrity of HTTP Requests
Assuming a 1-1 relationship between client and server, a HMAC with counter would solve this problem.
The client and server have a shared secret 128 bit key.
The client sends messages that have an HMAC with the secret key, and a counter. SHA-256 is recommended, as SHA-1 is on its way out (well SHA-1 HMAC is still considered secure, however that's another story).
e.g.
example.com?message=foo&counter=1&hmac=35ed8c76e7b931b06f00143b70307e90f0831682e7bdce2074ebb5c509d16cfb`
(This tool used for this post with secret bar.)
The HMAC is calculated over message=foo&counter=1.
Now, once the HMAC has been authenticated by the server and the counter checked, the server's counter is incremented to 2. Now the server will not accept any authenticated messages with a counter less than 2.
JSON Web Tokens could be used so that you are doing the above in a standard format. You could do the above with multiple clients, however you would need to keep track of the counter server-side for each client, and the client would have to identify itself in the message. Managing the shared secrets is the trickiest bit if you decide on a different key per client.
Related
Background: I'm working on a mobile application (online chat) that use persistent TCP connections to a backend server. In the previous version, we used a well-known encryption scheme to protect the payload and sign using a shared secret key. This provided sufficient protection against data tampering and reverse engineering. But the traffic was still susceptible to replay attacks: someone can capture the network traffic and replay it and server would accept the requests since it had no idea whether it was real or replayed. To counter this, we introduced timestamps and nonces which provided protection against replay attacks.
All well and good. A decision was later made to use TLS (Transport Layer Security) to protect all communication between clients and servers. TLS was implemented and to provide an extra layer of protection against Man-in-the-Middle attacks, we pinned the public certificate of the server on clients.
Which brings me to the question: is it still necessary to use custom encryption and guard against replay attacks since TLS already provides protection against these? In my research, I found that TLS provides protection during transmission i.e. prevents man in the middle but what if the originator itself wants to cheat the system and modify payload or replay traffic? Does TLS protect against an intentional cheater trying to somehow intercept the traffic in the network layer of their OS? My understanding was that TLS provides end-to-end encryption but I have a suspicion that TLS traffic could, in fact, be replayed if the originator wishes to.
I'm also aware of memory-based attacks where the attacker would modify the memory on the device this modifying payload (may be even record and replay.) But even our custom encryption + replay guard won't protect against these.
From: https://www.owasp.org/index.php/Transport_Layer_Protection_Cheat_Sheet#Benefits
"TLS also provides two additional benefits that are commonly overlooked; integrity guarantees and replay prevention. A TLS stream of communication contains built-in controls to prevent tampering with any portion of the encrypted data. In addition, controls are also built-in to prevent a captured stream of TLS data from being replayed at a later time.
It should be noted that TLS provides the above guarantees to data during transmission. TLS does not offer any of these security benefits to data that is at rest. Therefore appropriate security controls must be added to protect data while at rest within the application or within data stores."
TLS only protects the transport and thus it provides protection against modifying or replaying of the encrypted data only. It does not protect against any kind of modifications or replaying of the data before the encryption or after decryption. Sending the same data again over a TLS connection is actually perfectly valid.
But, the nonce and timestamp you use to detect replay do not protect against modification or replaying too. The sender can still use the same data but "protect" these with the new nonce and the new timestamp. It is simply not possible to protect against this kind of client based attacks as long as the attacker is able to modify the client code which is usually the case if the client runs on a system owned by the attacker like in case of a smartphone game where the owner of the smartphone likes to cheat.
I’m trying to implement a message level encryption. Here is the current situation:
We have a mobile app client connects to server via oneway certificate https and we have username/password authentication and secure token for subsequent client to server invocations. So intention of message level encryption is not try to prevent 3rd part sniffing information or stealing client identity, instead to prevent the client user him/her self to something like,
1) Inspect and try to understand server - client protocol
2) Forgery request with other application than our app client
The initial idea is to use symmetric algorithm (DES, AES, or some simpler algorithm, as long as it could not be simply cracked by statistical or mathematical method without knowing the key). And the key is generated from a hash (SHA etc) from a string concatenated from a salt pre-agreed between client and server, and some information server tell client in non-encrypted content (for instance, the first call from client to server happens in non-encrypted context, and server returns a timestamp to client and also remember it for later key generating).
Does this do what I want to do for 1&2? And what is the major vulnerability if any?
There is no "secure" solution to what you try to achieve as long as you publish your software, because any key you use for the message level encryption will have to reach your software somehow. Either it is compiled into it, then a dedicated attacker can read it out of the binary, or it is transferred to the running software using the network, then the attacker can emulate the protocol your software is using to get the key.
The best you can hope for is to make it difficult for a reverse engineer to get to the key. That means you could assemble it in such a way that no complete piece of it can be found in the binary. But still if someone attaches a debugger at runtime she could still read it out of a variable trivially. In the end it remains an arms race between you and the reverse engineer.
This is an interview question I couldn't solve:
You have a client and a server. The client can send a file to the server, and the server executes the file. Here is the communication protocol:
The client sends a HELLO command to the server
The server sends a challenge to the client
The client sends the file, a hash of the file, the challenge, and a hash of the challenge.
The server verifies that the challenge and the file match to their hash sent by the client, and sends OK/Error whether the verification succeed.
A third party can intercept the communication and change the message. Your job, as a third party, is to intercept the communication and send a different file, but you don't know the hash algorithm.
What are some possible solutions here?
What I thought about is to try and learn the hash algorithm, but they said it's too complex and there's a simpler way.
If you can capture two connections (either at the same time or consecutive ones), then you act as follows:
during first connection MITM sends its fake file as a challenge to the client. The client returns the hash of the fake file. Now the connection must be interrupted to avoid the unnecessary entries in the log of the server or the client (which would happen if you let the procedure continue).
As we have interrupted first connection, the client will re-connect. On this stage MITM lets the client get the real challenge and send the new challenge, its hash and the client's file and its hash to MITM. MITM replaces the file and its hash with the fake file and the fake hash, obtained during the first attempt and sends everything to the server.
This scheme requires that (a) the client can handle large file as a challenge (ie. it doesn't have a fixed-sized buffer or length checks), (b) the client will re-connect if the first connection is dropped, and (c) we can let entries about dropped connection get into server's or client's logs.
Requirement "a" seems to be quite serious and in real-world environments validity checks for input parameters (such as challenge) are mandatory.
I have a client-server game, where the client connects to a server and stays connected during the game (approx 5-60 min).
I want new clients to be able to register securely, as well as allowing existing clients to authenticate with no worries that the login credentials are exposed.
The thing is that for performance reasons it would be best to stick with a simple and cheap encryption like RC4 for the game session, but a symmetric key does not make it easy to secure the registration procedure.
Since I will want to keep a separate login server anyway, my idea is like this:
Client sends a HTTPS request to the login server with credentials (or registration information)
The login server collects the user information, and generates a temporary RC4 session encryption key
The user information + RC4 session + timestamp + digest (I can rely on both servers to be synchronized timewise) with a secret symmetric key, shared between game server and login server.
The packaged data + RC4 session encryption key + ip address to the game server is sent as a reply to the HTTPS request to the client.
The client opens a connection to the game server, sends an initial unencrypted hello message with the encrypted user information as a payload.
The game server unpacks the data packaged in (3). It now knows the user and the RC4 encryption key it is supposed to use.
If the timestamp indicates that the login credentials has expired, an error is returned to the client (who is then to retrieve new information). If the decrypted user data cannot be verified with the digest a different error is returned.
If everything checks ok, the server sends an unencrypted LOGIN_OK, and the RC4 encrypted communication starts.
Possible security concerns:
The game server 100% trusts the user info it has decrypted. This makes the servers completely decoupled which is nice, but if the key is compromised, users could completely fake their user info. This could be alleviated somewhat by rotating these keys, so that every day or month has a new key. Both game and login servers could get this from a third server that manages their keys. It might be overkill since: a) in case of a break-in where source code is exposed on the servers, they can be restarted with a new key b) a good enough key + encryption should make brute force attacks hard (suggestions on algorithm?)
RC4 isn't the most secure algorithm, but I make sure to throw away the first 512 bytes or so and each key is only valid for a limited time, e.g. 24h.
Doesn't seem susceptible to man-in-the middle from what I can see: SSL secures the RC4 session key, in (5) the RC4 session key sent to the game server is encrypted as well. All that is possible is DoS and to cause the user request a key again. If the data in (2) is cached until it expires, this should not create a new packet.
The encryption in (3) could be improved by adding random bits to the key. Those random bits are sent together with the encrypted packet, and presented to the game server in (5). In (6) the game server adds those random bits to his key and uses the result to decrypt the data. This way and attacker cannot see when the packed data changes.
Are there any vulnerabilities I'm overlooking here?
A summary of payloads created:
Client login-credentials (protected by SSL), sent to login server
User info + timestamp + temporary game server session key + digest encrypted by login server using a secret key shared with game server, given to the client that - without modifying it - passes it to the game server. Ought to be temper resistant because: a) client does not know the secret key b) has timestamp to avoid resending same data c) digest to verify content was encrypted correctly
temporary game server session key sent by the login server to the client together with the encrypted payload. Protected by SSL.
Client game server login packet, consists of encrypted packet received by login server.
A summary of encryption keys:
Temporary game server session key: randomly generated by login server for encrypted game server <-> client communication. Generated by login server, given to client and game server.
Secret user info encryption key. Shared between game server and login server, used pass user info to game server with client as messenger. Client does not possess this key.
First of all I wouldn't use RC4. There are both faster and more secure stream ciphers around so if you control both client and server then you can probably do better than RC4. Discarding only 512 bytes may not be enough for the Fluhrer, Mantin and Shamir attack, but even if you discard more bytes there's also the Klein's attack etc. I don't know if it's worth the trouble.
Second, make sure that the keys you generate are random. I know it seems obvious but for an example see: http://www.debian.org/security/2008/dsa-1571
But the real problem is this part: "The game server 100% trusts the user info it has decrypted. This makes the servers completely decoupled which is nice, but if the key is compromised, users could completely fake their user info."
You have to assume that the user knows the key. His game client has to know the key so it can communicate with the server. If the user can use his real data to log in via ssl, get a key for stream cipher and then send whatever info he wants to the game server then all the attacker has to do is just get an account and do whatever he wants.
It doesn't matter how often you change the key because every time you change it you have to still give it to the client so you might as well change it after every byte and it still wouldn't matter here.
This is more important than the cipher used or the key generation because no one will brute force the key if he just gets it.
You should never trust the client. You should store the client data on the server side and match it with the key or sign the data and verify it or use HMAC etc. because if the game server 100% trusts the user info then you will have problems sooner o later. There is pretty much no way around it.
It sounds like you're trying to reinvent SSL. Why not issue each client a certificate (signed by your own root authority), and have them connect to the game server over SSL, with mutual authentication?
I understand you cannot use SSL between the game server and the client as you don't want to go through the handshake again.
The protocol seems ok from a first glance. There is no replay attack also as you really need the symmetric session key to do anything meaningful. The best thing you can do is switch to AES is also very fast and very secure. I highly doubt you will see any performance hit by switching to AES.
Also the first security concern bullet point you mentioned is not really a concern. Well it is a concern for all clients on the desktop, for example your browser has the same problem talking over HTTPS. So you don't really have to solve it. Your game logic somehow has to look for bad behavior if you want to actively monitor manipulation attempts. You cannot solve it by re-keying.
I ended up also posting on sci.crypt and I'll try to summarize the suggested changes (as far as I understand them) below in case it might be of interest.
Step 1: Client sends a HTTPS request to the login server with credentials
Assuming that the credentials take the form of a login token, also add a self-assigned unique id.
Step 3: The user information + RC4 session + timestamp + digest
Use an encryption algorithm that ensures integrity, instead of using a digest explicitly. E.g. AES-GCM or AES-CCM. Add the extra id field in step 1. Add the ip to the game server as well.
Step 4: The packaged data + RC4 session encryption key + ip address to the game server is sent as a reply.
Giving the timestamp to the client will allow the client to know when the session has expired. This avoids unnecessary connects to the game server with expired credentials.
Step 5: The client opens a connection to the game server, sends an initial unencrypted hello message with the encrypted user information as a payload.
Add the self-assigned id in step 1 unencrypted to the payload.
Step 6: The game server unpacks the data packaged in (3). It now knows the user and the RC4 encryption key it is supposed to use.
The game server matches both its own ip with the encrypted ip, as well as the encrypted id with the id given by the client. The first prevents the user from going to a different server with the same credentials.
Step 8: If everything checks ok, the server sends an unencrypted LOGIN_OK, and the RC4 encrypted communication starts.
At this point the game server cannot be sure of the client's identity. Use the session key and encrypt nonce + strictly increasing session id + login success state using AES-GCM/CCM and send it to the client.
The client decrypts and checks the login success state. If this is true, then the client knows that the game server knows the session key (GCM/CCM verifies that the packet has not been tampered with). The client returns sid + nonce.
The server verifies sid + nonce is the same as the values sent.
Finally the client and server creates new session keys by hash-ing the session key with sid + nonce + salt to create the key for the consequent communication, to prevent a possible replay attack.
Regarding RC4
There are vulnerabilities in RC4, but it probably would suffice for this scheme because of the rather aggressive key rescheduling. However, there are modern ciphers which are more secure and faster, such as Snow 2.0 or Trivium.
Just use SSL to the game server. Modern cryptanalysis has resulted in a few very fast implementations of some of the better encryption algorithms. For example, well optimized AES implementations can easily encrypt at better than 150MB/s on any remotely modern machine. Also while AES is held with high regard, it does have two weaknesses that I know of, but when used correctly those weaknesses become insignificant.
I noticed that you failed to mention that you would be using an advanced key scheduling algorithm between the client and the game server. Failing to do so would make the weaknesses of the encryption algorithm much more severe. SSL/TLS should do the key scheduling for you.
Given the fact that for pure REST each resource request from the client carries authentication information, it appears to me that the only way to secure against replay or copy-and-paste attacks requires that the REST request runs inside an HTTPS protected channel.
Is that assertion correct?
False. The fact that each request carries authentication does not mean that a request can be replayed. HTTP DIGEST can be used for authentication and a digest authentication cannot be replayed, because the challenge from the server will carry a different nonce on each connection and clients cannot reuse a nonce:
Server nonce is allowed to contain timestamps. Therefore the server may
inspect nonce attributes submitted by
clients, to prevent replay attacks.
Server is also allowed to maintain a list of recently issued or used server
nonce values to prevent reuse
Requiring HTTPS is not bad on itself, it certainly adds increased privacy and tampering protection for the traffic, but it is not required to prevent replay and copy-paste attacks.
The assertion itself is incorrect since it lacks context. REST is an architectural style. Although HTTP is the typical example of supporting protocol for a RESTful system, the security of such a system would depend on a number of factors.
First of all, there are HTTP authentication mechanisms that are orthogonal to the application running on top of HTTP, e.g. HTTP Basic and HTTP Digest. While HTTP Basic isn't secure, there are mechanisms that can help protect against replay attacks in HTTP Digest, which doesn't send the authentication information in clear either. (There can be and there are other authentication mechanisms that are orthogonal to the application and that can be more secure than HTTP Digest too.)
There are also specifications for securing HTTP requests at the message level (e.g. HTTPsec).
However, the wide availability of SSL/TLS stacks on various languages/platforms/OS makes it rather convenient for many applications to protect the communication between the client and the server via HTTPS.
Not using HTTPS doesn't mean that your system is going to be insecure (although it might require a bit of work to protect the data); and using HTTPS isn't a guarantee that your application will be secure either. Security is a broad topic and you need to consider the range of threats you want to be protected against before making an evaluation.