My client-secure algorithm need a Random oracle... And I am using a HTTPS webservice for it,
https://www.random.org/integers/?num=1&min=0&max=255&col=1&base=16&format=plain&rnd=new
... But many people notice that
random numbers transferred over the public internet are not cryptographically secure for most purposes
... So, the question is:
the described architecture is secure? Need to use HTTPS POST instead GET? Need to add some cryptographic layer in the response?
there are a way to check/quantify "how much secure" is it, to compare with another solutions?
Context
It is not "so simple" and I really need a webservice, that is like a random oracle, must be an exteral device input (the oracle)... I can't use a local client algorithm (ex. local CSPRNG). The focus in the question is the secure communication protocol, for a very simple webservice (simple and fast RPC).
PS: here a javascript fragment example of client-side service request: xmlHttp.open("GET", url, true) where url is the random.org link above.
Most systems provide a Cryptographically secure pseudorandom number generator (CPRNG), use that.
Per the context update use HTTPS and pin the certificate so you know there is no MITM and you are contacting the correct site. Note that you still have to trust the site.
Related
A classic dumb thing to do is pass something security related info via a GET on the query string ala:
http://foo?SecretFilterUsedForSecurity=username
...any yahoo can just use Fiddler or somesuch to see what's going on....
How safe is it to pass this info to an app server(running SSL) via a POST, however? This link from the Fiddler website seems to indicate one can decrypt HTTPS traffic:
http://fiddler2.com/documentation/Configure-Fiddler/Tasks/DecryptHTTPS
So is this equally dumb if the goal is to make sure the client can't capture / read information you'd prefer them not to? It seems like it is.
Thanks.
Yes, it's "equally dumb". SSL only protects data from being read by a third party; it does not prevent the client (or the server) from reading it. If you do not trust the client to read some data, they should not be given access to that data, even just to make a POST.
Yes, any user can easily examine the data in a POST request, even over HTTPS/SSL, using software like Burp Suite, Webscarab, or Paros Proxy. These proxies will complete the SSL transaction with the server, and then pass on the data to the client. All data passing through the proxy is stored and is visible to the client.
Perhaps you are trying to store sensitive/secret data on the client-side to lighten the load on your server? the way to do this so that the user cannot look at it (or change it) even with a proxy, is to encrypt it with a strong symmetrical secret key known only to the server. If you want to be sure that the encrypted data is not tampered with, throw on an HMAC. Make sure you use a sufficiently random key and a strong encryption algorithm and key length such as AES 256.
If you do this you can offload the storage of this data to the client but still have assurance that it has not changed since the server last saw it, and the client was not able to look at it.
This depends on who you're trying to protect your data from, and how much control you have over the client software. Fundamentally, in any client-server application the client must know what it is sending to the server.
If implemented properly, SSL will prevent any intermediary sniffing or altering the traffic without modifying the client. However, this relies on the connection being encrypted with a valid certificate for the server domain, and on the client refusing to act if this is not the case. Given that condition, the connection can only be decrypted by someone holding the private key for that SSL certificate.
If your "client" is just a web browser, this means that third parties (e.g. at a public wi-fi location) can't intercept the data without alerting the person using the site that something is suspicious. However, it doesn't stop a user deliberately by-passing that prompt in their browser in order to sniff the traffic themselves.
If your client is a custom, binary, application, things are a little safer against "nosy" users: in order to inspect the traffic, they would have to modify the client to by-pass your certificate checks (e.g. by changing the target URL, or tricking the app to trust a forged certificate).
In short, nothing can completely stop a determined user sniffing their own traffic (although you can make it harder) but properly implemented SSL will stop third-parties intercepting traffic.
The other, more important reason not to add confidential information into URL with GET requests is that the web server and any proxies on the way will log it. POST parameters don't get logged by default.
You don't want your passwords to show up in server logs - logs are usually protected much, much less than, for example, the password database itself.
I'm building a web API very similar to what StackOverflow provide.
However in my case security is importance since data is private.
I must use HTTP.
I can't use SSL.
What solution(s) do you recommend me?
EDIT: authentication != encryption
Nearly every public API works by passing an authentication token for each web request.
This token is usually assigned in one of two ways.
First, some other mechanism (usually logging into a website) will allow the developer to retrieve a permanent token for use in their particular application.
The other way is to provide a temporary token on request. Usually you have a webmethod in which they pass you a username / password and you return a limited use token based on if it is authenticated and authorized to perform any API actions.
After the dev has the token they then pass that as a parameter to every webmethod you expose. Your methods will first validate the token before performing the action.
As a side note the comment you made about "security is important" is obviously not true. If it was then you'd do this over SSL.
I wouldn't even consider this as "minimal" security in any context as it only provides a false belief that you have any sort of security in place. As Piskvor pointed out, anyone with even a modicum of interest could either listen in or break this in some way.
First of all, I suggest you read this excellent article: http://piwik.org/blog/2008/01/how-to-design-an-api-best-practises-concepts-technical-aspects/
The solution is very simple. It is a combination of Flickr like API (token based) and authentication method used by the paiement gateway I use (highly secure), but with a private password/salt instead.
To prevent unauthorized users from using the API without having to send the password in the request (in my case, in clear since there is no SSL), they must add a signature that will consist of a MD5 hashing of a concatenation of both private and public values:
Well know values, such as username or even API route
A user pass phrase
A unique code generated by the user (can be used only once)
If we request /api/route/ and the pass phrase is kdf8*s#, the signature be the following:
string uniqueCode = Guid.NewGuid().ToString();
string signature = MD5.Compute("/api/route/kdf8*s#" + ticks);
The URL of the HTTP request will then be:
string requestUrl =
string.Format("http://example.org/api/route/?code={0}&sign={1}", uniqueCode, signature);
Server side, you will have to prevent any new requests with the same unique code. Preventing any attacker to simply reuse the same URL to his advantage. Which was the situation I wanted to avoid.
Since I didn't want to store code that were used by API consumer, I decided to replace it by a ticks. Ticks represents the number of 100-nanosecond intervals that have elapsed since 12:00:00 midnight, January 1, 0001.
On server side, I only accept ticks (timestamp) with a tolerance of +-3 minutes (in case client & server are not time synchronized). Meaning that potential attacker will be able to use that window to reuse the URL but not permanently. Security is reduced a little, but still good enough for my case.
Short answer: if it's supposed to be usable through usual clients (browser requests/AJAX), you're screwed.
As long as you are using an unencrypted transport, an attacker could just remove any sort of in-page encryption code through a MITM attack. Even SSL doesn't provide perfect security - but plain HTTP would require some out-of-page specific extensions.
HTTP provides only transport - no secure identification, no secure authentication, and no secure authorization.
Example security hole - a simple HTTP page:
<script src="http://example.com/js/superstrongencryption.js"></script>
<script>
encryptEverything();
</script>
This may look secure, but it has a major flaw: you don't have any guarantee, at all, that you're actually loading the file superstrongencryption.js you're requesting. With plain HTTP, you'll send a request somewhere, and something comes back. There is no way to verify that it actually came from example.com, nor you have any way to verify that it is actually the right file (and not just function encryptEverything(){return true}).
That said, you could theoretically build something very much like SSL into your HTTP requests and responses: cryptographically encrypt and sign every request, same with every response. You'll need to write a special client (plus server-side code of course) for this though - it won't work with standard browsers.
HTTP digest authentication provides very good authentication. All the HTTP client libraries i've used support it. It doesn't provide any encryption at all.
My API (a desktop application) communicates with my web app using basic HTTP authentication over SSL (Basically I'm just using https instead of http in the requests). My API has implemented logic that makes sure that users don't send incorrect information, but the problem I have is that someone could bypass the API and use curl to potentially post incorrect data (obtaining credentials is trivial since signing up on my web app is free).
I have thought about the following options:
Duplicate the API's logic in the web app so that even if users try to cheat the system using curl or some other tool they are presented with the same conditions.
Implement a further authentication check to make sure only my API can communicate with my web app. (Perhaps SSL client certificates?).
Encrypt the data (Base 64?)
I know I'm being a little paranoid about users spoofing my web app with curl-like tools but I'd rather be safe than sorry. Duplicating the logic is really painful and I would rather not do that. I don't know much about SSL client certificates, can I use them in conjunction with basic HTTP authentication? Will they make my requests take longer to process? What other options do I have?
Thanks in advance.
SSL protects you from the man-in-the-middle attacks, but not from attacks originated on the client side of the SSL. A client certificate built into your client API would allow you to identify that data was crafted by the client side API, but will not help you figuring out if client side manually modified the data just before it got encrypted. Technically ssavy user on the client end can always find a way to modify data by debugging through your client side API. The best you can do is to put roadblocks to your client side API, to make it harder to decipher it. Validation on the server side is indeed the way to go.
Consider refactoring your validation code so that it can be used on both sides.
You must validate the data on the server side. You can throw nasty errors back across the connection if the server-side validation fails — that's OK, they're not supposed to be tripped — but if you don't, you are totally vulnerable. (Think about it this way: it's the server's logic that you totally control, therefore it is the server's logic that has to make the definitive decisions about the validity of communications.)
Using client certificates won't really protect you much additionally from users who have permission to use the API in the first place; if nothing else, they can take apart the code to extract the client certificate (and it has to be readable to your client code to be usable at all). Nor will adding extra encryption; it makes things much more difficult for you (more things to go wrong) without adding much safety over that already provided by that SSL connection. (The scenario where adding encryption helps is when the messages sent over HTTPS have to go via untrusted proxies.)
Base-64 is not encryption. It's just a way of encoding bytes as easier-to-handle characters.
I would agree in general with sinelaw's comment that such validations are usually better on the server side to avoid exactly the kind of issue you're running into (supporting multiple client types). That said, you may just not be in a position to move the logic, in which case you need to do something.
To me, your options are:
Client-side certificates, as you suggest -- you're basically authenticating that the client is who (or what, in your case) you expect it to be. I have worked with these before and mutual authentication configuration can be confusing. I would not worry about the performance, as I think the first step is getting the behavior your want (correctness first). Anyway, in general, while this option is feasible, it can be exasperating to set up, depending on your web container.
Custom HTTP header in your desktop app, checking for its existence/value on the server side, or just leveraging of the existing User-Agent header. Since you're encrypting the traffic, one should not be able to easily see the HTTP header you're sending, so you can set its name and value to whatever you want. Checking for that on the server side is akin to assuring you that the client sending the request is almost certainly using your desktop app.
I would personally go the custom header route. It may not be 100% perfect, but if you're interested in doing the simplest possible thing to mitigate the most risk, it strikes me as the best route. It's not a great option if you don't use HTTPS (because then anyone can see the header if they flip on a sniffer), but given that you do use HTTPS, it should work fine.
BTW, I think you may be confusing a few things -- HTTPS is going to give you encryption, but it doesn't necessarily involve (client) authentication. Those are two different things, although they are often bundled together. I'm assuming you're using HTTPS with authentication of the actual user (basic auth or whatever).
I am looking for something like https, but backwards. The user generates their own private key (in advance) and then (only later) provides the web application with the associated public key. This part of the exchange should (if necessary) occur out-of-band. Communication is then encrypted/decrypted with these keys.
I've thought of some strange JavaScript approaches to implement this (From the client perspective: form submissions are encrypted on their way out while (on ajax response) web content is decrypted. I recognize this is horrible, but you can't deny that it would be a fun hack. However, I wondered if there was already something out there... something commonly implemented in browsers and web/application servers.
Primarily this is to address compromised security when (unknowingly) communicating through a rogue access point that may be intercepting https connections and issuing its own certificates. Recently (in my own network) I recreated this and (with due horror) soon saw my gmail password in plain text! I have a web application going that only I and a few others use, but where security (from a learning stand point) needs to be top notch.
I should add, the solution does not need to be practical
Also, if there is something intrinsically wrong with my thought process, I would greatly appreciate it if someone set me on the right track or directed me to the proper literature. Science is not about finding better answers; science is about forming better questions.
Thank you for your time,
O∴D
This is already done. They're called TLS client certificates. SSL doesn't have to be one-way; it can be two-party mutual authentication.
What you do is have the client generate a private key. The client then sends a CSR (Certificate Signing Request) to the server, who signs the public key therein and returns it to the client. The private key is never sent over the network. If the AP intercepts and modifies the key, the client will know.
However, this does not stop a rogue AP from requesting a certificate on behalf of a client. You need an out-of-band channel to verify identity. There is no way to stop a man in the middle from impersonating a client without some way to get around that MITM.
If a rogue access point can sniff packets, it can also change packets (an ‘active’ man-in-the-middle attack). So any security measure a client-side script could possibly provide would be easily circumvented by nobbling the script itself on the way to the client.
HTTPS—and the unauthorised-certificate warning you get when a MitM is trying to fool you—is as good as it gets.
SSL and there for HTTPS allows for client certificates. on the server side you can use these environment variables to verify a certificate. If you only have 1 server and a bunch of clients then a full PKI isn't necessary. Instead you can have a list of valid client certificates in the database. Here is more info on the topic.
Implementing anything like this in JavaScript is a bad idea.
I don't see, why you are using assymetric encryption here. For one, it is slow, and secondly, it is vulnerable to man in the middle anyhow.
Usually, you use an asymmetric encryption to have a relatively secure session negotiation, including an exchange of keys for a symmetric encryption, valid for the session.
Since you use a secure channel for the negociation, I don't really understand why you even send around public keys, which themselves are only valid for one session.
Asymmetric encryption makes sense, if you have shared secret, that allows verifying a public key. Having this shared secret is signifficantly easier, if you don't change the key for every session, and if the key is generated in a central place (i.e. the server and not for all clients).
Also, as the rook already pointed out, JavaScript is a bad idea. You have to write everything from scratch, starting with basic arithmetic operations, since Number won't get you very far, if you want to work with keys in an order of magnitude, that provides reasonable security.
greetz
back2dos
I've done a little googling but have been a bit overwhelmed by the amount of information. Until now, I've been considering asking for a valid md5 hash for every API call but I realized that it wouldn't be a difficult task to hijack such a system. Would you guys be kind enough to provide me with a few links that might help me in my search? Thanks.
First, consider OAuth. It's somewhat of a standard for web-based APIs nowadays.
Second, some other potential resources -
A couple of decent blog entries:
http://blog.sonoasystems.com/detail/dont_roll_your_own_api_security_recommendations1/
http://blog.sonoasystems.com/detail/more_api_security_choices_oauth_ssl_saml_and_rolling_your_own/
A previous question:
Good approach for a web API token scheme?
I'd like to add some clarifying information to this question. The "use OAuth" answer is correct, but also loaded (given the spec is quite long and people who aren't familiar with it typically want to kill themselves after seeing it).
I wrote up a story-style tutorial on how to go from no security to HMAC-based security when designing a secure REST API here:
http://www.thebuzzmedia.com/designing-a-secure-rest-api-without-oauth-authentication/
This ends up being basically what is known as "2-legged OAuth"; because OAuth was originally intended to verifying client applications, the flow is 3-parts involving the authenticating service, the user staring at the screen and the service that wants to use the client's credentials.
2-legged OAuth (and what I outline in depth in that article) is intended for service APIs to authenticate between each other. For example, this is the approach Amazon Web Services uses for all their API calls.
The gist is that with any request over HTTP you have to consider the attack vector where some malicious man-in-the-middle is recording and replaying or changing your requests.
For example, you issue a POST to /user/create with name 'bob', well the man-in-the-middle can issue a POST to /user/delete with name 'bob' just to be nasty.
The client and server need some way to trust each other and the only way that can happen is via public/private keys.
You can't just pass the public/private keys back and forth NOR can you simply provide a unique token signed with the private key (which is typically what most people do and think that makes them safe), while that will identify the original request coming from the real client, it still leaves the arguments to the comment open to change.
For example, if I send:
/chargeCC?user=bob&amt=100.00&key=kjDSLKjdasdmiUDSkjh
where the key is my public key signed by my private key only a man-in-the-middle can intercept this call, and re-submit it to the server with an "amt" value of "10000.00" instead.
The key is that you have to include ALL the parameters you send in the hash calculation, so when the server gets it, it re-vets all the values by recalculating the same hash on its side.
REMINDER: Only the client and server know the private key.
This style of verification is called an "HMAC"; it is a checksum verifying the contents of the request.
Because hash generation is SO touchy and must be done EXACTLY the same on both the client and server in order to get the same hash, there are super-strict rules on exactly how all the values should be combined.
For example, these two lines provides VERY different hashes when you try and sign them with SHA-1:
/chargeCC&user=bob&amt=100
/chargeCC&amt=100&user=bob
A lot of the OAuth spec is spent describing that exact method of combination in excruciating detail, using terminology like "natural byte ordering" and other non-human-readable garbage.
It is important though, because if you get that combination of values wrong, the client and server cannot correctly vet each other's requests.
You also can't take shortcuts and just concatonate everything into a huge String, Amazon tried this with AWS Signature Version 1 and it turned out wrong.
I hope all of that helps, feel free to ask questions if you are stuck.