I have a webserver which only allows downloading of files when you pass along a token on the querystring. This token (basically a hash) is compared to the hash result of the filepath + remote ip.
This has worked perfectly! However since a project of mine has gained more public interest obscure issues have started to show.
People retrieve the signed urls from an API. API load balances the request and returns a valid URL for the requesting IP. However, some people have some really funky network setups and can basically change their IP address from request to request! Eg. Some university networks route all internet through a loadbalanced proxy.
This is very problematic... I have been trying to come up with an alternative but I'm kinda stuck. The only thing I can come up with is an expiring link instead which is not IP bound but that brings other kinds of issues.
What kind of options do I have other than expiring URLs? What could I do to still use the IP based system? I cannot trust proxy headers. I need to work with information the users cant directly manipulate...
I have also thought about generating a token for an IP range instead. But its not ideal and only solves the problem when the IPs are within range (of which I have already seen cases). So I'm stuck at the moment.
Would love to read about what you guys might suggest.
What you are after is impossible I'm afraid.
You basically want to authenticate a user (the person) without.. well, authenticating the user. Anything you have in the request can be forged by the client, which means another client can make the very same request whatever you do, as long as clients cooperate and share data. The only difference that is somewhat harder to forge over TCP is the client IP, which on the other hand changes sometimes as you also noted.
So there is no cryptographic (=reasonably secure) solution.
If you want to make it somewhat harder for an attacker and your clients are browsers, you can try fingerprinting the browser. That's suprisingly accurate in many cases, but obviously can also be forged. It does raise the bar a bit for people trying to share links though.
Related
I am creating a Chromium/Electron based Mac app. The app is essentially a browser for my customers to use a web service that I have no control over. My requirement is that users of my app (who may have root access on their Mac) should not be able to view the URLs the app is visiting, and should be unable to gain access to the cookies the app is storing. Normally it is not hard to MITM yourself, or attach a debugger to an app and dump memory to see the URLs and cookies.
How can I prevent these types of leaks to the user? If it's impossible, it may be acceptable to make it very hard so that a very high level of sophistication is needed.
Your users have full control of their devices, it is not possible to securely prevent them from proxying or exploring what your client-side app does. Obfuscation would seem like an option, but in the end, the http request that leaves your app will traverse the whole OS through different layers, and your user can easily observe that, if not else then in network packets (but usually much easier).
The only way it is possible to prevent the user from knowing what's happening is if you have your own backend. The frontend app (Electron) would make a request to your backend, which in turn could make any request with any parameters without the user being aware.
Note though that your backend could still be used as a proxy or oracle just like if the user was connecting to the real service. This might or might not be a problem in your case, depending on what you actually want to achieve and why.
The app is essentially a browser for my customers to use a web service that I have no control over. My requirement is that users of my app (who may have root access on their Mac) should not be able to view the URLs the app is visiting, and should be unable to gain access to the cookies the app is storing
Basically, you cannot (you could with the appropriate infrastructure. But you lack that infrastructure).
Network communications can be secured, to a point, using HTTPS (if you can't even use that, then you're completely out of luck - users wouldn't even need root access to the Mac to sniff traffic). You need to verify the server certificate to be sure you're connecting to the correct server.
One thing you might do - effectual just against wannabes, I'm afraid - is first run a test API call on some random server and verify that the connection either fully succeeds, with the proper server identification and matching IP, if the server exists, or that it properly fails if the server never existed. Anything else would be a telltale that someone has taken over the network layer, and at that point you could connect to a different server, making different calls, and lament that the server isn't answering properly.
Strings in memory can be (air quote) protected (end air quote) by having them available only for the shortest time, and otherwise stored in a different form - you can have for example an URL and a random byte sequence with the same length, then store the sequence and the XOR of the URL and the sequence. You can then reconstruct the URL every time you need it, remembering to clear it off any app caches it might find its way into. Also, just for the lols, you can keep a baker's dozen of different URLs sprinkled in the clear throughout the code. A memory dump at that point will turn out nothing useful.
Files, of course, can be encrypted with any one of several schemes - the files residing on the same machine that has to know how to decode them makes all such schemes ultimately vulnerable, but there again, you can try and obfuscate things. I once stored some information in a ZIP file - but it was just the header of an encrypted ZIP file, with the appropriate directory entry block glued at the end. The data were actually just gzipped in the clear, there was no password whatsoever. The guys that tried to decode the file thought it was a plain encrypted Zip file with the extension changed, wasted a significant amount of time with several Zip cracking tools, and ended up owing me a beer.
More than that, there is not much that can realistically be done.
A big advantage would be in outsourcing the API calls and "cookie" maintenance to an external service that you control, e.g. on Amazon AWS or Azure or similar. Then you could employ all kinds of protection schemes (for example: all outbound API calls could be stored in an opaque object, timestamped, nonced, and encrypted with your server's public key, and the responses sent encrypted with your client's unique key). Since this is relatively simple and cost-effective, it would also be my recommendation.
I have cors installed and only my website is whitelisted, how reliable is this? Can bad actors still call my api if they are not calling it from my website?
Next I want to rate limit each user on my website, (the users are not registered or signed in),
I want to restrict each user to make no more than 1 request per second.
How can each user be identified? and then how can each user be limited?
Too many separate questions packaged together here. I'll tackle the ones I can:
I have cors installed and only my website is whitelisted, how reliable is this? Can bad actors still call my api if they are not calling it from my website?
CORS only works with cooperating clients. That means browsers. Your API can be used by anybody else with a scripting tool or any programming language or even a tool like CURL. So, CORS does not prevent bad actors at all. The only thing it prevents is people embedding calls to your API in their own web page Javascript. It doesn't prevent anyone from accessing your API programmatically from whatever tool they want. And, they could even use your API in their own web-site via a proxy. It's not much protection.
How can each user be identified? and then how can each user be limited?
Rate limiting works best when there's an authentication credential with each request because that allows you to uniquely identify each request and/or ban or delay credentials
that misbehave. If there are no credentials, you can try to cookie them to track a given user, but cookies can be blocked or thrown away even in browsers to defeat that. So, without any sort of auth credential, you're stuck with just the requesting IP address. For some users (like home users), that's probably sufficient. But, for corporate users, many, many users may present as the same corporate IP address (due to how their NAT or proxy works), thus you can't tell one user at a major company from another purely by IP address. If you had a lot of users from one company simultaneously using the site, you could falsely trigger rate limiting.
I am building a Web Application using Angular 2 and the backend service built in ASP.NET Core Web API.
For authentication, I am thinking of using JWT and storing the token in a Secure HttpOnly Cookie.
For extra security, I am also thinking of capturing the IP Address for the user on the initial login and on each request after the initial login, revoking the token if the IP Address changes.
So the questions I have are:
Is this extra level of security worth it?
Will there be any problems with the IP check I am thinking of using? Based what I know about networking, I don't think an IP Address will legitimately change between request. Even if it does, I think it would be very rare. However I am not going to pretend I know enough about networking to confirm that.
Edit 1
(In response to an answer).
Thank you for answering my question. I have responded to a few of your responses.
My initial thought was that using JWT in a cookie to connect to an API is not the typical use case, why don't you use a standard MVC app then, but that's not your question and actually it's equally secure as long as the token is in a secure, httponly cookie (and of course the implementation is correct). It's just a bit unusual I think.
I am not sure why you consider using cookies this way unusual?
Is it because most of the time cookies are used for session state? I personally think storing a token in a secure cookie instead of keeping the token in a http header or local storage should be a very typical use case because of how much more secure it is. Unless I am missing something?
So I guess I will ask what is the disadvantage of doing it this way?
It depends. If you are worried about session theft, probably yes. If you keep the token in an httponly cookie (protected against xss), that's more secure than a token anywhere else, but still, your threat model may show different threats and validate your concern. The usual problem is you can't do this, see below.
This application will be dealing with a lot of PPI information so I do have a concern on token theft.
Most probably, there will be problems. It depends on your users, how and from where they use your application. If they use mobile devices, IP addresses will change a lot and such a solution is out of the question. If they are corporate users in a company internal network, it can be feasible. Anything inbetween is a gray area. A typical home user will have their IP changed once in a while, most people get dynamic IP allocation from their internet providers. An IP lease typically lasts a few weeks (at least where I live), but ISPs can configure it any way they want, it can be a day or even shorter.
My impression with IP address lease renew is majority of the time the client gets the same IP address. However I should not make that assumption I suppose?
However I can see this can be more of a problem with mobile devices. Some of the clients will be on the road often so this is a good point you have made that can become a problem.
One typical solution you can choose to do is offer this option on the login screen. If a user chooses to use IP address validation, he opts for greater security but accepts the fact that sometimes he may have to log in again. Or he can choose lower security with his session being more stable. Whether it's worth to explain this to your users is I think a business decision.
Never thought about giving the client an option which does sound like a good idea.
Edit 2
(In response to an answer).
Also I'm not sure whether your JWT only has a session id or if your server is stateless and all session data is in the JWT. In the first case, you don't even need the JWT, you could just pass the session id as normal, and standard .Net MVC does that for you. If it's session data too, JWTs are unencrypted by default, so session contents will be visible to endusers, which may or may not be a problem. (And a JWT is protected from tampering by its signature, so it's only about confidentiality, not integrity). Storing session data in the JWT and the JWT in the cookie may also face cookie size issues, depending on your target browsers.
My backend ASP.NET Core Web API will be stateless. The decision has already been made to use Angular so discussing is a moot point.
As for why I think using a JWT this way is a little unusual: I think JWTs are mostly used when tokens need to be passed to different URLs (to different services). For this purpose, httpOnly cookies are obviously inadequate because of the same origin rule. If you can afford using httpOnly cookies, you could just store your session info on the server side.
A much as I would like to discuss the above topic because my solution could be flawed, I think the powers that be may close this post for getting off topic?
Might be more appropriate to ask a new question targeted toward the above subject?
As for lease renews resulting in the same IP: Well, they don't always. It depends on your business case, but some ISPs give you IPs only for a short time. If it's ok for your users to get logged out once in a while, then it may be ok for wired (home) users. And it is definitely a big problem with mobile devices.
My initial thought was that using JWT in a cookie to connect to an API is not the typical use case, why don't you use a standard MVC app then, but that's not your question and actually it's equally secure as long as the token is in a secure, httponly cookie (and of course the implementation is correct). It's just a bit unusual I think.
On to the point, your question is very valid as is your concern about problems.
Is this extra level of security worth it?
It depends. If you are worried about session theft, probably yes. If you keep the token in an httponly cookie (protected against xss), that's more secure than a token anywhere else, but still, your threat model may show different threats and validate your concern. The usual problem is you can't do this, see below.
Will there be any problems with the IP check I am thinking of using?
Most probably, there will be problems. It depends on your users, how and from where they use your application. If they use mobile devices, IP addresses will change a lot and such a solution is out of the question. If they are corporate users in a company internal network, it can be feasible. Anything inbetween is a gray area. A typical home user will have their IP changed once in a while, most people get dynamic IP allocation from their internet providers. An IP lease typically lasts a few weeks (at least where I live), but ISPs can configure it any way they want, it can be a day or even shorter.
So reality is if you have a normal, usual userbase, you will most probably run into problems.
One typical solution you can choose to do is offer this option on the login screen. If a user chooses to use IP address validation, he opts for greater security but accepts the fact that sometimes he may have to log in again. Or he can choose lower security with his session being more stable. Whether it's worth to explain this to your users is I think a business decision.
Update in response to Edit 1 :)
As for why I think using a JWT this way is a little unusual: I think JWTs are mostly used when tokens need to be passed to different URLs (to different services). For this purpose, httpOnly cookies are obviously inadequate because of the same origin rule. If you can afford using httpOnly cookies, you could just store your session info on the server side. Also I'm not sure whether your JWT only has a session id or if your server is stateless and all session data is in the JWT. In the first case, you don't even need the JWT, you could just pass the session id as normal, and standard .Net MVC does that for you. If it's session data too, JWTs are unencrypted by default, so session contents will be visible to endusers, which may or may not be a problem. (And a JWT is protected from tampering by its signature, so it's only about confidentiality, not integrity). Storing session data in the JWT and the JWT in the cookie may also face cookie size issues, depending on your target browsers.
As for lease renews resulting in the same IP: Well, they don't always. It depends on your business case, but some ISPs give you IPs only for a short time. If it's ok for your users to get logged out once in a while, then it may be ok for wired (home) users. And it is definitely a big problem with mobile devices.
I think you can do it with JWT and IP. When the user logs in. Capture the IP for the length of the session. At every login Capture IP then use that to validate the Token is from the owner who started the session. If another IP hits the system. force a revalidate and new token. IP+JWT+Password = login. If you had mobile apps that required 1 login and always remember the login. User never has to enter login again. Then cache the userid\password in the application {securely} and then resend it automatically when the IP changes. JWT is secure when using SSL Difference between SSL and JWT
Sorry for reviving this, but lately I have been thinking a lot about encryption and security and thought of something (that I guess is pretty similar to what HTTPS does)
When user logs in, the server responds with a normal greeting (user info, JWT and whatever other data you need to pass) + you will pass a public key
Have a backend that supports any asymmetric encryption method (I like RSA) and have your front (also needs to run the same encryption method) end receive the public key, encrypt the data, and send it to the server with every subsequent request.
If any of the data that the user needs to provide changes, revoke.
You can even keep track of a clock, if its off by too much, revoke.
For extra layer, have the client transmit a public key on login/signup and boom, hermetic comms like a hazmat suit.
I want to host copyrighted data on a Amazon S3 bucket (to have a larger bandwidth available than what my servers can handle) and provide access to these copyrighted data for a large numbers of authorized clients.
My problem is:
i create signed expiring HTTPS URL for these resources on the server side
these URL are sent to clients via a HTTPS connection
when the client uses these URL to download the contents, the URL can be seen in clear for any man-in-the-middle
In details, the URL are created via a Ruby On Rails server using the fog gem.
The mobile clients I'm talking about are iOS devices.
The proxy I've used for my test is mitmproxy.
The URL I generated looked like this:
https://mybucket.s3.amazonaws.com/myFileKey?AWSAccessKeyId=AAA&Signature=BBB&Expires=CCC
I'm not a network or security expert but I had found resources stating nothing was going clear over HTTPS connections (for instance, cf. Are HTTPS headers encrypted?). Is it a misconfiguration of my test that led to this clear URL? Any tip on what could have gone wrong here? Is there a real chance I can prevent S3 URL to go clear over the network?
So firstly, when sending a request over SSL all parameters are encrypted. If you were to look at the traffic going through a normal proxy you wouldn't be able to read them.
However, many proxies allow interception of SSL data by creating dummy certificates. This is exactly what mitmproxy does. You may well have enabled this and not realised it (although you would have had to install a client-side certificate to do this).
The bottom line is that your AWS URLs could be easily intercepted by somebody looking to reverse engineer your app, either through a proxy or by tapping into the binary itself. However, this isn't a 'bad thing' per se: Amazon themselves know this happens, and that's why they're not sending the secret key directly in the URL itself, but using a signature.
I don't think this is a huge problem for you: after all, you're creating URLs that expire, so even if someone can get hold of them through a proxy they'll only be able to access the URL for as long as it is valid. To access your resources post-expiry would require direct access to your secret key. Now, it actually turns out this isn't impossible (since you've probably hard-coded it into your binary), but it's difficult enough that most users won't be bothering with it.
I'd encourage you to be realistic with your security and copyright prevention: when you've got client-side native code it's not a matter of if it gets broken but when.
I'm currently developing a web application that has one feature while allows input from anonymous users (No authorization required). I realize that this may prove to have security risks such as repeated arbitrary inputs (ex. spam), or users posting malicious content. So to remedy this I'm trying to create a sort of system that keeps track of what each anonymous user has posted.
So far all I can think of is tracking by IP, but it seems as though it may not be viable due to dynamic IPs, are there any other solutions for anonymous user tracking?
I would recommend requiring them to answer a captcha before posting, or after an unusual number of posts from a single ip address.
"A CAPTCHA is a program that protects websites against bots by generating and grading tests >that humans can pass but current computer programs cannot. For example, humans can read >distorted text as the one shown below, but current computer programs can't"
That way the spammers are actual humans. That will slow the firehose to a level where you can weed out any that does get through.
http://www.captcha.net/
There's two main ways: clientside and serverside. Tracking IP is all that I can think of serverside; clientside there's more accurate options, but they are all under user's control, and he can reanonymise himself (it's his machine, after all): cookies and storage come to mind.
Drop a cookie with an ID on it. Sure, cookies can be deleted, but this at least gives you something.
My suggestion is:
Use cookies for tracking of user identity. As you yourself have said, due to dynamic IP addresses, you can't reliably use them for tracking user identity.
To detect and curb spam, use IP + user browser agent combination.