I understand there is little or no standard for implementing gzip compression for data transfers to a webserver (as opposed to the standardized option available for the response). But I'm curious what would be the direction to go if I did want to do my own implementation of this this at the server level. Is this something a custom module for IIS could handle?
Related
So we were starting a new project from scratch and one of the developers suggested why have any GET API requests as POST API's are better in every which way. (At least when using a mobile client)
On further looking into this it does seem POST can do everything GET can do and it can do it better -
slightly more secure as parameters are not in URL
larger limit than GET request
So is there even a single reason to have a GET API ? (This will only be used from a mobile client so browser specific cacheing doesn't affect us)
Is there ever a need to have GET request API as POST is better in every way?
In general, yes. In your specific circumstances -- maybe no.
GET and POST are method tokens.
The request method token is the primary source of request semantics
They are a form of meta data included in the http request so that general purpose components can be aware of the request semantics and contribute constructively.
POST is, in a sense, the wildcard method - it can mean anything. But one of the consequences of this is - because the method has unconstrained semantics, general purpose components can't do anything useful other than pass the request along.
GET, however, has safe semantics (which includes idempotent semantics). Because the request is idempotent, general purpose components know that they can resend a GET request when the server returns no response (ie messages being lost on unreliable transport); general purpose components can know that representations of the resource can be pre-fetched, reducing perceived latency.
You dismissed caching as a concern earlier, but you may want to rethink that - the cache constraint is an important element that helped the web take over the world.
Reducing everything to POST reduces HTTP from an application for transferring documents over a network to dumb transport.
Using HTTP for transport isn't necessarily wrong: Simple Object Access Protocol (SOAP) works that way, as does gRPC. You still get authorization, and conditional requests; features of HTTP that you might otherwise need to roll your own.
You aren't doing REST at that point, but that's OK; not everybody has to.
That doesn’t mean that I think everyone should design their own systems according to the REST architectural style. REST is intended for long-lived network-based applications that span multiple organizations. If you don’t see a need for the constraints, then don’t use them. (Fielding, 2008)
I got a rtsp server inside my ip camera (i think the software is live555, but doesnt really matters since rtsp is a standard), and i don't have an option to add basic auth (nor a reliable android client that supports it) so i made the endpoint to be something different than unicast or multicast, like
rtsp://www.example.com/secretcode
I know this is unsecure, but its the furthest i can go with this system, and i feel more or less comfortable with it. Comfortable unless a thought that a attacker may obtain a list of the published stream endpoints come to my mind.
Can it be?
There is no standard way to get the list of RTSP session registered in an RTSP server.
The rfc says :
The path components of the RTSP URL are opaque to the client and do
not imply any particular file system structure for the server.
However it is possible to implement a feature returning the list of valid RTSP URLs for instance returning in the body of DESCRIBE when URL doesnot exist.
Then it depends on the implementation of your RTSP server. Live555 doesnot implement (by now) a way to provide this list, however it is possible to implement it.
Obviously brute force is possible, I guess it is what you means by "I know this is unsecure".
I am architecting a project which uses jQuery to communicate with a single web service hosted inside sharepoint (this point is possibly moot but I include it for background, and to emphasize that session state is not good in a multiple front end environment).
The web services are ASP.Net ASMX services which return a JSON model, which is then mapped to the UI using Knockout. To save the form, the converse is true - JSON model is sent back to the service and persisted.
The client has unveiled a requirement for confidentiality of the data being sent to and from the server:
There is data which should only be sent from the client to the server.
There is data which should only appear in specific views (solvable using ViewModels so not too concerned about this one)
The data should be immune from classic playback attacks.
Short of building proprietary security, what is the best way to secure the web service, and are there any other libraries I should be looking at to assist me - either in JavaScript, or .Net
I'll post this as an answer...for the possibility of points :)...but I don't know that you could easily say there is a "right way" to do this.
The communications sent to the service should of course be https. That limits the man-in-the-middle attack. I always check to see that the sending client's IP is the same as the IP address in the host header. That can be spoofed, but it makes it a bit more annoying :). Also, I timestamp all of my JSON before send on the client, then make sure it's within X seconds on the server. That helps to prevent playback attacks.
Obviously JavaScript is insecure, and you always need to keep that in mind.
Hopefully that gives you a tiny bit of insight. I'm writing a blog post about this pattern I've been using. It could be helpful for you, but it's not done :(. I'll post a link sometime tonight or tomorrow.
Like a lot of developers, I want to make JavaScript served up by Server "A" talk to a web service on Server "B" but am stymied by the current incarnation of same origin policy. The most secure means of overcoming this (that I can find) is a server script that sits on Server "A" and acts as a proxy between it and "B". But if I want to deploy this JavaScript in a variety of customer environments (RoR, PHP, Python, .NET, etc. etc.) and can't write proxy scripts for all of them, what do I do?
Use JSONP, some people say. Well, Doug Crockford pointed out on his website and in interviews that the script tag hack (used by JSONP) is an unsafe way to get around the same origin policy. There's no way for the script being served by "A" to verify that "B" is who they say they are and that the data it returns isn't malicious or will capture sensitive user data on that page (e.g. credit card numbers) and transmit it to dastardly people. That seems like a reasonable concern, but what if I just use the script tag hack by itself and communicate strictly in JSON? Is that safe? If not, why not? Would it be any more safe with HTTPS? Example scenarios would be appreciated.
Addendum: Support for IE6 is required. Third-party browser extensions are not an option. Let's stick with addressing the merits and risks of the script tag hack, please.
Currently browser venders are split on how cross domain javascript should work. A secure and easy to use optoin is Flash's Crossdomain.xml file. Most languages have a Cross Domain Proxies written for them, and they are open source.
A more nefarious solution would be to use xss how the Sammy Worm used to spread. XSS can be used to "read" a remote domain using xmlhttprequest. XSS isn't required if the other domains have added a <script src="https://YOUR_DOMAIN"></script>. A script tag like this allows you to evaluate your own JavaScript in the context of another domain, which is identical to XSS.
It is also important to note that even with the restrictions on the same origin policy you can get the browser to transmit requests to any domain, you just can't read the response. This is the basis of CSRF. You could write invisible image tags to the page dynamically to get the browser to fire off an unlimited number of GET requests. This use of image tags is how an attacker obtains documnet.cookie using XSS on another domain. CSRF POST exploits work by building a form and then calling .submit() on the form object.
To understand the Same Orgin Policy, CSRF and XSS better you must read the Google Browser Security Handbook.
Take a look at easyXDM, it's a clean javascript library that allows you to communicate across the domain boundary without any server side interaction. It even supports RPC out of the box.
It supports all 'modern' browser, as well as IE6 with transit times < 15ms.
A common usecase is to use it to expose an ajax endpoint, allowing you to do cross-domain ajax with little effort (check out the small sample on the front page).
What if I just use the script tag hack by itself and communicate strictly in JSON? Is that safe? If not, why not?
Lets say you have two servers - frontend.com and backend.com. frontend.com includes a <script> tag like this - <script src="http://backend.com/code.js"></script>.
when the browser evaluates code.js is considered a part of frontend.com and NOT a part of backend.com. So, if code.js contained XHR code to communicate with backend.com, it would fail.
Would it be any more safe with HTTPS? Example scenarios would be appreciated.
If you just converted your <script src="https://backend.com/code.js> to https, it would NOT be any secure. If the rest of your page is http, then an attacker could easily man-in-the-middle the page and change that https to http - or worse, include his own javascript file.
If you convert the entire page and all its components to https, it would be more secure. But if you are paranoid enough to do that, you should also be paranoid NOT to depend on an external server for you data. If an attacker compromises backend.com, he has effectively got enough leverage on frontend.com, frontend2.com and all of your websites.
In short, https is helpful, but it won't help you one bit if your backend server gets compromised.
So, what are my options?
Add a proxy server on each of your client applications. You don't need to write any code, your webserver can automatically do that for you. If you are using Apache, look up mod_rewrite
If your users are using the latest browsers, you could consider using Cross Origin Resource Sharing.
As The Rook pointed out, you could also use Flash + Crossdomain. Or you could use Silverlight and its equivalent of Crossdomain. Both technologies allow you to communicate with javascript - so you just need to write a utility function and then normal js code would work. I believe YUI already provides a flash wrapper for this - check YUI3 IO
What do you recommend?
My recommendation is to create a proxy server, and use https throughout your website.
Apologies to all who attempted to answer my question. It proceeded under a false assumption about how the script tag hack works. The assumption was that one could simply append a script tag to the DOM and that the contents of that appended script tag would not be restricted by the same origin policy.
If I'd bothered to test my assumption before posting the question, I would've known that it's the source attribute of the appended tag that's unrestricted. JSONP takes this a step further by establishing a protocol that wraps traditional JSON web service responses in a callback function.
Regardless of how the script tag hack is used, however, there is no way to screen the response for malicious code since browsers execute whatever JavaScript is returned. And neither IE, Firefox nor Webkit browsers check SSL certificates in this scenario. Doug Crockford is, so far as I can tell, correct. There is no safe way to do cross domain scripting as of JavaScript 1.8.5.
UPDATE : GWT 2.3 introduces a better mechanism to fight XSRF attacks. See http://code.google.com/webtoolkit/doc/latest/DevGuideSecurityRpcXsrf.html
GWT's RPC mechanism does the following things on every HTTP Request -
Sets two custom request headers - X-GWT-Permutation and X-GWT-Module-Base
Sets the content-type as text/x-gwt-rpc; charset=utf-8
The HTTP request is always a POST, and on server side GET methods throw an exception (method not supported).
Also, if these headers are not set or have the wrong value, the server fails processing with an exception "possibly CSRF?" or something to that effect.
Question is : Is this sufficient to prevent CSRF? Is there a way to set custom headers and change content type in a pure cross-site request forgery method?
If this GWT RPC is being used by a browser then it is 100% vulnerable to CSRF. The content-type can be set in the html <form> element. X-GWT-Permutation and X-GWT-Module-Base are not on Flash's black list of banned headers. Thus it is possible to conduct a CSRF attack using flash. The only header element you can trust for CSRF protection is the "referer", but this isn't always the best approach. Use token based CSRF protection whenever possible.
Here are some exploits that i have written which should shed some light on the obscure attack i am describing. A flash exploit for this will look something like this and
here is a js/html exploit that changes the content-type.
My exploit was written for Flex 3.2 and the rules have changed in Flex 4 (Flash 10) Here are the latest rules, most headers can be manipulated for requests POST only.
Flash script that uses navigateTo() for CSRF:
https://github.com/TheRook/CSRF-Request-Builder
GWT 2.3 introduces a better mechanism to fight XSRF attacks. See GWT RPC XSRF protection
I know I asked this question, but after about a days research (thanks to pointers from Rook!), I think I have the answer.
What GWT provides out-of-the-box will not protect you from CSRF. You have to take steps documented in Security for GWT Applications to stay secured.
GWT RPC sets "content-type" header to "text/x-gwt-rpc; charset=utf-8". While I didn't find a way to set this using HTML forms, it is trivial to do so in flash.
The custom headers - X-GWT-Permutation and X-GWT-Module-Base, are a bit more tricky. They cannot be set using HTML. Also, they cannot be set using Flash unless your server specifically allows it in crossdomain.xml. See Flash Player 10 Security.
In addition, when a SWF file wishes to
send custom HTTP headers anywhere
other than its own host of origin,
there must be a policy file on the
HTTP server to which the request is
being sent. This policy file must
enumerate the SWF file's host of
origin (or a larger set of hosts) as
being allowed to send custom request
headers to that host.
Now GWT's RPC comes in two flavours. There is the old, custom-serialization format RPC, and the new, JSON based de-RPC. AFAICT, client code always sets these request headers. The old style RPC doesn't now enforce these headers on server side, and thus a CSRF attack is possible. The new style de-RPC enforces these headers, and thus it may or may not be possible to attack them.
Overall, I'd say if you care about security, make sure you send strong CSRF tokens in your request, and don't rely on GWT to prevent it for you.
I'm not sure, if there's an easy way (I'd be extremely interested in finding that out, too!), but at least there seem to be some advanced ways to achieve arbitrary cross site requests with arbitrary headers: http://www.springerlink.com/content/h65wj72526715701/ I haven't bought the paper, but the abstract and introduction do sound very interesting.
Maybe somebody here already read the full version of the paper, and can expand a little bit?