I am using Node.js and Express.js 3.x.
As one of our authorization headers we are passing in the username. Some of our usernames contain umlaut characters: ü ö ä and the likes of.
For usernames with just 'normal' characters, all works fine. But when a jörg tries to make a request, the server doesn't recognize the umlaut character in the header.
Trying to simulate the problem I:
Created some tests that set the username header with the umlaut character. These tests pass, they are able to pass in the umlaut correctly.
Used 'postman' and 'advanced rest client' Chrome extensions and made the request manually against the server - in this case it failed. I saw the server is unable to recognize the umlaut character, it juts interpreted it as some sort of ?.
Is there any limitation on custom HTTP header values characters that forbids using these kind of characters? Any idea why it would work in tests but not from my browser extension? Am I forgetting to set some character set somewhere?
Summary of what was written in the other related question and in comments:
You may put any 'printable' ASCII char in your custom header value field.
If you want to use special characters, encode these characters following whatever rules/charset/encoding you choose. As long as this encoding it uses simple ASCII chars, it's OK. An example is using UTF-8 and encoding string chars to \u%%.
On the server side - you must manually make sense out of the encoded string, probably by decoding it using the rules of the character set/encoding paradigm you chose before.
Related
We are currently using OWASP Antisamy project to protect our application against XSS attacks. Every input field is sanitized when any given form is submitted to server. It works fine, but we have issues with the fields like company name, organization name, etc.
Ex: Ampersand is escaped for AT&T and the company name is displayed wrong (displayed with escaped characters).
We manually update the fields on database to fix this issue. However, this is a pain in the neck as you can imagine.
Is there a way to address this using OWASP antisamy or should we use a different library?
You should only encode on output, not on input. If a user enters AT&T in your application, this should be stored at AT&T in the database. There is no need to encode it, but of course make sure that you are using parameterised queries which will prevent characters such as ' from breaking out of the SQL command context and causing SQL injection.
When you output, this is the only time you need to encode characters. e.g. AT&T should be encoded as AT&T when output to HTML, so it is displayed in the browser as AT&T.
It seems like your application is encoding the input and also encoding the output, so strings like above will be double encoded at then output as AT&T in your HTML, causing the problem. Remove your input encoding to solve this.
The reason you should only encode when output is that if you decide you want to output data to a different format such as JSON or JavaScript, then the encoding is different. O'leary would become O\x27leary if encoded properly for JavaScript, which would not display properly in HTML where the encoding is O'leary.
In a Grails architecture, we have some requests where it is possible to send put in %0d and %0a in the request paramaters. This is Ascii encoded text for newline and carriage return.
In this request the parameters sent over are also returned. This means we should be sending back %0a %0d but we are not. You can actually see the new line and carriage return in the response.
I am told this is a security risk because we interpret the text. IS this correct?
And is there a grails solution whereby you ensure none of Controllers whether they be returning Json to Ajax request or Model to a GSP decode the encoded text.
Thanks
Look at the security section of the grails docs - especially the XSS Prevention section.
If you're not encoding the request parameters - users of your app could inject own code to attack your application.
I've no details about your app, but you should encode your parameter as html.
You could set this as default within your Config.groovy:
grails.views.default.codec = "html"
If you set this default you have to be aware of double encoding.
This could happen if a tagLib or a plugin also encodes the parameter as html explicitly.
I have a question about encoding/decoding strings.
Well, there is web page, where I send some data with simple php POST form.
When I open Chrome Developer Toolbar -> Network, in "Form Data" all parameters are displayed normally, except this, "uid", which is encoded ( %25%DC%BE%60%A0W%94M ) somehow.
When I clicked on "view URL encoded", it showed me this "%2525%25DC%25BE%2560%25A0W%2594M", I tried online tools such http://meyerweb.com/eric/tools/dencoder/ to get human readable string of this encoded parameter, but no luck.
Can anyone explain to me, how can I get the original value of this parameter? Not encoded, in human readable format?
Thanks a lot : )
This decoder works better:
http://www.opinionatedgeek.com/dotnet/tools/urlencode/Decode.aspx/
The %25 that you see is the actual percent character % being encoded
http://en.wikipedia.org/wiki/Percent-encoding
Percent-encoding, also known as URL encoding, is a mechanism for
encoding information in a Uniform Resource Identifier (URI) under
certain circumstances.
...
it is also used in the preparation
of data of the "application/x-www-form-urlencoded" media type, as is
often used in the submission of HTML form data in HTTP requests.
If you're having problems with online decoders, and (seeing as its a relatively short string) why not give it a go by hand?
http://www.degraeve.com/reference/urlencoding.php
This table maps characters to their URL-encoded equivalent, just do a Ctrl+F of the % encoded characters and decode it yourself.
A few of the characters look wierd because they aren't English characters. %DC is Ü for example. its possible the encoders you are trying don't recognise non-english characters
In my application I am creating a activation key of 64 characters.
It is having special characters like
+ =
While framing url we are doing url encode. But if user do a copy of the url from his email client , in some client url is getting url decoded.
Some suggestion is to remove these special character with charters which don't require url encode.
Will it going to create any issue in security as character set will be limited ?
If you use just alphanumeric characters, and make it case-insensitive, you will have 36 possible characters. Since your key is 64 characters long, you end up with a keyspace that is quite big (Larger than a GUID).
Depending what you are trying to protect this against (I assume brute forcing legitimate keys), so long as your algorithm for generating them is sufficient, you should not be at any risk of generating keys that are easily guessed.
We'd like to double-check our http headers for security before we send them out. Obviously we can't allow '\r' or '\n' to appear, as that would allow content injection.
I see just two options here:
Truncate the value at the newline character.
Strip the invalid character from the header value.
Also, from reading RFC2616, it seems that only ascii-printable characters are valid for http header values Should also I follow the same policy for the other 154 possible invalid bytes?
Or, is there any authoritative prior art on this subject?
This attack is called "header splitting" or "response splitting".
That OWASP link points out that removing CRLF is not sufficient. \n can be just as dangerous.
To mount a successful exploit, the application must allow input that contains CR (carriage return, also given by 0x0D or \r) and LF (line feed, also given by 0x0A or \n)characters into the header.
(I do not know why OWASP (and other pages) list \n as a vulnerability or whether that only applies to query fragments pre-decode.)
Serving a 500 on any attempt to set a header that contains a character not allowed by the spec in a header key or value is perfectly reasonable, and will allow you to identify offensive requests in your logs. Failing fast when you know your filters are failing is a fine policy.
If the language you're working in allows it, you could wrap your HTTP response object in one that raises an exception when a bad header is seen, or you could change the response object to enter an invalid state, set the response code to 500, and close the response body stream.
EDIT:
Should I strip non-ASCII inputs?
I prefer to do that kind of normalization in the layer that receives trusted input unless, as in the case of entity-escaping to convert plain-text to HTML escaping, there is a clear type conversion. If it's a type conversion, I do it when the output type is required, but if it is not a type-conversion, I do it as early as possible so that all consumers of data of that type see a consistent value. I find this approach makes debugging and documentation easier since layers below input handling never have to worry about unnormalized inputs.
When implementing the HTTP response wrapper, I would make it fail on all non-ascii characters (including non-ASCII newlines like U+85, U+2028, U+2029) and then make sure my application tests include a test for each third-party URL input to makes sure that any Location headers are properly %-encoded before the Location reaches setHeader, and similarly for other inputs that might reach the request headers.
If your cookies include things like a user-id or email address, I would make sure the dummy accounts for tests include a dummy account with a user-id or email address containing a non-ASCII letter.
The simple removal of new lines \n will prevent HTTP Response Splitting. Even though a CRLF is used as a delimiter in the RFC, the new line alone is recognized by all browsers.
You still have to worry about user content within a set-cookie or content-type. Attributes within these elements are delimited using a ;, it maybe possible for an attacker to change the content type to UTF-7 and bypass your XSS protection for IE users (and only IE users). It may also be possible for an attacker to create a new cookie, which introduces the possibility of Session Fixation.
Non-ASCII characters are allowed in header fields, although the spec doesn't really clearly say what they mean; so it's up to sender and recipient to agree on their semantics.
What made you think otherwise?