Express eTag always changing - node.js

I have a rest like API through Node Express.
The ETag is default, not explicitly turned on or off. However whenever I test hitting the server, it always gives me a new ETag, even if the returned JSON/HTML is exactly the same. I also checked the returned header and they look the same. I tested this with two types of content, an API and a static HTML content like a privacy page.
Any idea how to check what's making it different each time?

Express' default behavior is to provide a "strong"-ly validated etag which will only be the same as a previous response if the current response is precisely the same, byte-for-byte.
You could try setting express' etag to weally validate the response, which indicates to the browser that the current response is semantically equivalent as a previous one with the same value, that is, while they might not be byte-for-byte the same, they encapulate or represent the same meaning. To do this, use app.set('etag','weak')
Finally, if this doesn't work for you, you can create your own etag validation function using app.get('etag',function(body,encoding){...}) where you return a hash generated from your content; this allows you to control what express (and thus, the browser) considers being different means in the context of your response.
More than you ever wanted to know about etags can be found at Wikipedi:HTTP_ETag

Related

Express doesn't allow changing the CSP header

For some reason the latest Express.JS versions are forcing a default, restrictive CSP (Content-Security-Policy) header value.
I'm trying to instantiate a middleware in order to change CSP to a more permissive one (that's currently on my needs for the project) but Express.JS seems to ignore every value for the Content-Security-Policy header. Calling res.setHeader("...", "some value") does work on that middleware when I change the key name to everything but "Content-Security-Policy". I'm not defining this header anywhere else, so it seems to come from Express.JS itself. What's exactly going on, how to correctly make Express.JS to recognize it?
Example: requesting the main page shows the default restrictive header for CSP, ignoring the value I set on line 29
Another example: changing the header to another name that's not CSP correctly enlists it to the headers

Cache-Control headers - ignore query parameters

I am setting up a Cloudfront distribution for my companies website.
We would like to set the caching time by using the Cache-Control headers on the server-side (Node.Js with Express), like this:
if (req.url.startsWith('/static')) {
res.setHeader('Cache-Control', 'public,max-age=500');
}
At first, this seems to work well, but one of the criteria for the cache is failing, and that is, to ignore query string parameters.
For example, the request "domain.com/static/logo" and "domain.com/static/logo?foo=bar" should be interpreted as the same resource, and cached as one.
I wonder if it is possible to cache a resource while ignoring its query string parameters, using only the Cache-Control headers.
Thank you.
Bydefault CloudFront does remove the query string and also doesn't consider it into the cache , this is a default behaviour of CloudFront so that there are not multiple cache copies based on different query string parameter.
If you don't seem this behaviour, you may have "Query string" set to Forward all and cache based on call in CloudFront's cache behaviour.
https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/QueryStringParameters.html

Azure Active Directory OAuth 2.0 Authorization gives Bad Request

When requesting an authorization code, if the state url parameter has following value, https://login.microsoftonline.com/oauth2/authorize gives me a Bad Request.
state=%3C%3CMULE_EVENT_ID%3D0-6cadfe22-e9ea-11e6-99ff-205120524153%3E%3E
If I remove the encoded values: << and >>, it works well. Currently I have some limitations and I cannot remove those values.
In the documentation is says that "state" is a value included in the request that will also be returned in the token response. It can be a string of any content that you wish.
The double << >> appears to be semantically incorrect, although those characters are allowed in https://www.rfc-editor.org/rfc/rfc6749#appendix-A.5 (referencing ABNF syntax for that field, which is essentially all printable characters including space, VSCHAR, https://www.rfc-editor.org/rfc/rfc5234).
However, when we look at the intended use of the state field, it is to be used to send a token back from the service, for your application to be able to validate the local state to avoid CSRF attacks.
In most cases, a short string should suffice, and you will probably do yourself a favor if you keep the string short, saving bytes on the wire and additional parsing overhead.
There is a good overview of using the oauth2 endpoint with here (admittedly with Bing Ads, but the principals and advice are applicable to this case):
https://msdn.microsoft.com/en-us/library/bing-ads-user-authentication-oauth-guide.aspx
If I can find the exact restrictions on the state field, I shall update my answer.
Well, the documentation seems a bit wrong then. I tested various state strings, and what makes it fail consistently is starting the state string with %3C. So a less-than sign is fine in some places in the string.
EDIT: There is something really odd going on.
This fails:
state=MUL%3CE_EVENT_ID%3D0-6cadfe22-e9ea-11e6-99ff-205120524153%3E%3E
But this works:
state=MULE%3C_EVENT_ID%3D0-6cadfe22-e9ea-11e6-99ff-205120524153%3E%3E
But this also fails:
state=MULE_%3CEVENT_ID%3D0-6cadfe22-e9ea-11e6-99ff-205120524153%3E%3E
My theory is that it doesn't allow anything that looks like a valid HTML tag. That's why it would allow %3C_....%3D, but *%3Ca%3e is not. You can replace a with any characters a-z. So HTML elements are a no-no :)

Handling POST parameter differences on identical URLs in Fiddler recording?

I need to record how my client side scripts act in a 3rd party web app. So I am trying to use Fiddler to record the traffic on the 3rd party's machine and then run it here.
Sounds great, but my scripts make AJAX calls to an ASPX (123.aspx) page, and the calls use only POST params, not GET.
This means that the (123.aspx) request URLs recorded in Fiddler are identical, and playback doesn't work properly (every AJAX request matches the first recorded match, not the one with the same POST params).
E.g. let's says the requests are recorded like this
123.aspx [POST param: searchquery=xyz]
123.aspx [POST param: searchquery=abc]
then when I playback the SAZ file, I always get the response for 123.aspx [POST param: searchquery=xyz], even if searchquery=abc.
How can I get Fiddler to treat requests differently if the POST params are different?
I saw extraction rules, and was a little unsure about them, the Telerik documentation returns no results for 'extraction'... are they what I need?
Thanks
Please see the introductory blog post for details about Fiddler AutoResponder's Import for Playback mode which helps address some problems you may encounter when trying to playback a previously-captured SAZ file.
Now, the "Import for playback" mode will not address all possible problems; for instance, if you have multiple POSTs to the same URL captured but their ordering is not the same as was captured when you try to replay the scenario, they will play back in the wrong order and things will be broken.
From the Fiddler Book:
Matching Against Request Bodies
In some cases, a site may use the same request URL for many unrelated operations, specifying the operation desired in the request’s body instead of the URL. You may extend your Match Condition to examine a POST or PUT request’s body by specifying the URLWithBody: prefix for your Match Condition. When this prefix is used, the portion of the string up to the first space character is used as the Match Condition for the request’s URL, while the remainder of the string is used as a Match Condition for the string-representation of the request’s body. For performance reasons, you should specify the URL portion of the Match Condition as narrowly as possible to minimize the number of request bodies that the AutoResponder needs to evaluate. If a request has no body, it will not match any URLWithBody rule.
Your Match Condition may specify the EXACT:, NOT:, and REGEX: prefixes for both the URL and the body. For example:
URLWithBody:upload.php TextToFindInBody
URLWithBody:login.php EXACT:Action=Login
URLWithBody:ping.php NOT:POST Data I Do Not Care About
URLWithBody:EXACT:https://example.com/upload.php REGEX:^.+TextToFind.*$
URLWithBody:REGEX:^.+/upload.php.*$ REGEX:^.+TailOfPOST$
Keep in mind that most POSTs from Web Forms encode the body text, so you should ensure that your Match Condition accounts for such encoding. For instance, to match the following POST:
POST http://www.enhanceie.com/sandbox/FileForm.asp HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Content-Length: 54
2=This+is+some+text&fileentry2=&_charset_=windows-1252
Your Match Condition should be:
URLWithBody:/sandbox/FileForm.asp This+is+some+text

Excel file as HTTP post method response

In my application, I use GET method to get a Excel file back. The url actually contains the parameters that server needs to generate the Excel file. However, sometimes, the parameters may become so long (more then 2000 characters).
I am considering using POST method, but it does not seem that POST method can return a document. Am I right?
Nop, how you request, say GET, PUT and POST, doesn't necessarily affect the server you response.
How to make the response depends on the program on server. If you want to send a binary file after a POST request is totally cool.
Just take a look at sites like megaupload, rapidshare, etc. All of them will send you a file after you POST the recaptcha code.

Resources