I am developing an ASP.NET Web API application which responds to the clients with a custom datetime http header. While I have read several articles describing how to remove response headers from ASP.NET/IIS, this one always seem to be resilient, i can't get rid of it. It seems to be placed in the response pipeline somewhere out of the programmer's/administrator's control, at the very end.
I know it may be a bad practice not to include the "Date" header in the response but, as i mentioned, the custom datetime header (which is in ticks instead of a string representation) makes the default one redundant; furthermore, this is a private API, so i know exactly who and how uses it.
Is it thus possible in any way to remove this header in IIS (v7+) for a specific site (or directly from the Web API application)?
Edit:
I tried (without success) the following techniques:
Creating a custom handler to remove the header right from the Web API project
Registering a custom IHttpModule
Explicit removal of headers in web.config in <httpProtocol><customHeaders> section
Remove HTTP response headers in IIS Manager
Header removal code in protected void Application_PreSendRequestHeaders(object sender, EventArgs e) method in Global.asax.cs
According to HTTP Spec, Date header is mandatory, except for these conditions which I dont think apply to your case:
Origin servers MUST include a Date header field in all responses, except in these cases:
1. If the response status code is 100 (Continue) or 101 (Switching
Protocols), the response MAY include a Date header field, at
the server's option.
2. If the response status code conveys a server error, e.g. 500
(Internal Server Error) or 503 (Service Unavailable), and it is
inconvenient or impossible to generate a valid Date.
3. If the server does not have a clock that can provide a
reasonable approximation of the current time, its responses
MUST NOT include a Date header field. In this case, the rules
in section 14.18.1 MUST be followed.
This is not going to be possible from within the actual WebApi/Mvc pipeline, so options like action filters and delegating handlers are out.
Instead you will probably need to implement a custom IHttpModule and register it inside IIS. There is an article here you should read and follow. The approach is very simple and easy to adapt.
Just replace the set in that example with:
HttpContext.Current.Response.Headers.Remove("Date");
Related
We are developing a chatbot to handle internal and external processes for a local authority. We are trying to display contact information for a particular service from our api endpoint. The HTTP request is successful and delivers, in part, exactly what we want but there's still some unnecessary noise we can't exclude.
We specifically just want the text out of the response ("Response").
Logically, it was thought all we need to do is drill down into ${dialog.api_response.content.Response} but that fails the HTTP request and ${x.content} returns successful but includes Tags, response and the fields within 1.
Is there something simple we've missed using composer to access what we're after or do we need to change the way our endpoint is responding 2? Unfortunately the MS documentation for FrwrkComp is lacking to say the very least.
n.b. The response is currently set up as a (syntactically) SSML response, this is just a test case using an existing resource.
Response in the Emulator
Snippet from FwrkComp
Turns out it was the first thing I tried just syntactically correct. For the case of the code given it was as simple as:
${dialog.api_response.content[0].Response}
We would like to use PUT, PATCH and DELETE methods when building our internal API in Domino, but I can't configure those methods. Server refusing to pass them.
When I used method PUT Domino replied to me:
Error code: 405 | Request method is not allowed by the server
After that I enabled methods via notes.ini
HTTPEnableMethods=GET,POST,PUT,DELETE,HEAD,PATCH
That seems to help a bit but now it says something about URL, but I do not really get what it means.
Error code: 405 | The HTTP method is not allowed for the specified URL
I have made tests on two different setups: with and without internet sites documents enabled in server documents.
Does anybody what I need to do to solve the problem above?
UPDATE
I just noticed a help text on Internet Site for "Methods" field.
GET, HEAD, and POST are the most commonly used methods. OPTIONS and TRACE are useful for debugging. PUT and DELETE should only be enabled if the Web site includes special CGI programs or Java applications that implement them.
Based on that it seems it is not possible to make PUT and DELETE work out of the box.
I have a scenario where my URL will be either contains a comma delimiter with value or without.
i.e. /api/parameters/XXXXXXXXXX?tables=x0 or tables=x0;x1;x2.
now based on this URL I want to check in the varnish that, if URL contains multiple values as tables then separate that out and pass each table name in seperate URL (/api/parameters/XXXXXXXXXX?tables=x0, /api/parameters/XXXXXXXXXX?tables=x1, /api/parameters/XXXXXXXXXX?tables=x2) either to cache if miss then backend server.
then based on the response of this need to combine the result and return it to the client.
my question here is:
How to segregate the value from the URL and pass a modified URL to varnish cache or backend.
after returning the result I want to return it as a combined JSON object in a sequence of which it was originally requested with a comma delimiter(i.e. x0 result;x1 result;x2 result).
It is possible to turn a single request into multiple subrequests in Varnish. Unfortunately this cannot be done with the open source version, only with the Enterprise version.
vmod_http
https://docs.varnish-software.com/varnish-cache-plus/vmods/http/ describes how you can perform HTTP calls from within Varnish using vmod_http.
By sending HTTP requests to other URLs through Varnish, you can get multiple objects out of the cache and aggregate them into a single response
No looping
The fact that Varnish doesn't have loops makes matters a bit more complicated. You'll have so set an upper limit to the amount of values the tables querystring parameter has and you'll have to check the values using individual if-statements.
Returning the combined JSON output
Once you have fetched the results from the various URLs, you can create a JSON string and return it via return(synth(200,req.http.json)). Where req.http.json contains the JSON string.
This will create a synthetic response.
In Varnish Enterprise it is also possible to cache synthetic output. See https://docs.varnish-software.com/varnish-cache-plus/vmods/synthbackend/ to learn more about vmod_synthbackend.
Varnish Enterprise disclaimer
The solution I suggested in my answer uses Varnish Enterprise, the commercial version of Varnish. It extends Varnish capabilities with additional VMODs and features, which you can read about here. One easy way to try it out without upfront licensing payments, if you’re interested, is to spin up an instance on cloud infrastructure:
Varnish Enterprise on AWS
Varnish Enterprise on Azure
Varnish Enterprise on GCP
Is it possible to validate if an HTTP request originated from the client as GET, but was intercepted in between and converted to POST, or vice versa?
It is one of the security validations that is required as part of the project I am working on, but not getting enough clue about it. One of the way we thought of using as validation is to check if it is a GET request with a body than it could be POST. But that is just one case. Also if a POST is changed to GET by forging the request, I believe the data in the body can also be removed.
edit: Added more information about application and the intercepter
It is a regular Java web application developed using Struts with JSPs on the client side. The request from the web pages are being intercepted using Burp Suit Proxy to change the payload in the request.
I know that the default Varnish vcl_fetch looks at beresp.ttl and beresp.http.* to reference the HTTP headers returned from the backend, but is it possible to examine the content of the response also? Our backend sometimes fails with junk HTML but with a status of 200 OK. We'd like to be able to run a regex on the result and retry if possible.
I understand that versions of Varnish <= 3.0 don't stream anyway and download the entire object before passing to the client, but I can't find the appropriate field in beresp in the documentation - I'm looking for something like beresp.http.content
Yes and no. It's accessible, but only through inline C, not VCL configuration (to the best of my knowledge). However, it's not easy to do and not really recommended due to the additional overhead of parsing body text. That said, you can see an attempt at something like what you're looking for here: rewrite vmod for varnish 3
If your junk HTML responses are of a specific length, you can retry the request based on the response's Content-Length header. Alternatively, you might consider adding client-side JS to evaluate the HTML and make an AJAX request to a URL to clear the cache of any junk pages. Lastly, if you know that only a specific subset of your site that returns invalid results, you can try proxying those URLs through something like OpenResty with LuaJIT or nginx with the subs module enabled, and do the body parsing there.