We're trying to get Webdav running in Kubernetes using an Azure Files storage backend, which is mounted in the container on /dav/data. The container itself is running Alpine Linux 3.12.1, in which we're installing all our required Apache packages.
All is well and good when not mounting this storage, or when using a different storage backend. However, when mounting the Azure Files storage, stuff starts to break.
Uploading files works without issue, but downloading does not; most software complains about invalid HTTP headers/responses. When investigating this further, I see that the beginning of the headers seems to be getting cut off.
Example headers of a correct response (obtained by not mounting the volume):
HTTP/1.1 200 OK
Date: Tue, 01 Dec 2020 13:54:53 GMT
Server: Apache/2.4.46 (Unix)
Last-Modified: Tue, 01 Dec 2020 13:51:02 GMT
ETag: "bla"
Accept-Ranges: bytes
Content-Length: 985
Connection: close
Example headers of an incorrect response:
s: bytes
Content-Length: 985
Connection: close
Everything up to the first s in the Accept-Ranges header seems to be getting eaten somewhere. There also seem to be a number of extra null bytes at the end of the response.
In an effort to get to the bottom of this I looked into logging as much as I possibly could, and stumbled upon the DumpIO module, which would allow me to log both the response headers as well as the body. For some reason, loading this module, setting DumpIOOutput On and LogLevel dumpio:trace7 actually fixes the issue. Response headers are fine, and the response body is exactly what you'd expect. And it's driving me nuts.
I suspect there's some kind of weird buffer/window issue being caused by an interaction between Apache and the mounted volume, but I haven't been able to figure out what.
We've since changed the storage backend used for the volume, but I'd still really like to know what caused this issue.
I've also been able to reproduce this locally in Docker.
Having the exact same issue after upgrading the AKS cluster from 1.17 to 1.18. Headers are malformed. Tried updating to a newer Apache version but doesn't work. Temporarily switched from azureFile to azureDisk and that works! Will see if I can create an AKS bug report for this
In case you haven't come across the culprit yet (or anyone else coming across this while googling like I did). The issue is Apache's EnableMMAP setting.
Solution: In your Apache conf, set EnableMMAP off.
Sources:
https://cloudiseasy.com/2021/06/13/deploying-apache-server-on-aks-with-azure-files/
https://httpd.apache.org/docs/2.4/mod/core.html#enablemmap
Apache is adding header to images resulting in corrupting images
Related
I configured a Frontdoor on Azure, with 2 rules
(HTTP endpoint) Rewrite HTTP to HTTPS
(HTTPS endpoint) HTTPS to backend pool
When requesting the http endpoint Frontdoor answer this:
HTTP/1.1 302 Found
Location: https://example.com/
Server: Microsoft-IIS/10.0 <------
X-Azure-Ref: xxxxxxxx
Date: Wed, 08 Jul 2020 12:00:00 GMT
Content-Length: 0
Is it possible to remove this header ? I know it's a managed resource but I can't find any documentation on this matter/if it's normal.
I don't believe it's my backend answering because my https endpoint doesn't answer me that ... but maybe?
You can set Rules engine configuration in Front Door.
The server information may not be displayed at present due to my environment. But you can still refer to my screenshots for configuration.
I found that the information such as Date, although I modified it, still does not take effect. It may be related to the information returned by the azure server and cannot be modified. You can try to modify the Server information.
After the modification, if the Server information remains unchanged, there is no way to modify it.
Related similar posts you can refer to:
ASP.NET MVC 5 Azure App ZAP Scan indicates Proxy Disclosure vulnerability - how can we prevent that?
In the webapp, no matter the program is modified or in other ways, the server information cannot be modified.
So if the above method can be modified successfully, it can be of great help to you.
If it doesn't work, you don't have to spend time to deal with this problem. You can raise a ticket in portal to confirm.
Maybe it sounds a novice question in Varnish Cache world, but why in WordPress it seems that is a need to install a external cache plugin, to working fully cached?
Websites are correctly loaded via Varnish, a curl -I command:
HTTP/1.1 200 OK
Server: nginx/1.11.12
Date: Thu, 11 Oct 2018 09:39:07 GMT
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
Vary: Accept-Encoding
Cache-Control: max-age=0, public
Expires: Thu, 11 Oct 2018 09:39:07 GMT
Vary: Accept-Encoding
X-Varnish: 19575855
Age: 0
Via: 1.1 varnish-v4
X-Cache: MISS
Accept-Ranges: bytes
Pragma: public
Cache-Control: public
Vary: Accept-Encoding
With this configuration, by default WordPress installations are not being cached.
After test multiple cache plugins -some not working, or not working without complex configuration- i found the Swift Performance, in their Lite version, simply activating the Cache option, here really takes all advantages and here i can see varnish is working fully with very good results in stress test.
This could be ok for a single site on a single environment, but in shared hosting terms, when every customer can have their own WP (or other CMS) installation could be a problem.
So the key is there are no way to take full caching advantage from Varnish without installing 3rd party caching (and complex) plugins? Why not caching all by default?
Any kind of suggestions and help will be high welcome, thanks in advance.
With this configuration, by default WordPress installations are not being cached
By default, if you don't change anything in neither Wordpress or Varnish configuration, things would work together in a way that Wordpress pages are cached for 120 seconds. So real caching is possible, but it will be a short lived cache and highly ineffective one.
Your specific headers indicate that no caching should happen. They are either sent by Varnish itself (we're all guilty of copy pasting stuff without thinking what it does), or a Wordpress plugin (more often bad ones, than good). Without knowing your specific configuration, it's hard to decipher anything.
Varnish is a transparent HTTP caching proxy. Which means it’s just going to, by default, use HTTP headers, which are sent by backend (Wordpress), like Cache-Control, to make a decision on whether resource can be cached and for how long.
Wordpress, in fact, does not send cache related headers other than in a few specific areas (error pages, login POST submission, etc).
The standard approach outlined here is configuring Varnish with the highest TTL. With that:
Varnish has no idea when you update an article contents, or change theme. Typical solution to this lies in using cache invalidation plugin like Varnish HTTP Purge.
A plugin requirement comes from necessity to purge cache, when content is changed.
Suppose that you update a Wordpress page's text. You had that same page previously visited and it went into Varnish cache for storage. What happens upon the next visit, is that Varnish will serve the same, now stale content to all the next visitors.
The Wordpress plugins for Varnish, like Varnish HTTP Purge, will hook into Wordpress in a way that they will instruct Varnish to clear cache when pages are updated. This is their primary purpose.
That kind of approach (high TTL and cache purging) is de-facto standard with Varnish. As Varnish has no information about when you update content, the inner workings of purging cache is with the application itself. The cache purging feature is either bundled into CMS code itself (Magento 2, for example has it out of the box, without any extra plugins), or a Wordpress plugin.
I've got 6 identical machines running IIS and Apache. Today one of them decided to just stop serving requests. I can access all of the webapps when I try from localhost/resource but when I try from url/resource I get a 404. I did a Get request against the machine that isn't working and I get this back:
Server: Microsoft-HTTPAPI/2.0
Connection: close
Compared to a working server:
Server: Microsoft-IIS/8.5
X-Powered-By: ASP.NET
Content-Type: text/html
Tried searching for this problem but came up with nothing, anyone got any idea's?
Windows has an HTTP service that manages calls to IIS and other HTTP enabled services on a windows machine. Either you need to configure it to handle your calls, or, in the case of WAMP or similar non-IIS-web-server-on-windows scenarios you may just need to turn it off.
When you see "Microsoft-HttpApi/2.0" returning error, such as 400 "bad URL" or "bad header", etc. the problem is most likely because the HTTP.sys service is intercepting your http request and terminating it because it does not meet with the minimum validation rules that are configured.
This configuration is found in the registry at HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\HTTP\Parameters. In my case, it was choking because I had a RESTful call that had a 400 character segment in the url which was 160 characters more than the default value of 260, so I
added the registry parameter UrlSegmentMaxLength with a DWORD value of 512,
stopped the service using net stop http
started the service using net start http
I've run into these issues before and it is easy to troubleshoot but there is very little on the web that addresses it.
Try these links
"the underlying problem is that the client has sent a request to IIS that breaks one or more rules that HTTP.sys is enforcing"
enabling logging on HTTP.sys is described here
a list of the HTTP.sys parameters that you can control in the registry is found here.
A bit late, so put here for posterity ;-)
After trying all sorts of solutions found on the web, I almost gave up, but found this little nugget.
If the response's Server header returns Microsoft-HttpApi/2.0, it means that the HTTP.sys is being called, not IIS.
As a result, a lot of the workarounds will not work (URLScan, etc).
This worked however:
Open regedit
Navigate HKLM\SYSTEM\CurrentControlSet\Services\HTTP\Parameters\
If DisableServerHeader doesn't exist, create it (DWORD 32bit) and give it a value of 2. If it does exist, and the value isn't 2, set it to 2.
Finally, restart the service by calling net stop http then net start http
src: WS/WCF: Remove Server Header
Set below registry flag to: 2
HKLM\SYSTEM\CurrentControlSet\Services\HTTP\Parameters\DisableServerHeader
Setting this to 2 will ensure that self host WCF services no longer sends the SERVER header and thus ensure we are security compliant.
Please note that this disables ALL server headers.
The default value of 0 enables the header, and the value of 1 disables server header from DRIVER (http.sys), but app can still have headers.
For me I had to restart the server for the changes to take effect.
Hope this helps someone
I was working on our web app on a client's site and ran into an issue where the site root pages loaded, but the reports folder always returned a 404 for files that existed in the folder. The 404 page showed the .Net version of 2 when the application was set to 4, and a test of a non-existent page in the root returned a 404 page showing .Net 4.
I tried just http://localhost/reports and got back a Microsoft Reporting Services page. Not part of my application.
Be sure to check just the default document of the folder when a unexpected 404 comes up and the file exists.
This question and series of replies helped me get to the bottom of the related issue I was having. My issue centered around using just a subdomain to go to our server (e.g. typing "www/somepath" into the browser while on our corporate network), which had worked in the past on an older server, but no longer worked when the system was upgraded to a new server. I saw the unexpected Microsoft-HttpApi/2.0 string in the header when using the Chrome Devtools to inspect the Network traffic.
My HTTP.sys process was already logging, so I could verify that my traffic was going to that service and returning 404 NotFound status codes.
My resolution was to add a binding to the IIS site for the subdomain, making IIS respond instead of the HTTP.sys process, as described in this server fault article - https://serverfault.com/questions/479274/why-is-microsoft-httpapi-returning-404-to-my-network-switch
In my case, running Windows 10 Pro, it was the Windows MultiPoint Service.
By executing:
net stop wms
Port 80 was released.
I'm having trouble with a particular version of Pocket IE running under Windows Mobile 5.0. Unfortunately, I'm not sure of the exact version numbers.
We had a problem whereby this particular 'installation' would return a locally cached version of a page when the wireless network was switched off. Fair enough, no problem. We cleared the cache of the handheld and started sending the following headers:
Expires: Mon, 26 Jul 1997 05:00:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Last-Modified: Thu, 30 Jul 2009 16:42:08 GMT
The Last-Modified header is calculated on the fly and set to 'now'.
Even still, the handheld seems to be caching these pages: the page is sent with the headers but then when they disconnect the wireless network and click a link to the page (that was not supposed to be cached) it still returns this cached file.
Is there some other header/s that should be sent, or is this just a problem with Pocket IE? Or is it possibly something entirely different?
Thanks!
I'm not sure I can answer your question since I have no Pocket IE to test with, but maybe I can offer something that can help.
This is a very good caching reference: http://www.mnot.net/cache_docs/
Also, I'm not sure whether your example is the pasted results of your headers, or the code that you've set up to send the headers, but I believe the collection of headers in most language implementations (and by extension I assume most browser implementations) is treated as a map; therefore, it's possible you've overwritten "no-store, no-cache, must-revalidate" with the second "Cache-Control" header. In other words, only one can get sent, and if last wins, you only sent "post-check=0, pre-check=0".
You could also try adding the max-age=0 header.
In my experience both Firefox and IE have seemed more sensitive to pages served by HTTPS as well. You could try that if you have it as an option.
If you still have no luck, and Pocket IE is behaving clearly differently from Windows IE, then my guess is that the handheld has special rules for caching based on the assumption that it will often be away from internet connectivity.
Edit:
After you mentioned CNN.com, and I realized that you do not have the "private" header in Cache-Control. I think this is what is making CNN.com cache the page but not yours. I believe "private" is the most strict setting available in the "Cache-Control header. Try adding that.
For example, here are CNN's headers. (I don't think listing "private" twice has any effect)
Date: Fri, 31 Jul 2009 16:05:42 GMT
Server: Apache
Accept-Ranges: bytes
Cache-Control: max-age=60, private, private
Expires: Fri, 31 Jul 2009 16:06:41 GMT
Content-Type: text/html
Vary: User-Agent,Accept-Encoding
Content-Encoding: gzip
Content-Length: 21221
200 OK
If you don't have the Firefox Web Developer Toolbar, it's a great tool to check Response Headers of any site - in the "Information" dropdown, "View Reponse Headers" is at the bottom.
Although Renesis has been awesome in trying to help me here, I've had to give up.
By 'give up' I mean I've cheated. Instead of trying to resolve this issue on the client side, I went the server side route.
What I ended up doing was writing a function in PHP that will take a URL and essentially make it unique. It does this by adding a random GET parameter based on a call to uniqid(). I then do a couple of other little things to it: make sure I add a '?' or a '&' to the URL based on the existence of other GET parameters and make sure that any '#' anchor items are pushed right to the end and then I return that URL to the browser.
This essentially resolves the issue as each link the browser ever sees is unique: it's never seen that particular URL before and so can't retrieve it from the cache.
Hackish? Yes. Working? So far, so good.
We have a problem with an IIS5 server.
When certain users/browsers click to download .zip files, binary gibberish text sometimes renders in the browser window. The desired behavior is for the file to either download or open with the associated zip application.
Initially, we suspected that the wrong content-type header was set on the file. The IIS tech confirmed that .zip files were being served by IIS with the mime-type "application/x-zip-compressed".
However, an inspection of the HTTP packets using Wireshark reveals that requests for zip files return two Content-Type headers.
Content-Type: text/html;
charset=UTF-8
Content-Type:
application/x-zip-compressed
Any idea why IIS is sending two content-type headers? This doesn't happen for regular HTML or images files. It does happen with ZIP and PDF.
Is there a particular place we can ask the IIS tech to look? Or is there a configuration file we can examine?
I believe - and i may be wrong that the http 1.1 header sends multiple headers definitions and the most specific has precedence .
so in your example here it is sending 2 text/html and then application/x-zip-commercial so the second one would be the most specific - if that cant be handled on the client then the more general one is used (the first one in this case ) -
I have read through this http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html and that sort of points to what you are saying - not sure if this is what is actually happening though.
Of course i may be totally wrong here
Make sure that you don't have any ISAPI filters or ASP.net HTTP modules set up to rewrite the headers. If they don't check to see if the header already exists, it will be appended rather than replaced. We had issues a while ago with an in-house authentication module not correctly updating the headers so we were getting two Authorization headers, one from IIS and one from our module.
What software has been installed on the server to work with .zip files?
It looks like IIS picks up MIME translations from the registry, perhaps zip-software you use has registered the MIME-type. This doesn't explain why IIS would respond with two content-type headers, so any ISAPI filter and other Mime-table is suspect.
This may be related to this knowledge base article. It is suggesting that IIS may be gzipping the already zipped file, but some browsers just buck pass straight to a secondary application giving you bad data (as it has been zipped twice). If you change the mime type of the zip extension to application/octet-stream this may not happen.
It sounds like there may be a issue with your configuration of IIS. However that is not possible to tell from your post if this is the case.
You can have mime types configured on several levels on your IIS. My IIS 5 knowledge is a bit rusty, as far as I can remeber this behavior is the same for IIS 6. I tried to simulate this on a IIS 6 enviroment, but only ever received one mime type depending on the accepted header
I have set the the header for zip files on the site to application/x-zip-compressed and for the file I have explicity set it to
tinyget -srv:dev.24.com -uri:/helloworld.zip -tbLoadSecurity
WWWConnect::Connect("server.domain.com","80")
IP = "127.0.0.1:80"
source port: 1581
REQUEST: **************
GET /helloworld.zip HTTP/1.1
Host: server.domain.com
Accept: */*
RESPONSE: **************
HTTP/1.1 200 OK
Content-Length: 155
Content-Type: text/html
Last-Modified: Wed, 29 Apr 2009 08:43:10 GMT
Accept-Ranges: bytes
ETag: "747da786a6c8c91:0"
Server: Microsoft-IIS/6.0
Date: Wed, 29 Apr 2009 10:47:10 GMT
PK??
? ? ? helloworld.txthello worldPK??¶
? ? ? ? helloworld.txtPK?? ? ? < 7 ? hello world sample
WWWConnect::Close("server.domain.com","80")
closed source port: 1581
However I dont feel this prove much. It does however raise a few questions:
What is all the mime maps that have been setup on the server (ask the server admin for the metabase.xml file, and then you can make sure he has not missed some setting)
Is those clients on a network that is under your control? Probably not, I wonder what proxy server might be sitting inbetween your server and the clients
How does the IIS log's look like, for that request, I am spesifically intrested in the Accept header.
I wonder what fiddler will show?
I've encountered a similar problem. I was testing downloads on IIS 6 and couldn't figure out why a zipped file called test.zip was displaying as text in IE8 (it was fine in other browsers, where it would download).
Then I realised that for the test I'd compressed a very small text file. My guess is that IE sniffed the file, saw the text (which was pretty much uncompressed because of the small size) and decided it was plain text.
I tried again with a larger file and the download prompt appeared OK in IE8.
May not be relevant to your case, but thought I'd mention it.
Tim