In your opinion what is the best way to protect directory listing from external users?
Option 1: Blank index. This is the standar way that i have seen on several sites, it has te advantage of not showing anything but the disadvantage of implying that there is something there
Option 2: 404, send a fake 404 page and redirect, will this can cause problems with the webcrawlers?
Option 3: 401 error and redirection, this is similar to the blank index, except that it will show an "unauthorized" header, i think this will be a very bad option (because im implicity saying that there is something important inside), but i would like to hear your thoughts on this too
Thanks for your help if you know any other option that i might use please tell me as well
The 'best' way is to disable directory listing the server (this will normally cause a 403 error, see error 404 in the following list for discussion of information leakage)
The easiest way is a blank page (normally index.html or index.htm)
Other options with returning errorcodes:
403 (forbidden) is the default in apache httpd and i think this is better than a blank page.
404 is for 'not found' which is not the case here (could be used if nobody knows that the directory exists in order to prevent disclosure, but if ppl. know it exits it doesn't make any sense as its existance is already known) and
401 (authentication required) doesn't make any sense in any case
Other considerations
some browsers do not display custom error pages. If you want to provide a link to the main page (or somewhere else) a 'blank' page containing a link or a direct 301/302 redirect could be used.
Related
I am currently developing my website and have added some custom error pages, namely for HTTP 404 and 500. Obviously there are many more error codes than that, but these two are the most common.
What I am wondering is whether there is any rule of thumb for how many you create. Most websites have custom error pages (especially for these two errors), but some have it for others as well. Just how many error pages should I make?
There is no rule of thumb and there are so many different error codes that creating a custom page for each of them would be overkilling.
Check the different error codes here:
https://support.microsoft.com/en-us/kb/318380
If you want completeness, I recommend you to go to the Error Pages feature configuration in IIS, for example, and check the ones in there and it will be very easy for you to decide whether you want a custom message or not.
401
403
404
405
406
412
500
501
502
My suggestion, and again it is based on you wanting completeness of customization, would go with those. Otherwise, just stick with the defaults and the two you have already customized.
Hope it helps.
So... we have a custom CMS. We have a rewrite rule that any page request (when a file doesn't exist) goes to the root/index.cfm file. There we search our DB for the page in question. If the page exists, we serve up the correct template,etc. If the page doesn't exist I want to server up a 404 page. Now I "think" I cannot do this in IIS since I need to handle the request in CF, so it has to get through. The file will always exist. When the page doesn't exist I've tried using <cfheader statusCode="404" > and then include some html, it puts The resource you are looking for has been removed, had its name changed, or is temporarily unavailable. at the top of the page before my html. In order to get it to display the page I had to remove the 404 status code handler from IIS.
In addition when I fetch as Google, it get's a 301. However when I view response headers in Firefox I get.
Transfer-Encoding: chunked
Content-Type: text/html
Server: Microsoft-IIS/7.5
X-Powered-By: ASP.NET
Date: Wed, 16 Jan 2013 21:31:42 GMT
404 Not Found
I've tried a combination of redirecting and all sorts of things. I open to letting IIS handle the 404, if there is a way, but I cannot figure out how to get Coldfusion to correctly deliver a 404 so Google gets it right. Webmaster tools gets mad at me because I am delivering "Soft 404s" before this point, so I am trying to fix that.
I've also tried setting <httpErrors existingResponse="PassThrough" /> whatever the hell that does, but didn't work either. I've been looking up other threads trying to figure this out and just can't.
EDIT: Looking further into this, viewing the header info in both Firebug & Chrome I clearly see the headers say 404. Why would Fetch as Bing and Fetch as Google say differently?
I tested the fact that if I add .cfm to the URLs, it Fetch as Google will deliver see 404. However without the .cfm, it thinks it's 301. Firebug sees both as 404. This seems like a Google issue.
ANSWER Kind of:
So I was doing more testing this morning (Right after I added a bounty actually), and I noticed in webmaster tools, Google correctly noted one of my pages as a 404. So I started looking into it. I have an "Add Trailing Slash Rule". Google notices domain.com/page as a 301 (Correct I guess) to domain.com/page/. But it does notice domain.com/page/ as a 404. I think using the trailing slash rule as I have it is the right way, however, should I be doing something different, or is using the redirect with the slash the "correct" way of doing things, even though Google wants to ding me for it sometimes.
I'm not entirely sure I follow the specifics of your approach, so I will give you a few things that you need to look at in order to get this approach working well (or at least what has worked best for me).
Under "Error Pages", make sure that your 404 error page is set to "Execute a URL on this site" ( I generally set mine to something like "/404.cfm"). This will make sure that your ColdFusion page is called correctly for 404 pages (it sounds like you have this working correctly).
Under "Handler Mappings", double-click on the handler for ".cfm". Then click the "Request Restrictions..." button. It should open to the "mappings" tab. The "Invoke handler only if request is mapped to:" checkbox should NOT be checked.
This can really trip up this sort of operation because it means that IIS won't invoke ColdFusion if the file doesn't exist. This shouldn't be an issue if your 404 is set up correctly, but still something to look into.
While you are in the "Handler Mapping" section, look for the IsapiModule with a path of "*". Mine is always set to ColdFusion - not sure if that makes a difference or not.
The other thing to look at is the "Default Document" setting. Keep in mind that this could impact you when forwarding to a folder.
You might also look at your rewrite rule again and make sure it isn't adding slashes where one already exists.
I'm sending emails to customers, and I'm providing a custom URL for each, which when they go to, will log them in.
This is fine, except if they are using a shared browser that will remember the URL.
Is there any way at all to suggest to the browser that it shouldn't remember a URL?
Edit: This question has nothing to do with caching of the page.
Have the link log them in once. Then make them create credentials that let them access the site in the future. Whats to stop a random person from typing in the url and gaining access to the content?
Yes. You can redirect them with a 301 or 302. Then the browser won't save the URL they went to. At least that work with the Mozilla based browsers and I would imagine others too.
Another way, it is uglier though is to reply with an error and include a body which does a refresh. Whether that works in most browsers, probably not. However, browsers do not cache pages that return an error (404 Page Not Found would work, you could also use 403 Forbidden.)
Other than that, there isn't much you can do. JavaScript does not allow you to temper with the history anymore...
I would like to block my website and probably redirect them to a 404 page while i am updating it which can take some time.
Could a redirect to the 404 page everytime a user goes to my website work?
You shouldn't do that. Status code 503 "Temporarily Unavailable" is much better in this case.
RewriteRule . - [R=503,L]
This might work.
If it's just a temporary redirect during site-down maintenance then you probably don't want to use a 404 code. Take a look at the other codes available to you. For a scenario such as this, 307 (temporary redirect) would make a lot more sense. It would also be better if you have any SEO or rely on search crawlers at all, as they will remove results which now produce a 404 but are smart enough to keep results which temporarily produce a 307.
The redirect itself will work fine, just redirect all traffic to a static page. (Did you need advice on how to do that, or were you just looking for alternative options and viability? It's unclear from the question. If the former, I can't help much. It's been years since I've cracked open an .htaccess file.)
Basically, a 404 tells visitors: "This resource isn't here. Don't both asking again." Whereas a 307 tells visitors: "This resource is temporarily being handled by something else, but it hasn't really moved, please try again later."
Here's a simpler idea: just make a new index page that's your original, except with the content replaced with a "site currently being updated; please come back later" sort of message. And then you'd redirect all hits to your site to that index page.
That's what many sites I've seen tend to do, at least. And it makes sense, at least to me. I mean, would you rather your users not know why the pages they want to access are no longer there, or that they know the reason is because the site is being updated? It's basically the same as a 404 page, just with the specific information of why the desired pages aren't there.
EDIT: It seems I'm basically talking about a 503 page, going by David's link and Roland's answer.
That would work, but that would not only be wrong information (the page is not 'not found' - it's just currently being updated), but also mislead your users and crawlers. I would redirect them to a 'Update in progress' page and send this with the http status code 423 (LOCKED) to the client to provide a standard conform answer to exactly your scenario.
I have deleted a folder called forums from my website from 3 months. but in my Google Webmaster Tools it keeps saying that e.g. /forums/member.php?u=1092 is missing (404). is there any way to stop these messages and tell google that i am not going to re-upload it? is this going to affect on my SEO ranking?
I tried this code, but it's not working.
RewriteRule ^forums/(.*)$ http://www.mysite.com [301, L]
Thanks.
Have you tried changing the status code to 410?
410 Gone
The requested resource is no longer
available at the server and no
forwarding address is known. This
condition is expected to be considered
permanent. Clients with link editing
capabilities SHOULD delete references
to the Request-URI after user
approval. If the server does not know,
or has no facility to determine,
whether or not the condition is
permanent, the status code 404 (Not
Found) SHOULD be used instead. This
response is cacheable unless indicated
otherwise.
More detail available in the rfc.
Google on Removing my own content from Google.