My website uses a contextpath (eg: www.example.com/abc). The robots.txt is available at www.example.com/abc/robots.txt and I have given a 301 redirect in webserver to redirect www.example.com/robots.txt to www.example.com/abc/robots.txt.
My question is whether the search engines be able to read the robots.txt file since it has a 301 redirect?
Found that the search engines are honoring the 301 redirect and reading the file from the subfolder.
Robots.txt should be on root level
https://example.com/robots.txt - Correct
https://blog.example.com/robots.txt - Correct
https://example.com/abc/robots.txt - Not Correct
https://blog.example.com/abc/robots.txt - Not Correct
If it is on sub directory/sub folder then it will return 404 error(Because they make calls only on root directory), and Google will ignore your robots.txt completely if it is return 301 or 404 error.
Related
We are rebuilding a website for a client from ASP to WordPress. This website will have a different domain, url structure, and file extension. I am only just getting my head around htaccess 301 redirects, and I know enough that I can't do the following:
Redirect 301 http://www.site1.com/about_us.asp https://site2.com/about/
Redirect 301 http://www.site1.com/art-specs/ https://site2.com/specs/
Redirect 301 http://www.site1.com/page/product1/ https://site2.com/product1/
There are about 12 links in total that need to be redirected, and I want to make sure that it is done right the first time as a client's SEO rankings are on the line.
Is there a variation of the above format that I could use? Or a rewrite rule that needs to be done first? Any help (and explanations) would be greatly appreciated!
After looking more into it, I realised that the htaccess file shouldn't need anything other than relative access to the original domain.
i.e. You shouldn't need to declare: http://www.site1.com/about_us.asp since the server and domain should be configured in such a way that /about_us.asp means the same thing.
So the correct answer would be to:
[1] Configure the server (in my case cPanel) by having the original domain added as an addon domain (e.g http://www.site1.com/).
[2] In the htaccess file I would add each of the 301 redirects to the htaccess file:
Redirect 301 /about_us.asp https://site2.com/about/
Redirect 301 /art-specs/ https://site2.com/specs/
Redirect 301 /page/product1/ https://site2.com/product1/
...for each redirect
[3] And finally, adding the following to the bottom of the htaccess file will catch everything else and redirect them to the home page:
RedirectMatch 301 .* https://site2.com
I want to redirect 301 with htaccess file my broken URLs. What do I mean? If visitor come to this link: http://mydomain.com/blablablablablablablabla34234234.php he probably will get an 404 error so I want my "404 page" will be in sub folder. like this:
http://mydomain.com/somenonpage.php to http://mydomain.com/sub/somenonpage.php
http://mydomain.com/blablablablabla to http://mydomain.com/sub/blablablablabla
etc.
If you want to have a custom 404 page you do not do a 301 Redirect, 301 is to redirect an old link to a new URL. Say you had a popular post but decided to change the url for SEO reasons but the old url was heavily backlinked. You would then use a 301 to make sure that those backlinks got associated with your new url.
To make a custom 404 all you need to do is edit your http.conf file and add a path to your custom 404 page. By default I think the file belongs in your root directory named 404.html.
If you need the fileto have a different name or location you can edit the file path at the line beginning line ErrorDocument 404 /404.html.
I have my own domain:
http://cesarferreira.com
and I wanna make
http://cesarferreira.com/github
point to
https://github.com/cesarferreira
Without having to make a /github/ folder with an index.html with a redirect for each page I own (facebook, twitter, pinterest, etc)
Is there a way like for example htaccess catchig *.com/github and pointing to a given static url?
Thanks a lot
If your document root serves -
http://cesarferreira.com
you can put a redirect in .htaccess like -
Redirect /github https://github.com/cesarferreira
Take a look at URL rewriter 'http://httpd.apache.org/docs/2.0/misc/rewriteguide.html'. That should be able to do everything you want and more.
As long as it is enabled in apache then you can use it in .htaccess files also.
You can use mod_alias:
Redirect 301 /github https://github.com/cesarferreira
Or if you only want github to point only to the folder:
RedirectMatch 301 ^/github https://github.com/cesarferreira
You can put that in the htaccess file of your document root.
I use codeigniter and my robots.txt file located in the root, but it can be accessed only from main domain. Search robots trying to access it from subdomains(i use it for locales):
Example:
my.com/robots.txt - OK
en.my.com/robots.txt - FAIL
How can I redirect from xx.my.com/robots.txt to my.com/robots.txt ?
Thanks.
Try using a RedirectMatch
RedirectMatch 301 /robots.txt http://my.com/robots.txt
This is one of those super-simple questions that I can't seem to google an answer for, so apologies in advance.
When I ftp into my (shared) server, I have a file structure like this:
Root (/)
/public_html
/newdomain.com
I had an old website that lived in /public_html, it had heaps of content and excellent SEO. We changed our name and our domain (which lives in /newdomain.com, a folder inside /public_html), and set 301 redirects from all the old content to the new website.
I tried doing this myself, but it didn't work at all, so I got my host's techsupport to do it for me. There are several .htaccess files on my server though, and I don't know which ones are actually effective and which aren't.
Root has its own .htaccess file
public_html has its own .htaccess file
/newdomain.com DOESN'T have its own .htaccess file
Redirection 1 (currently is in both root and public_html's .htaccess files, and works)
I want to redirect http://olddomain.com/whatever -> http://newdomain.com/whatever (I've currently got each individual page doing its own separate 301 versus a single rule doing this). Achieved with Redirect 301 /article-name-here/ http://www.newsite.com/article-name-here/
Redirection 2 (currently is in both root and public_html's .htaccess files, and doesn't work).
I also want to do some internal redirections of http://newdomain.com/oldpage.html -> http://newdomain.com/newpage.html. I've tried redirection public_html's .htaccess file like so:
Redirect 301 http://newsite.com/badpage.html http://newsite.com/goodpage.html
But it's not working. Do I need to set up a new .htaccess in the newsite.com folder on my server? Or am I just completely missing the mark here?
Redirection 1
To redirect everything, just remove the article name:
Redirect 301 / http://www.newsite.com/
Or if you don't want to redirect the root (i.e. requests for /), then:
RedirectMatch 301 ^/(.+)$ http://www.newsite.com/$1
Redirection 2
If the /newdomain directory is the document root for http://newdomain.com/, then you'll need to create a new htaccess file there and include:
Redirect 301 /badpage.html /goodpage.html