I have changed the structure of the URLs of my site more than 6 months ago. I detect the use of legacy URLs and redirect to the new URL with a 301 status code. I verified with flidder that the status code is correctly returned upon the request. But bots (yahoo slurps, googlebot, etc.) are still hitting the old URLs. Is there something I am missing?
No, just it takes a very, very long time for crawlers to get the message. I have bots crawling addresses that have not existed since 2005 - when folk harp on with addresses being permanent, they really are.
Additionally, depending on how your URL's are structured, you can disallow the old addresses with robots.txt
Try this and this will only redirect to the bots.
if (preg_match("#(google|slurp#inktomi|yahoo! slurp|msnbot)#si", $_SERVER['HTTP_USER_AGENT'])) {
header("HTTP/1.1 301 Moved Permanently");
header("Location: http://www.your-main-site.com/");
exit;
}
If external sites have linked to your old pages and those links are still accessible for bots, the bots will keep coming and try to access the content.
mentioned you site address here:
http://www.your-main-site.com/
Thats we use to transfer the domain and sometime for blackhat seo.
Related
I recently moved a website to another domain, but I am keeping the original domain for a future website. The recently moved website is permanently moved, but I only want traffic to be redirected temporarily to the other domain.
I have read countless 301 vs 302 posts, articles, etc. and have failed to find any that address this particular situation. And no one answered this similar question:
Moving to a new domain and using the old one for a new website - how to handle 301 redirects?
But the difference with my question is: I don't even know that I should be using a 301 redirect for this. What's the best way to handle this situation in general?
301 Moved Permanently doesn't mean that the domain name should never be used again; it means that the content the user requested has moved permanently, which is exactly the situation you describe.
(308 Permanent Redirect is supposed to supplement 301, but it is sometimes treated oddly and doesn't work at all on Windows 7 or 8.1 under IE.)
When you decide to re-use the old domain for a new set of resources, you can simply stop redirecting away from it. By then you hope that people looking for the old resource have updated their bookmarks/links thanks to the 301. It might be polite to allow some duration of time for this transition to occur.
Neither 302 Found nor its replacement 307 Temporary Redirect is appropriate here, as both misrepresent the situation and do not signify that people should permanently look for the old content in the new location: they're going to be mighty confused when the original domain no longer redirects, allegedly the end of a temporary situation, but then also does not serve the expected content.
You can use the .htaccess file to redirect with 307 status code (the successor of 302 in HTTP/1.1)
Redirect 307 / http://other-domain/
Here's a link that you may find helpful. You can also redirect with 308 status code also as #István Rábel suggested
I have a question about redirection 301 or 302.
For ex. i have domain www.example.com.
It's a good site with a good domain history. Some 'BAD' webmasters make sometimes a redirect from bad-histories domain with search bans to redirect it to good domains to downgrade good sites in search system query as Google.
So question is: is it possible to know, have someone made this bad-redirect?
As i understand in each html response html headers are transmitted. May by there is a way to catch this responses to detect such redirects?
Is it well-known practice?
Or it is impossible and it's like paranoia?
I think is paranoia, but major browsers still keep the referrer in the redirection and can be read at destination. In php is:
$referral = $_SERVER['HTTP_REFERER'];
you can trap all referrer urls in a log file and I have no idea how to filter out good sites. this way you will discover only redirects only from websites heaving some traffic;
I have a relatively new site that has just started to pick up a bit of traction in the SERP's. My problem is that I have published it and had it indexed with PHP URL extensions, as follows:
www.example.com/page.php
www.example.com/product.php
And so on. Obviously it is a fairly easy matter of editing the .htacess file to remove these extensions. So I will end up with:
www.example.com/page
www.example.com/product
No problems there.
Because the site is still quite small, I can easily change all the links manually to drop the .php extension, and then update the sitemap. So Google, and all users, should have no way of reaching the .php pages, although of course they still exist if you were to manually type them in.
But, because Google has a 'record' of these pages existing (even though there are no direct links to reach them now), do I need to implement 301 redirects from the .php pages to the new non-php pages? I.e. will Google try to crawl those pages that are no longer in the sitemap, but once existed? In other words, since you can still reach www.example.com/page.php , even though will be no link on the site or in the sitemaps that will take you there, would I get penalised for having duplicate content - are 301 redirects basically required when doing this kind of thing, even if there are no links to the content anymore?
Thanks very much.
It is better to have 301 redirect for some time(month or two) even though you can change all your links to nonphp urls. This way any residual URLs(will always be there) that are hanging out there will be taken care and google will index nonphp urls from your 301 redirect. Once you are sure from Logs(depending on your system) that there are no more OLD urls coming in, you can remove the 301 redirects. This is little easier way of moving all your old URLs instead of abruptly throwing 404s. 301 helps to transfer SEO values of old URLs to new ones.
Another item to look out for is using rel="canonical" if you want your .php and nonphp pages to coexist. This signals that they are not duplicates.
I apologize if in the wrong place. I have someone that has done 301 redirects where dynamic pages are being redirected to show .html. (www.printe-z.com/computer-checks to www.printe-z.com/computer-checks.html) They feels they should now use a custom 404 page for the original page(s); www.printe-z.com/computer-checks. What do you think? Leave it the way it is?
301 is better for this scenario, specially for search engines: they will associate your old urls with the new ones so you'll keep your pages rank.
People accessing your old links will be benefited too since they will be automatically redirected to new urls.
If the page has moved use 301. If it is removed all together, 404.
301 tells crawlers such as googlebot that the original page is moved permanently to the target page. So keep it as it is.
For your information, most URL rewritings are done using 301 Permanent Redirects. Also sending domain.com visitors to www.domain.com is an example where 301 redirection is used.
People might have bookmarked the old urls, or there may be a link on a website to the old url. Unless you are sure this is no longer the case, which you never can, you can remove the 301. But this is in a perfect world.
If the 301's are a maintenance burden, or have some other negative side-effect and the 301 have been there for some time ( >1 year), you could just remove them. But if not just leave them be.
I have a big Joomla website that was moved from .com to .eu domain.
The sites are on one web server.
On the old website there are a lot of links that point directly to oldwebsite.com/somethinghere.
Is there a way with Htaccess file in the old domains folder to redirect
from oldwebsite.com/somethinhere to newwebsite.eu/somethinghere?
Or maybe a way to do this on the new website folder?
Thank you,
Chris.
There are many ways, some people recommend using RewriteEngine, but there is an easier way:
Redirect 301 / http://domain.eu/
This will automatically handle files in the domain -- for example, old.com/foo will be redirected to new.eu/foo
EDIT: Regarding "301", that is the code for a permanent redirect. There also exist temporary redirects (I believe the code is 302), but it sounds like what you need is exactly this - a permanent redirect.
EDIT2: Oh and, I think the / after domain.eu may be required, but I'm not 100% sure - doesn't hurt for it to be there, that's for sure! I know that because I use the same kind of a redirect on a production website.