I have a VPS Hosting account with 2 domains. I just looked on Google to view the pages indexed for one of the sites and noticed that one of the results was one of my nameservers. As far as I am concerned, Google should not be indexing this.
Additionally, when you click on the link and go to it, the page loads as the 2nd website's home page. I fear that this is a problem for 2 reasons... 1- Duplicate Content, 2- Associating content between 2 sites.
Name Servers:
NS1.DOMAINA.COM
NS2.DOMAINA.COM
Domains:
DOMAINA.com
DOMAINB.com
When I clicked on NS1.DOMAINA.COM and NS2.DOMAINA.COM, they both pulled content from DOMAINB.COM. The nameservers are being indexed as DOMAINA.COM pages however.
After reading several resources, it seems that this issue resulted in band-aid fixes such as robots.txt rules, etc. I would rather fix the problem but if that is not probable, I would prefer to do this in .htaccess. However, I am nor exactly proficient in htaccess so I would require examples or detailed explanations.
Thank you
Add a file named robots.txt at the root of the domain you wish to remove the indexation, and fill it with this:
User-agent: *
Disallow: /
You can also set this bunch of lines on your .htaccess :
Header set X-Robots-Tag "noindex, nofollow"
EDIT: Also, to block only the subdomain, add a rewrite rule:
RewriteEngine on
RewriteCond %{HTTP_HOST} ^DOMAINA.COM$
RewriteRule ^robotx\.txt$ robots-subdomain.txt
Related
I am trying to clean up a previously hacked WordPress site, and domain name reputation, the site has new hosting and is now on a different CMS system, but there are hundreds of spam links in Google I need to get rid of, they look like example.com/votes.php?10054nzwzm75042pw205039
Domain name, then votes.php?**** etc.. Numbers letters all sorts.
So how do I redirect ANYTHING that starts with the domain name then /votes.php?***
Any help greatly appreciated
Unless you have multiple domains, you don't need to explicitly check the domain name.
To send a "410 Gone" for anything that contains /votes.php in the URL-path (and any query string), you can do something like the following at the top of your root .htaccess file using mod_rewrite:
RewriteEngine On
# Serve a 410 Gone for any requests to "/votes.php"
RewriteRule ^votes\.php$ - [G]
A 410 is preferable to a "redirect" if you want to get these URLs removed from the search engines as quickly as possible.
To expedite the process of URL removal from Google then use Google's Removal Tool as well.
If you redirect these pages to the homepage then it will likely be seen as a soft-404 by Google and these URLs are likely to remain in the search results for a lot longer.
I just rebuilt my site to be responsive with Gantry 5 and Joomla. built in inmotion hosting server. moved it into a sub domain on my real sites server, www.umiultrasound.com/new. when i try to access the site to test it redirects to www.umiultrasound.com.
I have discussed this with my host, they say i have a wildcard redirect- i don't. they say to check my .htaccess file. i did but i do not see where i am redirecting / or www.mysite.com to itself.
they then recommended that i read one of your forum results. the one they sent didn't apply to me. hoping you can help
Check for any RewriteBase command in .htaccess. You could also temporarily copy the standard htaccess.txt over .htaccess to see if this makes any difference (backup .htaccess to .htaccess.old or similar first).
Also check your configuration.php file to see if the live_site parameter has been set.
I'm looking for some help with my .htaccess file.
In short, I have a bunch of content up in the subdirectory www.myserver.com/forum/ (this directory is not my choice and cannot be changed), which includes both a forum (myserver.com/forum/forum) and a wiki (myserver.com/forum/wiki) and this is abviously not an ideal layout. I do not own myserver.com and there is content there that is not mine, though I have full access to the entire server.
I also have a domain name www.mydomain.com, which I want to redirect and mask such that if a user goes to mydomain.com/<something> they will be shown the content from myserver.com/forum/<something> while still being shown mydomain.com/<something> in the address bar.
One other thing I'd like to see, although it isn't vital, would by for a 404 generated by someone typing in mydomain.com/somethingThatDoesNotExist should redirect to mydomain.com/404.php instead of myserver.com's default 404.
I've tried a number of different approaches and searched extensively for the past day or so online - I'm sure that the answer is even here on SO somewhere, but all of the guides/examples I've tried have not worked and I feel like I'm going in circles.
Many thanks in advance.
EDIT: And I do know for sure that .htaceess is enabled.
if a user goes to mydomain.com/ they will be shown the
content from myserver.com/forum/ while still being shown
mydomain.com/ in the address bar.
First thing you need to understand that this is ONLY possible if mod_proxy is enabled on the Apache server of mydomain.com.
Once you've enabled mod_proxy, mod_rewrite and .htaccess through httpd.conf on mydomain.com, put this code in its .htaccess under DOCUMENT_ROOT directory:
Options +FollowSymLinks -MultiViews
# Turn mod_rewrite on
RewriteEngine On
RewriteRule ^(.*)$ http://myserver.com/$1 [L,P]
I will have to think little more about your 404 requirement, but if you get this working I am sure we can find a workaround for that as well.
You could do some proxying with PHP or whatever other language they allow..
Take a look at file_get_contents behind a proxy?
We're moving a fairly large website from domain.one where it's been for a long time onto domain.two. If people still find links for domain.one we want them to redirect to an appropriate place on domain.two (if possible).
Domian.one is no longer required after the switch. I don't know anything about moving an entire domain so could use some advice on the best way to go about switching whilst retaining the SEO gained over the years.
Any help is appreciated.
Many thanks
Put this in an htaccess file in your root web directory. It will forward your users, and search engines, to the new URL on the new domain.
Options +FollowSymLinks
RewriteEngine on
RewriteRule (.*) http://www.newdomain.com/$1 [R=301,L]
ADD THE FOLLOWING LINES as posted by John .
Also log in to google webmaster tools->configuration->change of address
and change your url ther so that seacrh engine results are also changed.
THIS iS VERY IMPORTANT
I have a website at a.com (for example). I also have a couple of other domain names which I am not using for anything: b.com and c.com. They currently forward to a.com. I have noticed that Google is indexing content from my site using b.com/stuff and c.com/stuff, not just a.com/stuff. What is the proper way to tell Google to only index content via a.com, not b.com and c.com?
It seems as if a 301 redirect via htaccess is the best solution, but I am not sure how to do that. There is only the one htaccess file (each domain does not have its own htaccess file).
b.com and c.com are not meant to be aliases of a.com, they are just other domain names I am reserving for possible future projects.
robots.txt is the way to tell spiders what to crawl and what to not crawl. If you put the following in the root of your site at /robots.txt:
User-agent: *
Disallow: /
A well-behaved spider will not search any part of your site. Most large sites have a robots.txt, like google
User-agent: *
Disallow: /search
Disallow: /groups
Disallow: /images
Disallow: /news
#and so on ...
You can simply create a redirect with a .htaccess file like this:
RewriteEngine on
RewriteCond %{HTTP_HOST} \.b\.com$ [OR]
RewriteCond %{HTTP_HOST} \.c\.com$
RewriteRule ^(.*)$ http://a.com/$1 [R=301,L]
It pretty much depends of what you want to achieve. 301 will say that the content is moved permanently (and it is the proper way of transferring PR), is this what you want to achieve?
You want Google to behave? Than you may use robots.txt, but keep in mind there is a downside: this file is readable from outside and every time located in the same place, so you basically give away the location of directories and files that you may want to protect. So use robots.txt only if there is nothing worth protecting.
If there is something worth protecting than you should password protect the directory, this would be the proper way. Google will not index password protected directories.
http://support.google.com/webmasters/bin/answer.py?hl=en&answer=93708
For the last method it depends if you want to use the httpd.conf file or .htaccess. The best way will be to use httpd.conf, even if .htaccess seems easier.
http://httpd.apache.org/docs/2.0/howto/auth.html
Have your server side code generate a canonical reference that point to the page to be considered "source". Example =
Reference:
http://googlewebmastercentral.blogspot.com/2009/02/specify-your-canonical.html
- Update: this link-tag is currently also supported by Ask.com, Microsoft Live Search and Yahoo!.