Prevent Subdomains being viewed/crawled - .htaccess

cPanel only allows me to create 'AddOn' domains. I have pointed all my TLDS to the server which saves them under 'public_html/main/sites' directories '/site1.com' , '/site2.com' etc. mainwebsite.com will be served under 'public_html' and all my client sites under 'public_html/main/sites'
It also creates subdomains 'username.mainsite.com' How can i prevent google from indexing those subdomains yet still index the TLDS. And stop users from being able to access the TLD from the subdomain too?
If i created a RewriteRule would google still index the TLD? Or is there a better way to go about this?
RewriteEngine on
RewriteCond %{HTTP_HOST} !^www\.site1\.com$ [NC]
RewriteRule ^(.*)$ http://www.site1.com/$1 [L,R=301]

If the 301-redirect works (for you), it will work for search engines, too.
See Google’s documentation:
301 redirects are particularly useful in the following circumstances:
[…]
People access your site through several different URLs. If, for example, your home page can be reached in multiple ways - for instance, http://example.com/home, http://home.example.com, or http://www.example.com - it's a good idea to pick one of those URLs as your preferred (canonical) destination, and use 301 redirects to send traffic from the other URLs to your preferred URL.

Related

Fake subdomain for different users while still browsing same folder structure

I'd like to create fake subdomains for different users for more vanity, and to make the user (in this case a company) feel they are in a more isolated environment.
For the sake of maintainability, it's important for me that all users still browse the same files, to avoid having to update files for every single user that exists when updating the code.
My website has one public part at root, let's say www.example.com. I'd like to be able to fake the following kind of subdomains:
user1.example.com
The true URL would be www.example.com/member/?user=user1. I'd like for the folder structure to follow the same pattern. www.example.com/member/settings/?user=user1 would appear as user1.example.com/settings/ and so on.
I assume this would best be achieved with .htaccess, no?.
What is the proper .htaccess code for this?
Thank you!
based on the information you've provided here's a bit of .htaccess not tested but might do the job.
RewriteEngine on
RewriteCond %{HTTP_HOST} ^(www\.)?example.com$ [NC]
RewriteCond %{QUERY_STRING} ^user=(.*)$
RewriteRule ^(.*)$ http://%1.example.com/$2? [R=302,L]
Once you find it working fine, replace 302 to 301
Note: It's important to note that you'll need to add a subdomain wildcard
In the end, this was super easy to solve! It wasn't solved the way I first expected though, using .htaccess. I solved it with something called wildcard subdomain.
When you register a new subdomain, enter * in the domain prefix, such as *.example.com. A folder for the wildcard subdomain will be created on your server, such as _wildcard_.example.com. Whenever you access site1.example.com, fakesub.example.com etc, the browser of the visitor will read the files in the wildcard.example.com folder.
The beauty of it all is that if I create a certain subdomain that I want to use, for example forum.example.com, this real subdomain will have priority over the wildcard subdomain, and files will be fetched from the folder for this subdomain, as opposed to from the wildcard subdomain folder.
I use PHP and need to know the subdomain to fetch the appropriate database for the current user. To do this, I use the following code:
$subdomain = explode('.', $_SERVER['HTTP_HOST'])[0]
With a wildcard SSL cert I have all of these subdomains secure.

How to redirect entire domain to new domain except homepage via htaccess

So I have found some questions that are close to what I need, but I haven't found exactly what I'm looking for.
While I've worked with htaccess files in the past, it's pretty much been just copying and pasting code that is proven to work, or just doing simple 301 redirects. To be honest, I get pretty confused when looking at htaccess rewrite rules, so unfortunately are not going to be able to do from scratch. So I'm really hoping there is someone kind enough to provide a code example just using placeholder domain names.
Here is my situation. I'm doing work for a company who has split up into different divisions (4). So they want to take the site on their current domain, move it to a different domain. Then put up a simple html splash page on the old domain that provides links to the 4 different sites they will have (one of those being what was the old site on a new domain).
So I need to redirect everything from olddomain.com to newdomain.com with the exception of the homepage. Not sure if this matters, but site that was on the old domain was a WordPress site so technically is index.php. But the new site/splash page is named index.html.
Thanks in advance!
So I need to redirect everything from olddomain.com to newdomain.com with the exception of the homepage.
You can use this rule in the site root .htaccess of olddomain.com:
RewriteEngine On
RewriteCond %{HTTP_HOST} ^(?:www\.)?olddomain\.com$ [NC]
RewriteCond %{REQUEST_URI} !/(css|js|images)/ [NC]
RewriteRule . http://newdomain.com%{REQUEST_URI} [L,NE,R=301]
RewriteRule . will match everything except the home page just make sure this rule is the first rule in your .htaccess.

.htaccess - redirect dynamic url's from multiple domains to single dynamic url

I have an odd situation, where the client had been using multiple domains for the same site, but wasn't using 301 redirects on them. So the search engines have indexed:
www.site1.com/pageid=2
www.site2.com/pageid=2
www.site3.com/pageid=2
www.site4.com/pageid=2
all for the same page. I now need to redirect (all of them) to
www.site1.com/about
In the past I've successfully used this format:
RewriteCond %{QUERY_STRING} ^page_id=2$
RewriteRule ^/?site/$ http://www.site.com/about/ [L,R=301]
but that was just from a single domain. Is there a way to tell it to redirect that same page on site2, site3, and site4 also for the list of pages?
I'd like to keep this all within the same .htaccess file if at all possible.

Can you ReWrite a htaccess ReWriteRule?

Just wondering if its possible to 301 redirect an existing Rewriterule?
For example if I have the following line in my .htaccess file :
RewriteRule ^blue-widgets/ bluewidgets.php
and then I need to change my URL structure but the url "blue-widgets/" has a good ranking in the search engines which I dont wont to lose, is it possible to add another rewrite rule (301) that redirects that url too "newdirectory/blue-widgets/" ? If so, how is this done, is it a simple case of adding the new rewriterule under the existing one?
Does the fact that you have 2 rewrites, slow the page down or have any other problems?
You are confusing two quite different aspects: internal and external rewrites.
301 and 302 are external rewrites and in effect pass the redirect instruction back to the user's browser to do. 301 tells the browser (and the search engines) that the address change is permanent.
Rewrite rules without the [R] flag do an internal redirect -- that is a remapping inside the Apache / IIS subsystem than is not exposed to the outside world.
Yes, you can have multiple URI internally redirecting to the same target, but as you've written them, they will not be external and not 301s.
Try
RewriteRule ^blue-widgets/$ /new-directory/blue-widgets/ [L,NC,R=301]
RewriteRule ^new-directory/blue-widgets/$ bluewidgets.php [L,NC]
Does the fact that you have 2 rewrites, slow the page down or have any other problems?
The 301 to send blue-widgets to new-directory/blue-widgets is cached and will only happen once per client, so the performance should be minimally affected.
However, if you can, you should also change this link on your site to be new-directory/blue-widgets

URL/Subdomain rewrites (htaccess)

Say I have the following file:
http://www.example.com/images/folder/image.jpg
I want to serve it on
http://s1.example.com/folder/image.jpg
How can I do a htaccess rewrite to point it to it?
Like for example, I make a subdomain s1.example.com and then on that subdomain, I add a htaccess rule to point any files, to pull it from http://www.example.com/images/
Does serving files this way act as serving content from a cookieless domain?
First let me talk a bit about the concept of cookieless domains. Normally, when requesting anything over http, any relevant cookies are sent with the request. Cookies, are dependent on which domain they come from. The idea of using a cookieless domain is that you relocate static content that doesn't cookies, like images, to a separate domain so that no cookies will be sent with that request. This cuts out a small amount of traffic.
How much you gain from doing this depends on the type of page. The more images you have, the more you gain. If your site loads a big bunch of small images, such as avatars or image thumbnails, you might have a lot to gain. On the contrary, if your site doesn't use any cookies, you have nothing to gain. It's entirely possible that your page won't load noticeably faster, if it only uses a small amount of images, which will be cached between page loads anyway.
One thing to keep in mind, too, is that cookies set for example.com will also be sent with requests to s1.example.com as "s1." is a subdomain to example.com. You need to use www. (or any other subdomain of your choice) in order to separate the cookie spaces.
Secondly, if you decide that a cookieless domain is actually something worth trying, let's talk about the implementation.
Shikhar's solution is bad! While the solution appears to work on the surface, it actually defeats the purpose of using a cookieless domain. For every image, first the s1. url is tried. The s1. URL then makes a redirect to the www. domain which triggers a second http request. This is a loss, no matter how you look at it. What you need is a rewrite, which changes the URL internally on the web server, without the browser even realizing.
For simplicity, I'm assuming that all domains point to the same directory, so that www.example.com/something = example.com/something = s1.example.com/something = blub.example.com/something. This makes things simpler if you really need store the images physically in "www.example.com/images".
I'd recommend a .htaccess that looks a little something like this:
# Turn on rewrites
RewriteEngine On
# Rewrite all requests for images from s1, so they are fetched from the right place
RewriteCond %{HTTP_HOST} ^s1\.example\.com
# Prevent an endless loop from ever happening
RewriteCond %{REQUEST_URI} !^/images
RewriteRule (.+) /images/$1 [L]
# Redirect http://s1.example.com/ to the main page (in case a user tries it)
RewriteCond %{HTTP_HOST} ^s1\.example\.com
RewriteRule ^$ http://www.example.com/ [R=301,L]
# Redirect all requests with other subdomains, or without a subdomain to www.
# Eg, blub.example.com/something -> www.example.com/something
# example.com/something -> www.example.com/something
RewriteCond %{HTTP_HOST} !^www\.example\.com
RewriteCond %{HTTP_HOST} !^s1\.example\.com
RewriteRule ^(.*)$ http://www.example.com/$1 [R=301,L]
# Place any additional rewrites below.
Just for people's general info who like me may be investigating the benefits of this. From what I'm reading it isn't just cutting down on the upstream overhead of eliminating cookies sent with http requests. Apparently many browsers limit max connections to 1 domain/server to 6 concurrent. So if you have a separate domain on a diff server you get to double that to 12. Which to me would seem like the main potential here for a serious speed boost.
Though anyway, if I'm understanding this correctly. The other domain serving the static content needs to be located on another server from the main domain. Actually makes sense, avid firefox user and tweaker. When you check the about:config settings in firefox the max connections per server is set to 6 by default. A person can manually bump it up to a max of 8. But most firefox users probably don't spend enough time getting familiar with how to modify the browser and leave it to the default max of 6.
Not sure how many the other browsers set by default and then there is older browser versions that are still in use to consider. Bottomline ... makes perfect sense that enabling the browser to double the total number of connections using two servers would have to be a loadtime improvement. Using a sub-domain on the same server a person isn't going to be able to take advantage of that.
If you mean to redirect the traffic from www.example.com to s1.example.com, use the following htaccess on www.example.com
RewriteCond %{HTTP_HOST} ^(s1\.example\.com)
RewriteRule (.*) http://www.example.com%{REQUEST_URI}[R=301,NC,L]
If this is not what you are looking for, elaborate the question further.
I think you may have it backwards, (or very possibly I do). To clarify, if you're implementing a cookie-less subdomain & have a base URL of www. at least in this case, cookies are set on www, for example: a major cookie setter is google analytics, so when setting their script on my site it looks like this:
var _gaq = _gaq || [];
_gaq.push(['_setAccount', 'analytics-acc-#],
['_setDomainName', '[www.valpocreative.com][1]'],
['_trackPageview']);
You can see here that I set my main domain to www, correct me if i'm wrong in my case I would need to redirect www to non www subdomain & not the other way around. This is also the cname setup made on my cpanel (cname= "cdn" pointing to www.domain.com)

Resources