What is the best way to move a large and complex site (which will need extensive testing) to a new server? - dns

We need to move a large and complex site to a new server, and we need to ensure there is minimal downtime / disruption for our members.
This is what we have in mind:
Create a new domain (e.g. oursite.club) on the new server, for testing purposes. (We're planning on doing this because the temporary URL that works with some hosts / sites does not appear to work on ours)
Upload all existing code and import all databases to the new server.
Make any changes to allow for the new server path / domain name / different database names, where necessary.
Thoroughly test the .club site.
Add the current live domain (e.g. oursite.com) to the new server as an alias of the .club domain.
Once we're confident the new site works, change the DNS of the .com to point to the new server.
Also, add htaccess redirects on the old .com site to point to the same page on the .club domain, so that the site can still be accessed by those members for whom nameservers are not yet propagated.
After everything is working and all nameservers should have propagated, we want to make the .com the "real" domain and optionally retain the .club as an alias (i.e. swap the two domains on the new server).
This is what we have in mind:
Delete the add-on domain from the new server (i.e. the .club) and the alias domain (i.e. the .com) - but leave all the code / data where it is.
Re-add the .com, but point that at the server path where the .club code was uploaded.
Re-add the .club, but as an alias of the .com (or use URL forwarding at our domain registrar).
Make sure everything still works.
Update the htaccess redirects on the old server to send .com traffic to the .com domain on the new server, just in case there are any nameservers still pointing to the old server.
Since the nameservers for the two domains are already pointing to the new server, and that won't change, we thought this would offer the fastest way of swapping these two domains around.
We've done some testing with spare domains and a default WP blog, and this seems to work, but we wanted to call on your collective experience here to ask:
A. Will this work?
B. If not, what have we forgotten to take into account?
C. If so, good, but is there a better / easier way to accomplish this (especially the bit where we flip-flop the two domain names on the new server so that the current .com domain remains as the primary one to use)? For example, would it work if we add the .com to the new server (instead of creating a .club) and then edit our local hosts file during testing? Or can we use a VirtualHost directive to help with the domain swapping?
Although we've been building / running site for many years now, this is the first time we've had to move a membership site that is business-critical and where any excessive downtime / issues will cause us and/or our members real issues.
Many thanks!

I'm not sure if I'm missing something significant here (I don't entirely understand your need to flip flop the domains at the end), so I'll quite likely change this answer depending on comments etc (or delete it if I've completely missed a point!)
My approach to this would be
Set your DNS TTL for www.example.com to a very low number, 600 seconds perhaps. Don't plan the move until your current TTL expires (from the moment you make the change)
Create website on new server,configure the site under your example.club domain and example.com
Configure the hosts file on your testing machines with the IP Address of the server, and www.example.com This way your testing machines are going to actually test the full structure of example.com.
Test against example.club if you are concerned about broken links to the new domain, but since that is only temporary, it shouldn't really matter.
when it is tested and you are confident it is working, switch the DNS over.
Once you are comfortable that everything is working as expected reset your DNS TTL to a more sane figure.
As long as your database is in sync, there would be minimal (if any) downtime from this and at the end you can simply delete the example.club reference without any impact.
Depending on the value of the site, and the impact of potential downtime it might be worth putting a load balancer between both sites, and gradually increase the traffic that is sent to the new site so you can monitor progress. (you could use a load balancer from Azure which has no restrictions on being used against external sites)

Related

301 Redirect Best Practices - multiple sites to single site

I have around 15+ sites, and we want to drop these sites and merge them into only one site (create pages for each one in the new site).
The 15+ site's domains should be redirected to one site as below:
a.com -> z.com/a
b.com -> z.com/b
c.com -> z.com/c
..
Also, we want to redirect (301) page by page from the old domains to the new the new domain to keep the page's ranking:
a.com/about-us -> z.com/a/about-us
b.com/about/abouus -> z.com/b/about-s
c.com/contactus -> z.com/c/contact-us
Each one of the 15+ sites is running on its own server with a different platform while the new server is IIS.
Currently, I'm thinking of two approaches:
Point the old DNS records to the new server of z.com, and handle all the redirects on the server.
Keep the old site running, and configure redirect rules on each server to redirect each page to the matching page on the new site.
Which approach is better, any other approaches? So far I think the first approach is better since we will control all the redirects in one place - but from the performance wise, is it going to add more headache on the server?
The Scenario
You created a new site and you want to redirect all old sites(15+) to new site (page to page).
All old sites are running in it's own server on different platforms whereas the new one is IIS.
Your options
Point the old DNS records to the new server of z.com, and handle all the redirects on the server.
Advantage
Less cost - you can remove all old sites and the server cost can be saved.
Internal redirection. So, time will be less
Disadvantage
Complex to perform
take care of conflict of similar pages(double check the redirection path)
Keep the old site running, and configure redirect rules on each server to redirect each page to the matching page on the new site
If you can afford the cost of running 15 different sites on 15 servers just to redirect, then only go for this method.
It is just a waste of money and the redirection time will be more.
I think eventually you'll have to shutdown all the other sites as in the long term unlikely does it make sense to keep 15 sites running just to do the redirects.
So as I understand the question is rather on how to better organize the migration to the new system in the short term. So here are my thoughts on this:
how huge is your system
what's your QPS?
how many pages do you have across your sites farm?
do you need to remap URLS for a decent amount of pages?
what's the migration procedure? Will you switch your sites one-by-one or it's technically infeasible and they all need to be swtiched over at once?
If we're talking about a system handling 10 QPS and 1K pages or about a system handling
50K QPS and having 1B pages we need to dynamically remap system load may be a concern and p.2 may look better
rollbacks
note that DNS records can be cached by intermediate servers and if you need to quickly rollback to the previous version if something goes wrong it can be an issue
what kind of systems do you have
Is it actually possible to easily extract URLs from 15 diverse systems and put them to a single point without a risk to lose something valuable?
ease of maintenance
At first glance the first approach looks easier from the maintenance perspective, but I don't know what kind of system you use and how complex the redirection rules need to be.
If they are complex dynamic ones like a.com/product.php?id=1 => z.com/a/iPhone6S moving millions of such urls to a single point could be tricky
SEO
I don't follow the industry closely, but a few years ago both would work ok. I think it's worth consulting those keeping up to date with this industry - it changes very rapidly
Your first approach is definitely the best.
It is easy to maintain
You needn't to keep old infrastructure (though in your second case you'll need to keep only redirecting frontend like apache, nginx or lighthttpd)
There are no performance risks as from one visitor request to old location, redirect answer and than request to new location will come in turn and not simultaneously.
DNS records are not capable of HTTP redirection which is crucial for SEO. To make sure your redirect is 301 HTTP redirect you can use sniffer.
The answer is just make sure your redirect is 301 HTTP redirect so you get your SEO right. Other than that it's just a matter of taste / architecture / money rather than standards.
UPDATE
Read more:
wikipedia
Both Bing and Google recommend using a 301 redirect to change the URL of a page as it is shown in search engine results.
ehow.com
A 301 redirect is a search-engine friendly way to move a domain. The 301-redirected domain does not cause duplicate content in the search engines so that you do not harm your search engine rank. Using a new DNS setting is required to have a new domain name, but it does not redirect browsers or search engines. Both of these methods are used to move to a new domain.
webmasters.stackexchange.com
Duplicate content occurs when the same content is available on two different URLs. To prevent duplicate content on www vs no-www, use 301 redirects to redirect one to the other. To implement redirects, it is the webserver that needs to be configured properly. As long as DNS is pointing to the webserver (either CNAME or A record), then the webserver can be configured properly.
I was stuck with this problem some months back. I wanted to redirect a whole site into a new site's structure. The old site was .php which I know nothing about.
I figured I'd point the old website's DNS at my server and write some MVC code to catch every request, and then use a set of rules using the vb.net Like operator to compare the inbound Url with my ruleset.
It worked a treat. I redirect 300+ pages to my new site with about 10 rules. These include changes of folder structure, a forum (which was mainly junk but had a few good questions), and I implement a "catchall" rule which points to the new home page, in case I missed something.
It worked so well I've packaged it up as a commercial product and it publically available. It is free with link from the destination site (in your case just the single destination site).
https://301redirect.website/
There are a couple of demo videos on the homepage which will explain the setup in a few minutes.

Put a subfolder onto a different server with CNAME

Here's the situation. Website.com is an ASP site which needs a blog that is to be Wordpress. So the website.com/blog needs to be hosted onto a php-friendly server. The company hosting the ASP site doesn't want to have anything to do with Wordpress so we have to use some of the shared hosting providers.
How do I have the Blog section placed onto an entirely different server? I've heard this is done with CNAME, but I've never used it. Most of the research I've done revolves around subdomains, but I need a subfolder mapping, and there's not much to read about putting subfolders onto different servers with a different IP and everything.
Thanks.
There are a few different options:
you can bring the traffic to your own server and then redirect to
the correct location
you can bring the traffic to your own server and then proxy it to the correct location
you can direct the traffic to the correct location either via full page or an IFRAME type mechanism
Each option has some benefits and drawbacks depending your devs knowledge level and your infrastructure. Regarding subdomains, you could use a combination approach where you, for example, use subdomain.yourdomain.com to point to a server instance (can be the same server or a totally different one) that maps the subdomain.yourdomain.com name to a specific path, usually via Host header.
A CNAME is a function in DNS that says "Whatever thing you wanted to find for this name, use the same thing for that other name instead". When you're working with web stuff the "thing" in there is nearly always an IP address.
That is, what a CNAME can do for you is to say that when a user's web browser tries to look up the IP address for website.com, it will use the IP address for someotherwebsite.com. Note the total absence of anything web-related, like subfolders, in this. CNAMEs work on whole domain names, nothing else. Since you want to serve only a part of the stuff at a particular name from another server, CNAME cannot help you. CNAME is the wrong tool for you problem. Do not taunt happy fun CNAME.
In order to serve website.com/blog from another server than website.com, you pretty much have to do some sort of reverse proxying (where the ASP site's server relays requests between the user and the Wordpress server). It's probably easier and more robust to give the Wordpress site its own name (blog.website.com or something), and redirect to that from website.com/blog, but only you can know if that's politically possible in your case.

Domain name point to subdomain of a entire different domain

I am completely confused the last few days with this and I still haven't found an outcome that's worked.
Basically I have a domain name without hosting at letshost.ie which is dublinplasterer.ie
I also Have one domain & hosting with godaddy for domain- shanafagan.com which is my own site for web/graphic design service.
I created a subdomain= dublinplasterer.shanafagan.com and uploaded the site files.
Basically I want for example when someone types in dublinplasterer.ie in the address bar if goes to dublinplasterer.shanafagan.com but doesn't show dublinplasterer.shanafagan.com url, stays as dublinplasterer.ie
Im not even sure if this can be done at this stage. head is melted
shanafagan.com and the subdomain dublinplasterer.shanafagan.com have the same ip so how will that work if changing dns?
Any help would be greatly appreciated , am so stuck at this stage.
If you wanted to do it this way you would need a web server for domain2.com
Search for ProxyPass.
The way you should do this is add a second domain on your web hosting (cpanel for example) and point the other domain to this web server.
If you are struggling I recommend using a solution like cPanel that is widely used and simplifies much of the process. It is common enough you can google most issues.
So normally you would have started by making an add-on domain (instead of a subdomain) which would also create it's own subdomain anyway. To do that, you go to your cPanel in GoDaddy and find add-on domain, then make it "dublinplasterer.ie" (Don't add www. to it. Even though this name is hosted elsewhere, we will later go to your DNS files at that hosting and point it to your GoDaddy's name servers and this add-on helps it direct to the right root folder) then choose your local root folder for that site (I think you can actually make this the same as your other subfolder already hosting your files and then it will just pull the same site) or you can pick a different subfolder and then make the add-on domain. This tells any request to this name server that if it is a request for "dublinplasterer.ie" it needs to send it to the subfolder you specified.
If you don't make the subfolder the same as the one you already made, you can either load the same content into your new subfolder or create a CNAME record telling this add-on to point to your subfolder instead but that is more complex so go with the other route.
Lastly, you need to go to your original hosting at letshost.ie and under your domain name find the DNS records tab. Change the name servers to match your GoDaddy ones and now (may take a day or two to show as DNS changes often take days and can't be seen immediately but you can try using a different device/computer/mobile that hasn't loaded it previously to see if it will refresh the correct dns) it should work. Even though it points to your main site name servers, the add-on domain you made receives it and directs it to the subfolder you specified on the add-on domain.
Hope that helps, let me know if it works for you.

Website going down intermittenty, but server is always up

I have recently set up my domain, but my problem is when I navigate with/without www, the results are inconsistent. They are supposed to point to the same page, but sometimes, the www site has broken links to non-www components (e.g.: when I load http://www.my-domain.com, and if I have a resource referred as http://my-domain.com/navi.css, it breaks)
Have tried 301 rewriting, but it's not always consistent. What do I need to do to ensure 100% uptime for my site?
This could be because of a couple of reasons:
Your DNS has not propagated yet, meaning if you recently just changed DNS on the domain name give it up to 48 hours and flush cache from your browser.
The domain DNS is pointing to different IP addresses from your www. or * A records.
There is a problem with the hosting configuration (unlikely if you are using CPanel or any hosting control panel as this is mostly automated).
Also you should pick either with or without www for consistency and better SEO results. Pick one and stick to it.

Transferring Everything from One Domain to Another

I want to move from one domain to another one that is shorter and (hopefully) easier to remember, but I also want the transition to be as seamless as possible.
I'm a coder so I know about 301 redirects and I intend to use them, but is there something easier than manually adding entries in my .htaccess file?
The website is static with the exception of the blog, which is under the /blog/ directory and powered by WordPress. I want it redirected to my new domain name. File and folder structures don't change. Isn't there some kind of wildcard thing for this?
What type of web server is the site being hosted on? If its on Apache, you can use the RedirectMatch command
Why not leave the web site unchanged, and [temporailly] let the both the old and the new domains assume the IP of the site, at the level of DNS ?
In other words.
Let the DNS setup for the existing domain as-is (for now)
have the DNS of the new domain assingn the particular server name to the IP of the web server (or preferablly to a 2nd IP, also assigned to this very site).
In a few months / whenever the old domain is soon to expire, check what residual traffic may still be using the old domain name (Old IP), and for them only, add a small piece of text somewhere in the UI to inform them of the new domain.

Resources