DNS Settings for Google Cloud Storage and crawling problems - dns

I have a static website hosted on a Google Cloud Storage Bucket. I also have moved the DNS to Google Cloud DNS. So everything is in control on the Google Cloud Platform.
I only have some problems left:
How to solve www / non-www requests? I pointed the non-www now to a server outside the google cloud platform which redirects to www. I don't see another possibility to refer both directly to the same bucket.
How to solve http / https? SSL is not possible yet on buckets. The https version can be reached ofcourse, but I get a warning and the googlebot has problems with it too. For some reason they indexed some https versions of some pages while I have set canonical links to the https version. And I don't see any possibilities to set up redirects correctly.

Unfortunately, that is not possible at this time. When using CNAME redirects to GCS, a bucket must exactly match the specified domain. You may not map several domain names to one bucket or vice versa.
Furthermore, "domain.com" with no subdomain is, unfortunately, right out, as DNS CNAMEs cannot be set on root domains.

Related

Redirect www to non-www on Azure Dns Zone

I have created Azure DNS Zone to manage redirections which use domain from namecheap.
Let's assume that my website is www.johndoe.com and currently website working well for johndoe.com but when I call www.johndoe.com it doesn't work well.
I have CNAME record and in CNAME redirect to redirect but unfortunately it doesn't work.
When I have call www.johndoe.com response message is as following...
Our services aren't available right now
We're working to restore all services as soon as possible. Please check back soon.
Please take into account that DNS does not provide a redirect.
An additional A or CNAME record only provides your IP address to people that navigate to that specific URL. When www.johndoe.com and johndoe.com both resolve to your web server, this means each page on your site has two addresses. This is horrible for things like SEO.
For a true redirect (and a 301 is preferred for SEO), set something up in the service that's handling the traffic coming in.
For instance: if you have an App Service running ASP.NET, setup the redirect in web.config. If you have Front Door (or a CDN) in place, use the rules engine to create a redirect with the correct condition and action.
Here's an example redirect in an Azure Front Door Classic Resource (this one redirects the other way):

How can I use Azure CDN to deliver my entire site with root domain?

I created Azure CDN Verizon Premium.
Endpoint Hostname: https://mysitename.azureedge.net
Origin Hostname: https://mysitename.com
Origin host header: 123.456.789.12 ( My Server IP )
My Server is: CentOS + Litespeed
DNS Provider: Cloudflare
Normally, I point to my server from CF DNS
A - # - 123.456.789.12
CNAME - www - mysitename.com
With these settings, I can access my site (NO proxied)
I wish to use Azure CDN for the root domain.
I don't like to use www.mysitename.com, But I like to use root/naked domain (mysitename.com)
So, I changed my DNS like this:
CNAME - # - mysitename.azureedge.net
CNAME - www - mysitename.azureedge.net
Thanks to CNAME flattering in Cloudflare
(This technic worked with QUIC.Cloud CDN - when I used their NODE URL on CF DNS)
NB: Quic.cloud is still in Beta - But a nice service - worth trying.
So, I used the same trick to have in Azure CDN (I don't know it'll work or not)
After I put the endpoint hostname in my browser, I get a 504 timeout error. As you know, Azure is NOT allowing an SSL certificate on the root domain.
So, how can I fix this trouble?
What do exactly I want
A dynamic CDN, Not the traditional URL mapping CDN.
Need to deliver the entire files & folder to edge locations (Verizon & Akamai Dynamic Delivery)
Why don't like to use URL mapping
The site on a root domain and CDN in a subdomain is possible.
Ex: site URL mysite.com & CDN URL: cdn.mysitename.com (currently I'm using this setup in few sites)
But, the real problem is:
the CDN URL will be quick, Files can be delivered. But, The Website root domain, still deliver from the origin server.
So, TTFB is delayed when users from India access the site hosted in the USA.
Did you find a solution?
Yes, Cloudflare & QUIC is the best solution for this, because they both work through DNS level.
Cloudflare asks us to point our domain to theirs NS and they deliver dynamically (if proxy is enabled)
Each country has better TTFB than, origin (I tested with CF)
But why am I not using Cloudflare?
Cloudflare has some SEO issues. I faced many times on many occasions. CF is fast but dropped my ranking suddenly.
Why I'm not using QUIC.Cloud?
I'm using Quic on some small sites. It works well.
QUIC works, by pointing CNAME to their NODE URL (Their DNS solution is on the way). The entire site will be delivered through its node.
But, It's still in beta and not capable to deliver my requirements always as I expected. Maybe they will fix small bugs asap.
Then, what do you want?
I need to deliver my entire site through CDN. so I can increase the TTFB in every location. CDN for the root domain is the solution (I think so)
CNAME to endpoint hostname will work?
is there I miss anything, in my DNS or Azure CDN setup?
You need to have your DNS managed in Azure, and then create an Alias record for the root domain, pointing to the CDN.
https://learn.microsoft.com/en-us/azure/dns/dns-alias
After that you will want the SSL certificate, it is allowed on root domain, but it's not managed by Azure and you need to upload it to the Key Vault and bind it through that.
https://learn.microsoft.com/en-us/azure/cdn/cdn-custom-ssl?tabs=option-2-enable-https-with-your-own-certificate

Custom Domain to Azure CDN Endpoint with and without www

I have created an Azure CDN, an endpoint and working fine.
Created a CNAME www.mydomain.com to myazurecdn.azureedge.net
Created another CNAME mydomain.com to www.mydomain.com
In Azure Endpoint, Custom Domain, I updated www.mydomain.com and it got propagated.
Now, when I navigate to www.mydomain.com, everything works fine but when I navigate to mydomain.com i get below response.
Our services aren't available right nowWe're working to
restore all services as soon as possible. Please check back soon.
Note: As Azure endpoint accepts multiple custom domain, I could add both www.mydomain.com and mydomain.com but it is not correct way, as I have many <a> tags with www.mydomain.com/link.
What to be done now to instruct system to use same route for mydomain.com and www.mydomain.com?
Is domain redirect from domain.com to www.mydomain.com only option?
Have a look at this,
A custom domain and its subdomain can be associated with only a single
endpoint at a time. However, you can use different subdomains from the
same custom domain for different Azure service endpoints by using
multiple CNAME records. You can also map a custom domain with
different subdomains to the same CDN endpoint.
You can use different subdomains in the same endpoint. Alternatively, I think you also could create two endpoints, one is for the root domain without www, the other is for a subdomain with www.
I would say, you can implement redirection in your server or CDN.
When someone tries to access example.com, you can do a 301 by sending http://www.example.com in the location header.

Google Cloud DNS and 301 redirects

Namecheap and other registrars provide a way to setup HTTP 301 or 302 redirects when configuring DNS. It is a nice feature since otherwise you would need to implement it yourself. See screenshot below from Namecheap DNS configuration page.
Does Google Cloud DNS offer something similar?
Google Cloud DNS does not provide a "URL Redirect". This isn't actually a DNS feature - it requires a webserver to serve the 302 redirect. You could implement this with a small cloud funciton like:
exports.redirectFunc = function redirectFunc (req, res) {
res.set('Location', 'https://example.com/destination');
res.status(302);
res.end();
};
Firebase now has a very simple setup to achieve domain redirection, just do the following:
[Your firebase project] > Console > Hosting > Connect domain
Then it should display the following:
Although Google Cloud DNS does not have this feature, if you register your domain with Google Domains, it has a "subdomain forwarding" feature (https://support.google.com/domains/answer/6072198 that implements not only 301 / 302 redirects but also provides integrations for G Suite and other platforms that set up other types of synthetic records (https://support.google.com/domains/answer/6069273).
Google Domains runs on the same DNS serving infrastructure as Google Cloud DNS, but currently you have to choose between the "Google Domains name servers" and Google Cloud DNS name servers that serve managed zones that you can control programmatically with the Google Cloud DNS API. This means you have to choose between the Domains-specific features like synthetic records and the Cloud DNS API and GUI console. You can delegate subdomains of a domain hosted on the Google Domains name servers to specific managed zones in Cloud DNS, so it is possible to mix and match this a bit.
Though it isn't supported directly in Cloud DNS yet (see issue: https://issuetracker.google.com/issues/70980380) a workaround can be to deploy an HTTP(S) Load Balancer and set up a redirect in the URL-map and then map the root domain to the IP address of the load balancer via an A record in Cloud DNS (and the same load balancer can be used to serve all subdomains as well without requiring any other Cloud DNS records), see: https://cloud.google.com/load-balancing/docs/url-map-concepts#url-redirects

Disable azurewebsites.net subdomain

We have our company website hosted at Windows Azure Websites and set up an A-Record for our domain.
The problem is that one can also reach our site using the mangomint subdomain of http://azurewebsites.net (I don't want to post the full url here because Google shouldn't see this). This is bad in terms of SEO and duplicate content.
How can I disable the subdomain in favor of our main domain?
Well I don't think you can change the official azurewebsites.net DNS records, so you have only two solutions :
Find a way to disable this subdomain in the Azure Websites configuration EDIT: Not possible, from what astaykov said in the comments. thanks to him.
Make your website automatically redirect (HTTP 301, permanent redirect) the user when he lands on the subdomain.

Resources