We have an application that we are hosting in multiple environments in Microsoft Azure. We want to route the traffic based on subdomains, like xxx.mydomain.com should go to the webapp that I have in North Europe and yyy.mydomain.com and zzz.mydomain.com should go to the webapp that I have in the East US.
I know it sounds like simple DNS, but it is more than that. Because:
I need to be able to add or update entries dynamically using code so an API should be available for that.
A normal DNS entry has a 24 hours time to live meaning that if I want to move my app from one environment to another, for up to 24 hours, users will hit both environments.
I expect to have hundreds of thousands of subdomains. Azure DNS has a limit of 25,000 entries.
I've looked into Azure Traffic Manager. It doesn't seem to have an option for traffic based on subdomains.
Also, I've looked at Azure Application Gateway. It seems to be the correct choice and it supports API's, but I cannot find the limits for subdomains.
Any suggestions?
From the criteria, it seems you're looking for a load-balancer/proxy/application-delivery-controller solution that's controllable through an API. I'll add my 5 cents here, as we've just gone through very similar problem. However these are more of a suggestion to look for answers elsewhere then Azure.
Azure
Azure Traffic Manager or Azure Application Gateway have limits which you can't fit in. For example in Azure Application Gateway with 200 rules, you could potentially host only 200 HTTPS site, the moment you need to serve HTTP & HTTPS, you're limited to 100 sites per application gateway. You'd need to split your solution across multiple subscriptions in order to fit subscription wide limits. Also the application gateway API is a bit too convoluted for my liking.
Azure DNS is also a bit problematic, as DNS records can last up to 24 hours. You'd therefore loose the ability to switch/route traffic to a different origin instantly.
Self-hosted
You could look into more old school solutions, run HAProxy or Nginx and programmatically modify their configuration(text files) on the fly and reload the configuration. HAPRoxy also has a socket "API" that can simplify the configuration modification and reload for you.
There's also a new set of service mesh controllers such as Kong, which can run in the cloud natively and are meant for service mesh solutions, however Kong offers a simple API, where you could manage/route traffic easily.
SaaS
If you're into buying this as a Service, Edge Cloud providers such as Cloudflare, Fastly or others are indeed "one big proxy server" and it is possible to configure them programmatically to route traffic to different origins, it's what they do after all.
Azure Application Gateway is indeed perhaps one of the best options for your scenario.
As you already said, it has an api that you could use to dynamically add rules based on your subdomains.
The limits for Application Gateway only allow for 200 rules per gateway.
But you can have 1000 gateways per subscription so if you could chain the gateways, that will give you roughly 200.000 rules.
The Microsoft documentation doesn't show that you can request an increase in these limits but maybe if you ask really nice the might allow it.
Maybe this is not the answer to your question but it might be an answer.
If anyone interested, we've ended up using Azure DNS. We have contacted Microsoft and they confirmed that they can increase the quota to 500,000 which is more than enough for us. :)
Related
We have deployed our website recently using an application gateway but the bandwidth (125 CU) isn't enough to handle the very high traffic of the website.
We are thinking about switching the application gateway by a Frontdoor since it looks like its bandwidth is higher however we don’t know if there are anything we should know by using this method. (security, best pratice, cost, ...)
Our website is host in 2 same App service Plan (with 4 to 5 App Service in each of theme) in the same region.
Please help
Why don't you guys use actual firewall (not Azure WAF, rather something like PaloAlto, Fortigate, Cisco FTD etc) for this. Not only it will work as application gateway but will also provide you security and save you from a lot of attacks. It is fairly simple to deploy a firewall from market place, and vendor support is also great. Depending on how good you negotiate the price difference will be marginal.
can I use one CNAME record for two Azure Web App without a third-party service or services Azure? For example, so that I call testcname1.com/api in the code and the traffic goes for example to the web app test-api, and if I use testcname1.com/indentity the traffic goes to the web app test-indentity. Two Web Apps test-indentity and test-api one CNAME testcname1.com. Thanks.
On a pure DNS level, you can't point a CNAME entry to two different other adresses.
Azure Application Gateway is designed to do exactly what you want, but if I understand you correctly, you don't want to use other services.
As an alternative, you could set up a third AppService and host a reverse proxy, e.g. ngnix using WebApp for Containers. Or use IIS features directly, e.g. like this.
As Lex Li pointed out, you could also use one of your existing app services as a reverse proxy in addition to what it is already doing. Depends on how strongly you want to couple your services. Personally, I would probably use a dedicated instance just as a proxy, but both approaches can be valid.
Azure Traffic Manager does exactly what you want I think.
It is not free but for light/moderate load should be (very) cheap.
Does health checks as well.
from https://azure.microsoft.com/en-in/pricing/details/traffic-manager/
- $0.54 per million queries
- $0.36 per Azure endpoint/month
I have my website (abc.azurewebsites.net) hosted to Azure Web Apps using Visual Studio.
Now after 1 month I am facing problems with traffic management. My CPU is always 90 - 95% as the number of requests is too high.
Does anyone know how to add Traffic Management in this web app without changing the domain abc.azurewebsites.net? Is it hard coded in my application?
I thought of changing the web app to a Virtual Machine but now as it's already deployed I am scared of domain loss.
When you Scale your Web App you add instances of your current pricing tier and Azure deploys your Web App package to each of them.
There's a Load Balancer over all your instances, so, traffic is automatically load balanced between them. You shouldn't need a Virtual Machine for this and you don't need to configure any extra Traffic Manager.
I can vouch that my company is using Azure Web Apps to manage more than 1000 concurrent users making thousands of requests with just 2-3 instances. It all depends on what your application does and what other resources does it access too, if you implemented or not a caching strategy and what kind of data storage you are using.
High CPU does not always mean high traffic, it's a mix of CPU and Http Queue Length that gives you an idea of how well your instances are handling traffic.
Your solution might implementing a group of things:
Performance tweak your application
Add caching strategies (distributed cache like Azure Redis is a good option)
Increase Web App instances by configuring Auto-Scaling based on HTTP Queue Length / CPU.
You should not have to change your domain to autoscale a Web App, but you may have to change your pricing tier. Scaling to multiple instance is available at Basic pricing tier, and autoscaling starts at Standard tier. Custom domains are allowed at these levels but you don't have to change your domain if you don't want to.
Here is the overview of scaling a web app https://azure.microsoft.com/en-us/documentation/articles/web-sites-scale/
Adding a Virtual Machine (VM) is very costly as compared to adding instance. On top of it, Redundancy (recommended) for the VMs, adding NIC etc will blow up the cost. Maintenance is another challenge. PAAS (webApp etc) is always a better option than IAAS.
Serverless offerings like Azure Functions can also be thought of. They support http trigger and scale up really well.
For the last couple of weeks, we have been seeing an increasing amount of entries in the web logs of our Azure website whose originating IP address (in the c-ip column of the log) appears to be in the range 100.90.X.X. It has now reached more than half of all the traffic being logged, and is interfering with our ability to perform analytics and threat detection.
According to the Wikipedia entry on reserved IP addresses, this block is part of one "Used for communications between a service provider and its subscribers when using a Carrier-grade NAT, as specified by RFC 6598", so could this be a problem in Azure?
Looking at the logs, the traffic comes from many different user agents (both normal users and the common legitimate bots) and is requesting a broad range of resources, so does not immediately appear suspicious other than the IPs. It looks more like legitimate traffic is being given an incorrect (internal) IP.
It seems to be only affecting static content (e.g. images and XML files), but not ALL static content.
We are using a single Small Standard instance in Western Europe, with a single web app running on it. We are not using any scaling features. There is a linked SQL database, and the website runs primarily over HTTPs. 95%+ of our traffic comes from UK sources. We have not made any changes to logging, which is handled by Azure.
Is there any way that we can return to seeing the actual IPs here, or is this malicious traffic?
It’s possible to alter the logging, but not on an app. The app diagnostic setting is pretty rudimentary — just a switch “to log or not to log?”
What you’ll be interested in is this comparison between apps (it was called “sites” then), roles (available through Cloud Service)1 and virtual machines. The article mentions that there is more control over logging in the roles environment, which I would assume means that you can set up custom logs. This article details how to set up logging for the headers you choose in IIS. Now, you can fiddle with your IIS in a virtual machine, but there is a chance a cut-down version of this would work in a web role, for example. This article discusses how to enable diagnostics logging in your cloud service hosted application.
Moving to cloud service from app environment is not trivial, since you have many more things you must set up. Possibly you’re looking at changing your solution’s structure, maybe altering the architecture of your app. So I wouldn’t consider doing it just so I could see a client’s IP.
The simplest thing you can do is try attaching analytics. There used to be a solution straight from Azure, but I can't find it in the portal. Google analytics is my go-to solution for traffic analysis. It may get you the information you want.
It’s really annoying how Microsoft rebrands an azure service every few months.
We have a client that we have setup two webservers running about a dozen different websites spread over each webserver for each site (so site 1 is running on both webservers, site 2 is running on both ect).
We want to setup load balancing (presumably with Traffic Manager) so that it will spread the load and monitor the sites availability across both the web servers per site and not per virtual machine like the regular Azure load balanced sets do.
It seems that traffic manager might be able to do this but we don't know how it works and the documentation is not clear to us about if we can setup the TM per website and not per web server (as we understand it).
Can we setup any load balancing in Azure that will monitor the website itself and not the virtual machine? Because we want to run multiple sites on these two web servers, how do we setup TM do allow this to happen? Do we make a new TM service for each website?
thanks in advance.
UPDATE: do we even need to setup the regular load balancing set at all? Can I not just create a TM and click on the cloud services of each of the web servers and then add the port and relative path to one of the websites? and just repeat this for each website on the two servers? would that be how it should be setup?
Thanks again.
The available endpoints you can define in your Traffic Manager definition is only as granular as your cloud service URLs. So, the quick answer to your question is no.
You may want to look into using Application Request Routing in front of your two servers instead. It does introduce additional server(s) in your configuration but will give you a way to load balance across the sites.
http://www.iis.net/downloads/microsoft/application-request-routing
The answer to this question was custom end points from traffic manager. This will allow Azure to be able to monitor the service for each service or website on a web server within Azure.