I have always worked on mobile technologies and now I am stepping into backend systems, more specifically systems design. I keep coming across conflicting statements for the roles of api gateway and load balancer. Googling has only returned the same half a dozen results that mostly focus on the implementations of load balancer or api gateway service provided by some famous service. I will list here all the confusing I am facing, in hope someone can clarify all of them.
Sometimes, i come across that API Gateway is the single point of communication with client devices. On the other hand, some places mention that 'request goes to load balancer, which spreads it across servers equally'. So what is correct? API Gateway receives requests or load balancer?
Other places, when I googled the topic, say that the two are totally different. I've understood that API Gateway does a lot of stuff, like SSL termination, logging, throttling, validation, etc, but it also does load balancing. So API Gateway is a load balancer itself, equipped with other responsibilities?
On the topic, I want to understand if load balancer distribute load among servers of the same cluster or across different data centers or clusters? And what about API Gateway?
What is so specific to api gateway that it is a choice by default for micro-service architecture? Where are API gateways hosted? A DNS resolves domain name to a load balancer or api gateway?
As it might be clear, I am totally confused. In what systems does a load balancer benefit more than API Gateway, if the question is correct.
API Gateway and Load Balancer are 2 different things.
Load Balancer -> Its a software which works at protocol or socket level (eg. tcp, http, or port 3306 etc.) Its job is to balance the incoming traffic by distributing it to the destinations with various logics (eg. Round robin).
I doesn't offer features such as authorisation checks, authentication of requests etc.
Whereas
API Gateway -> Its a managed service provided by various hosting companies to manage API operations to seamlessly scale the API infra.
It takes cares of the access control, response caching, response types, authorisation, authentication, request throttling, data handling, identifying the right destinations based on custom rules, and seamless scaling the backend.
Generally Managed API gateways by default comes with scalable infra, so putting them behind load balancer might not make sense.
About resolving the Domain, most likely always the DNS resolves to the load balancer, which in turn fetches the response from the API gateway service.
DNS -> Load Balancer -> API gateway -> Backend service
Hope I could explain and clear your confusion.
API gateway predominately does API management and provides various other key features such as IAM (Identity and Access Management), Rate limiting, circuit breakers. Hence, it mainly eliminates the need to implement API-specific code for functionalities such as security, caching, throttling, and monitoring for each of the microservice. Microservices typically expose the REST APIs for use in front ends, other microservices and 3rd party apps with help of API gateway.
However, normally, the API Management does not include load balancing function, so it should be used in conjunction with a load balancer to achieve the same.
In system architecture based on Azure, there is Azure Application Gateway which is a load balancer that runs on Layer 7 and provides more features than traditional load balancer ( Layer 4 ) in terms of routing traffic using routing decisions based on additional attributes of HTTP request or content of traffic. This can also be termed as an application load balancer. It shall be used in conjunction by Azure API Management (API gateway). Azure has a Traffic Manager for operating at DNS level which uses DNS to direct client requests to the most appropriate service endpoint based on a traffic-routing method and the health of the endpoints. Traffic manager also uses the rules configured at the DNS level and enables dstribution of the the load over multiple regions and data centers. Within every region or data center, there shall be application gateways coupled with load balancers such that, the application gateways shall help in determining the application server to fetch response from and the load balancer shall help in load balancing.
System overview based on Azure :
Here are few related references:
Azure Application Gateway - https://learn.microsoft.com/en-us/azure/application-gateway/application-gateway-introduction
Azure Load Balancer- https://learn.microsoft.com/en-us/azure/load-balancer/load-balancer-overview
Azure Traffic Manager - https://learn.microsoft.com/en-us/azure/traffic-manager/traffic-manager-overview
Scenario Architecture - https://learn.microsoft.com/en-us/azure/traffic-manager/traffic-manager-load-balancing-azure
There are two scenarios to consider here to clarify the confusion. I have clarified this using microservices example as this would make sense there only.
Scenario 1: You have a cluster of API Gateways
User ---> Load Balancer (provided by Cloud Providers like AWS or your own) ---> API Gateway Cluster ---> Service Discovery Agent (like eureka) ---> Microservice A ---> Client Side Load Balancer ---> Microservice B
Scenario 2: You have a single API Gateway
User ---> API Gateway ---> Service Discovery Agent (like Eureka) ---> Microservice A ---> Client Side Load Balancer -> Microservice B
I hope you understand why we required Load Balancer before the API Gateway in Scenario 1, as there we had multiple instances of API gateway also to handle the large traffic and to avoid the burden on the single api gateway since gateway itself can have several tasks to manage as per the requirements, so to distribute the load among them, we have load balancer.
Inside the microservices environment, you may need load balancing concept at multiple places. Like to accept the outside network you will maintain a load balancer provided by Cloud provider like AWS, eureka (Service Discovery) also acts like a load balancer if there are multiple instances of same service are registered with it and at last we also have client side load balancing (each microservice has its own client side load balancer that maintains a local cache) when your microservices are trying to communicate within them just to avoid burden on service discovery agent(eureka) by not checking every time with it the address for the other microservices.
Note: In the flow diagram, please don't confuse the path from API
Gateway --> Service Discovery --> to Microservice as if the Gateway is
forwarding request to Service Discovery that later routes it forward.
Gateway is just checking for the services registry from the Discovery
agents and then routing it to the correct microservice (Not through
the Service Discovery agent)
Load Balancer :
Its main purpose is to distribute by load balancing traffic between multiple back end systems.
We can configure different routes for different back end systems.
We get a static ip address for the load balances end points (usually not available with API gateways)
Can configure health checks (usually not available with API gateways)
In case of cloud providers, usually "Pay for idle as well"
API Gateway :
This as well routes traffic to back end systems based on URL
BUT, its main purpose is targeted towards "API management".
Below are such key features which are usually not available in "Load Balancers",
Can implement rate limiting, bursting limits
Can do request validation and request/response mapping
Usually cloud API gateways allows to export/import cross API platform using swagger spec
Allows caching responses
In case of cloud providers, usually "Pay per use"
DNS is responsible for routing the request to the nearest ip address inside network for a given domain name.
Api gateway is responsible for authentication, finding the correct apis(with or without load balancer) to call and circuit braking, response consolidation.
Load balancer is resposnible for distributing incoming request to different machine having same service deployed on them, on the basis of load or maybe round robin fashion.
So one way of doing it is
DNS TO GATEWAY TO LB
NOTE : LB can be placed before gateway depending upon traffic and use case
Related
Is it possible to include a separate Azure App Service as part of another App Service?
For example lets say that I have a website called www.mycompany.com hosted in its own Azure App Service. I have another Azure App Service that I want to make it accessible by going to a specific URL in the first App Service.
So in other words when a request comes to www.mycompany.com/eu/ I want the content of this endpoint (/eu) to be served from the other app service. Would a load balancer work? The reason I want to do this is because the /eu endpoint has grown too big and I would like to separate that from the main site and host it on a separate app service. I hope my question is clear.
Thanks in advance.
For this purpose you could use Application Gateway.
In a certain sense it resembles a load balancer (it is a L7 LB indeed) as you indicated, but the product provides many additional features.
The following image, extracted from the product documentation, explains how it works:
Basically, as outlined in the aforementioned docs, when describing how an Application Gateway accepts a request (note they mention WAF in the explanation, an optional security threat prevention system):
Before a client sends a request to an application gateway, it
resolves the domain name of the application gateway by using a Domain
Name System (DNS) server. Azure controls the DNS entry because all
application gateways are in the azure.com domain.
The Azure DNS returns the IP address to the client, which is the
frontend IP address of the application gateway.
The application gateway accepts incoming traffic on one or more
listeners. A listener is a logical entity that checks for
connection requests. It's configured with a frontend IP address,
protocol, and port number for connections from clients to the
application gateway.
If a web application firewall (WAF) is in use, the application
gateway checks the request headers and the body, if present,
against WAF rules. This action determines if the request is valid
request or a security threat. If the request is valid, it's routed
to the backend. If the request isn't valid and WAF is in Prevention
mode, it's blocked as a security threat. If it's in Detection mode,
the request is evaluated and logged, but still forwarded to the
backend server.
The routing to one backend or another can be based on URL Paths:
You can find an example of this configuration in this related Microsoft article.
In your use case, you will need to define two backends, one for every App Service, and define routing rules as appropriate.
As indicated before, your DNS should point to the Application Gateway: it will handle the routing to one or other App Service based on the route, /eu/* or /*, provided by the client.
The order of the routes is important: once a rule is matched, it will be the one processed.
Application Gateway is a regional service: Azure Front Door leverages a similar functionality (and much more) globally.
Please, consider review the associated costs of these services.
I am answering my own question to hopefully help others in the same situation. This was simply done using a reverse proxy. This article explains everything you need to set it up - https://ruslany.net/2014/05/using-azure-web-site-as-a-reverse-proxy/
Environment Details: I have an application hosted in two azure environments for two clients. The application contains ASP.NET web API backend and Angular Frontend. Both applications hosted on two web servers (Windows VM). I'm using LB in the first environment (Environment 1) and using AGW in the second environment (Environment 2).
Problem: The issue that I'm having is, environment 1 API request response time is faster than the environment 2. Below is the screenshot of the browser inspect window for the same request.
According to the timing tab, environment 1 has a fast response time than environment 2.
Question: My question is whether this response time difference due to using LB and AGW?
The biggest difference between Azure load balancer and Azure application gateway is that they works at the different layer at OSI Model. Azure Load Balancer is a high-performance, low-latency Layer 4 load-balancing service (inbound and outbound) for all UDP and TCP
protocols. This might make a quick request and response relatively.
Application Gateway provides application delivery controller (ADC) as
a service, offering various Layer 7 load-balancing capabilities. Use
it to optimize web farm productivity by offloading CPU-intensive SSL
termination to the gateway.
Azure Load Balancer is a high-performance, low-latency Layer 4
load-balancing service (inbound and outbound) for all UDP and TCP
protocols. It is built to handle millions of requests per second while
ensuring your solution is highly available. Azure Load Balancer is
zone-redundant, ensuring high availability across Availability Zones.
For more references:
Overview of load-balancing options in Azure
Azure — Difference between Azure Load Balancer and Application Gateway
As developers we wrote microservices on Azure Service Fabric and we can run them in Azure in some sort of PaaS concept for many customers. But some of our customers do not want to run in the cloud, as databases are on-premises and not going to be available from the outside, not even through a DMZ. It's ok, we promised to support it as Azure Service Fabric can be installed as a cluster on-premises.
We have an API-gateway microservice running inside the cluster on every virtual machine, which uses the name resolver, and requests are routed and distributed accordingly, but the API that the API gateway microservice provides is the entrance for another piece of client software which our customers use, that software runs outside of the cluster and have to send requests to the API.
I suggested to use an Load Balancer like HA-Proxy or Nginx on a seperate machine (or machines) where the client software send their requests to and then the reverse proxy would forward it to an available machine inside the cluster.
It seems that is not what our customer want, another machine as load balancer is not an option. They suggest: make the client software smarter to figure out which host to go to, in other words: we should write our own fail-over/load balancer inside the client software.
What other options do we have?
Install Network Load Balancer Feature on each of the virtual machine to give the cluster a single IP address, is this even possible? Something like https://www.poweradmin.com/blog/configuring-network-load-balancing-in-windows-server/
Suggest an API gateway outside the cluster, like KONG https://getkong.org/
Something else ?
PS: The client applications do not send many requests per second, maybe a few per minute.
Very similar problem, we have a many services and Service Fabric Cluster that runs on-premises. When it's time to use the load balancer we install IIS on the same machine where Service Fabric cluster runs. As the IIS is a good load balancer we use IIS as a reverse proxy only for API Gateway. Kestrel hosting is using for other services that communicate by HTTP. The API gateway microservice is the single entry point for all clients and has always static URI inside SF, we used that URI to configure IIS
If you do not have possibility to use IIS then look at Using nginx as HTTP load balancer
You don't need another machine just for HTTP forwarding. Just use/run it as a service on the cluster.
Did you consider using the built in Reverse Proxy of Service Fabric? This runs on all nodes, and it will forward http calls to services inside the cluster.
You can also run nginx as a guest executable or inside a Container on the cluster.
We have also faced the same situation when started working with service fabric cluster. We configured Application Gateway as Proxy but it would not provide the function like HTTP to HTTPS redirection.
For that, we configured Nginx Instead of Azure Application Gateway as Proxy to Service Fabric Application.
Currently I have a Service Fabric cluster with 2 stateless services hosting Asp Web APIs. While creating the cluster also appropriate Azure Load Balancers got created.
Now I would like to add Application Gateway in front of my cluster for various reasons like SSL offloading, url-routing etc.
I'd like to understand how to configure the Application Gateway correctly. I see 2 options, not sure which one is valid:
Application Gateway replaces the existing Load Balancer and points directly to SF services hosting WebApi
I keep existing LB configuration and Application Gateway points to this LB (seems like 1 LB solution too many)
Which one is correct? Any advise how to configure?
Approach 2 is what we are using, We have kept the load balancer and that is routing any request received from the Application Gateway. We found this to be easiest and simplest choice, as this involves minimum changes to be done in Application Gateway.
Your two web api's can run on every node in the VM scale set. The Azure Load Balancer is used to distribute traffic over those nodes. Targeting a single service on a single node will reduce scalability and fault tolerance.
You could use the App Gateway to translate incoming request to different ports on the Load Balancer. (E.g. direct traffic to API 1 #url ~/1/ and API 2 #url ~/2/)
Favor using load balancing rules (using all nodes) over NAT redirections (to single nodes). This way you'll have a performant, reliable system.
Solution 2 would also provide possibly to create VPN connection e.g to manage your cluster. Then no need to expose management endpoint to the public. Internal lb also brings on additional features to utilize in the future.
I would go with your first option and to implement it create / modify your ARM template so that it doesn't contain the load balancer and instead contains the application gateway.
Here is a link to the quick starts for ARM templates which you can use. There isn't an out of the box example for service fabric with a gateway but it will give you a great starting place.
link
I'm trying to add web app endpoints from the same location, to an azure traffic manager, when I try to do this, it tells me that App Service will use load balancing to do this for me, when we apps are in the same location.
My understanding is that load balancing is for distributing requests between multiple VMs on one web app. The plan was to use out single DNS and allow traffic manager to determine which endpoint to go to using round-robin or failover. How will load balancing know to direct to one of the web apps from this single address?
Azure Web Apps already have built in load balancing between instances within the web app. So for example if you have a web app with 10 instances under the endpoint: tester.azurewebsites.net, Azure load balances appropriately across those instances.
When you bring in traffic manager, that is looking for different endpoints to facilitate between. Incoming requests will be routed based on proximity to endpoints it is managing, load and if the endpoint is available. Traffic Manager takes care of all of those complexities for you.
This allows you to have a single endpoint myapp.azurewebsites.net; which may route to myapp-west.azurewebsites.net and myapp-east.azurewebsites.net. That routing as I indicated is based on proximity, load and availability.
How it actually works is the magic sauce of Azure Traffic Manager. I use it in production and it has been working very well for me. I primarily use it for routing based on proximity, and have yet to experience a failure on a web app to test a production failover reroute.
Hope that helps!