I'm trying to load balance a nodejs Server sent event backend and I need to know if there is a way to distribute the new connections to the instances with the least connected clients. The problem I have is when scaling up, the routing continues sending new connections to the already saturated instance and since the connections are long lived this simply won't work.
What options do I have for horizontal scaling long lived connections?
It looks like you want a Load Balancer that can provide both "sticky sessions" and use the "least connection" instead of "round-robin" policy. Unfortunately, NGINX cannot provide this.
HAProxy (High Availability Proxy) allows for this:
backend bk_myapp
cookie MyAPP insert indirect nocache
balance leastconn
server srv1 10.0.0.1:80 check cookie srv1
server srv2 10.0.0.2:80 check cookie srv2
If you need ELB functionality and want to roll it all manually, take a look at this guide.
You might also want to make sure classic AWS ELB "sticky session" configuration or the newer ALB "sticky session" option does not meet your needs. ELB normally sends connection to upstream server with the least "load", and when combining with sticky sessions might be enough.
Since you are using AWS, I'd recommend Elastic Beanstalk for your Node.js application deployment. The official documentation provides good examples, like this one. Note that Beanstalk will automatically create an Elastic Load Balancer for you, which is what you're looking for.
By default, Elastic Beanstalk creates an Application Load Balancer for
your environment when you enable load balancing with the Elastic
Beanstalk console or the EB CLI. It configures the load balancer to
listen for HTTP traffic on port 80 and forward this traffic to
instances on the same port.
[...]
Note:
Your environment must be in a VPC with subnets in at least two
Availability Zones to create an Application Load Balancer. All new AWS
accounts include default VPCs that meet this requirement. If your
environment is in a VPC with subnets in only one Availability Zone, it
defaults to a Classic Load Balancer. If you don't have any subnets,
you can't enable load balancing.
Note that the configuration of a proper health check path is key to properly balance requests, as you mentioned in your question.
In a load balanced environment, Elastic Load Balancing sends a request
to each instance in an environment every 10 seconds to confirm that
instances are healthy. By default, the load balancer is configured to
open a TCP connection on port 80. If the instance acknowledges the
connection, it is considered healthy.
You can choose to override this setting by specifying an existing
resource in your application. If you specify a path, such as /health,
the health check URL is set to HTTP:80/health. The health check URL
should be set to a path that is always served by your application. If
it is set to a static page that is served or cached by the web server
in front of your application, health checks will not reveal issues
with the application server or web container.
EDIT: If you're looking for sticky sessions, as I described in the comments, follow the steps provided in this guide:
To enable sticky sessions using the console
Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.
On the navigation pane, under LOAD BALANCING, choose Target Groups.
Select the target group.
On the Description tab, choose Edit attributes.
On the Edit attributes page, do the following:
a. Select Enable load balancer generated cookie stickiness.
b. For Stickiness duration, specify a value between 1 second and 7 days.
c. Choose Save.
Related
We are having difficulties choosing a load balancing solution (Load Balancer, Application Gateway, Traffic Manager, Front Door) for IIS websites on Azure VMs. The simple use case when there are 2 identical sites is covered well – just use Azure Load Balancer or Application Gateway. However, in cases when we would like to update websites and test those updates, we encounter limitation of load balancing solutions.
For example, if we would like to update IIS websites on VM1 and test those updates, the strategy would be:
Point a load balancer to VM2.
Update IIS website on VM1
Test the changes
If all tests are passed then point the load balancer to VM1 only, while we update VM2.
Point the load balancer to both VMs
We would like to know what is the best solution for directing traffic to only one VM. So far, we only see one option – removing a VM from backend address pool then returning it back and repeating the process for other VMs. Surely, there must be a better way to direct 100% of traffic to only one (or to specific VMs), right?
Update:
We ended up blocking the connection between VMs and Load Balancer by creating Network Security Group rule with Deny action on Service Tag Load Balancer. Once we want that particular VM to be accessible again we switch the NSG rule from Deny to Allow.
The downside of this approach is that it takes 1-3 minutes for the changes to take an effect. Continuous Delivery with Azure Load Balancer
If anybody can think of a faster (or instantaneous) solution for this, please let me know.
Without any Azure specifics, the usual pattern is to point a load balancer to a /status endpoint of your process, and to design the endpoint behavior according to your needs, eg:
When a service is first deployed its status is 'pending"
When you deem it healthy, eg all tests pass, do a POST /status to update it
The service then returns status 'ok'
Meanwhile the load balancer polls the /status endpoint every minute and knows to mark down / exclude forwarding for any servers not in the 'ok' state.
Some load balancers / gateways may work best with HTTP status codes whereas others may be able to read response text from the status endpoint. Pretty much all of them will support this general behavior though - you should not need an expensive solution.
We ended up blocking connection between VMs and Load Balancer by creating Network Security Group rule with Deny action on Service Tag Load Balancer. Once we want that particular VM to be accessible again we switch the NSG rule from Deny to Allow.
The downside of this approach is that it takes 1-3 minutes for the changes to take an effect. Continuous Delivery with Azure Load Balancer
If anybody can think of a faster (or instantaneous) solution for this, please let me know.
I had exactly the same requirement in an Azure environment which I built a few years ago. Azure Front Door didn't exist, and I had looked into using the Azure API to automate the process of adding and removing backend servers the way you described. It worked sometimes, but I found the Azure API was unreliable (lots of 503s reconfiguring the load balancer) and very slow to divert traffic to/from servers as I added or removed them from my cluster.
The solution that follows probably won't be well received if you are looking for an answer which purely relies upon Azure resources, but this is what I devised:
I configured an Azure load balancer with the simplest possible HTTP and HTTPS round-robin load balancing of requests on my external IP to two small Azure VMs running Debian with HAProxy. I then configured each HAProxy VM with backends for the actual IIS servers. I configured the two HAProxy VMs in an availability set such that Microsoft should not ever reboot them simultaneously for maintenance.
HAProxy is an excellent and very robust load balancer, and it supports nearly every imaginable load balancing scenario, and crucially for your question, it also supports listening on a socket to control the status of the backends. I configured the following in the global section of my haproxy.cfg:
global
log /dev/log local0
log /dev/log local1 notice
chroot /var/lib/haproxy
stats socket /run/haproxy/admin.sock mode 660 level admin
stats socket ipv4#192.168.95.100:9001 level admin
stats timeout 30s
user haproxy
group haproxy
daemon
In my example, 192.168.95.100 is the first HAProxy VM, and 192.168.95.101 is the second. On the second server, these lines would be identical except for its internal IP.
Let's say you have an HAProxy frontend and backend for your HTTPS traffic to two web servers, ws1pro and ws2pro with the IPs 192.168.95.10 and 192.168.95.11 respectively. For simplicity sake, I'll assume we don't need to worry about HTTP session state differences across the two servers (e.g. Out-of-Process session state) so we just divert HTTPS connections to one node or the other:
listen stats
bind *:8080
mode http
stats enable
stats refresh 10s
stats show-desc Load Balancer
stats show-legends
stats uri /
frontend www_https
bind *:443
mode tcp
option tcplog
default_backend backend_https
backend backend_https
mode tcp
balance roundrobin
server ws1pro 192.168.95.10:443 check inter 5s
server ws2pro 192.168.95.11:443 check inter 5s
With the configuration above, since both HAProxy VMs are listening for admin commands on port 9001, and the Azure load balancer is sending the client's requests to either VM, we need to tell both servers to disable the same backend.
I used Socat to send the cluster control commands. You could do this from a Linux VM, but there is also a Windows version of Socat, and I used the Windows version in a set of really simple batch files. The cluster control commands would actually be the same in BASH.
stop_ws1pro.bat:
echo disable server backend_https/ws1pro | socat - TCP4:192.168.95.100:9001
echo disable server backend_https/ws1pro | socat - TCP4:192.168.95.101:9001
start_ws1pro.bat:
echo enable server backend_https/ws1pro | socat - TCP4:192.168.95.100:9001
echo enable server backend_https/ws1pro | socat - TCP4:192.168.95.101:9001
These admin commands execute almost instantly. Since the HAProxy configuration above enables the stats page, you should be able to watch the status change happen on the stats page as soon as it refreshes. The stats page will show the connections or sessions draining from the server you disabled over to the remaining enabled servers when you disable a backend, and then show them returning to the server once it is enabled again.
I need help in this scenario
My an ec2 instance is receiving requests, I don't want on client end to change server path based on few requests (especially on chat)
My first ec2 instance bypass requests based on few path patterns to my other created instance. (In other words, I want to redirect traffic from first to second)
Is there any way to fulfill above scenario.
You can use AWS CloudFront as a proxy for your use case where you can plugin the two EC2 instances behind CloudFront as Origins and Add behavior rules (Path rules) to switch traffic to one or the other.
Your client will send requests only to the CloudFront URL(Or DNS mapped through Route53) and won't be knowing about the EC2 instances behind. This approach will work, if your EC2 instances are publicly accessible but will be cost effective and reduce the load on your services, if you happen to cache content.
Alternative approach is to use an Application Load Balancer with path based routing configuration.
Following tutorial will guide you through the steps.
Tutorial: Use Path-Based Routing with Your Application Load Balancer
I have create an architecture on AWS (hope it should not be wrong) by using the ELB, autoscaling, RDS and one node ec2 instance outside the ELB. Now I am not getting, that, how I can implement the SSL on this architecture.
Let me explain this in brief:
I have created one Classic Load Balancer.
Created on autoscaling group.
Assign instances to autoscaling group.
And lastly I have created one Instance that I am using for the node and this is outside the Load Balancer and Autoscaling group.
Now when I have implemented the SSL to my Load Balancer, the inner instances are communicating with the node instance on the HTTP request and because the node instance is outside the load balancer so the request is getting blocked.
Can someone please help me to implement the SSL for this architecture.
Sorry if you got confused with my architecture, if there is any other best architecture could be possible then please let me know I can change my architecture.
Thanks,
When you have static content, your best bet is to serve it from Cloudfront using an S3 bucket as its origin.
About SSL, you could set the SSL at your ELB level, follow the documentation .
Your ELB listens on two ports: 80 and 443 and communicates with your ASG instances only using their open port 80.
So when secure requests come to the ELB, it forwards them to your server ( EC2 in the ASG ). Then, your server, listening on port 80, receives the request; if the request have the X-FORWARDED-PROTO HTTPS, the server does nothing, otherwise it sets it and forward/rewrite the URL to be a secure one and the process restart.
I hope this helps and be careful of ERR_TOO_MANY_REDIRECTS
Have you considered using an Application Load Balancer with two target groups and a listener rule?
If the single EC2 instance is just hosting static content, and is serving content on a common path (e.g. /static), then everything can sit behind a shared load balancer with one common certificate that you can configure with ACM.
"because the node instance is outside the load balancer so the request
is getting blocked."
If they're in the same VPC you should check the security group that you've assigned to your instances. Specifically you're going to want to allow connections coming in to the ports 443 and/or 80 on the stand-alone instance to be accessible from the security group assigned to the load balancer instances - let's call those 'sg-load_balancer' (check your AWS Console to see what the actual security group id is).
To check this - select the security group for the stand-alone instance, notice the tabs at the bottom of the page. Click on the 'Inbound' tab. You should see a set of rules... You'll want to make sure there's one for HTTP and/or HTTPS and in the 'Source' instead of putting the IP address put the security group for the load balancer instances -- it'll start with sg- and the console will give you a dropdown to show you valid entries.
If you don't see the security group for the load balancer instances there's a good chance they're not in the same VPC. To check - bring up the console and look for the VPC Id on each node. That'll start with vpc_. These should be the same. If not you'll have to setup rules and routing tables to allow traffic between them... That's a bit more involved, take a look at a similar problem to get some ideas on how to solve that problem: Allowing Amazon VPC A to get to a new private subnet on VPC B?
I fear that this might be a programming question, but I am also hopeful that it is common enough that you might have some suggestions.
I am moving to a fail-over environment using AWS elastic load balancers to direct the traffic to the EC2 instances. Currently, I have set up the ELB with a single EC2 instance behind it. You will see why in a moment. This is still in test mode, although it is delivering content to my customers using this ELB -> EC2 path.
In each of my production environments (I have two) I have an AWS certificate on the load balancer and a privately acquired security certificate on the EC2 instance. The load balancer listeners are configured to send traffic received on port 443 to the secure port (443) on the EC2 instance. This is working; however, as I scale up to more EC2 instances behind the load balancer, I have to buy a security certificate for each of these EC2 instances.
Using a recommendation that was proposed to me, I have set up a test environment with a new load balancer and its configured EC2 server. This ELB server sends messages received on its port 443 to port 80 on the EC2 system. I am told that this is the way it should be done - limit encryption/decryption to the load balancer and use unencrypted communication between the load balancer and its instances.
Finally, here is my problem. The HTML pages being served by this application use relative references to the embedded scripts and other artifacts within each page. When the request reaches the EC2 instance (the application server) it has been demoted to HTTP, regardless of what it was originally.This means that the references to these embedded artifacts are rendered as insecure (HTTP). Because the original page reference was secure (HTTPS), the browser refuses to load these insecure resources.
I am already using the header X-Forwarded-Proto within the application to determine if the original request at the load balancer was HTTP or HTTPS. I am hoping against hope that there is some parameter in the EC2 instance that tells it to render relative reference in accordance to the received X-Forwarded-Proto header. Barring that, do you have any ideas about how others have solved this problem?
Thank you for your time and consideration.
First of all it is the right way to go by having the SSL termination at ELB/ALB and then having a security group assigned to EC2 that only accepts traffic from ELB/ALB.
However responding with https urls based on the X-Forwarded-Proto request headers or based on custom configuration, needs to be handle in your application code or webserver.
My application requirement :
""The URL of your web form, to be displayed in a frame in the Worker's web browser. This URL must use the HTTPS protocol.""
My AWS EC2 instance has node js running on it. For some reason I am having issues running it as a production server serve -s build
But when I npm start in my project folder it runs a development server on port 3000 and I can access it via http://ec2----------.compute-1.amazonaws.com:3000/
But this does not work with https. Is there a way I can access the same url using https? Something like :
https://ec2----------.compute-1.amazonaws.com:3000/
The ways that I have looked so far : Reverse Proxy and Nginx.
But could not understand it well.
If you use an elastic load balancer in front of the EC2 instance then AWS provides a very easy way to get HTTPS working. If you want to access the instance directly you will need to configure HTTPS in your node.js or use an HTTPS service to proxy the traffic to your node.js app.
Step 1 : Choosing the Load Balancer
The 2 choices when you create a load balancer :
Application Load Balancer : If your application is running on particular ports or in dev mode or you need path-based routing. It is a good option in terms of the routing decision are done at the application layer. It can only listen from HTTP and HTTPS.
Classical Load Balancer : If you need to take the routing decisions right from the transport layer. You may choose one.
I will continue with the Application Load Balancer, although most of the stages are same.
Step 2 : Configuring the Load Balancer
Simple and quick configuration :
Name : Name your load balancer.
Scheme :
internet facing : choose this if you want the requests from the client over the internet.
internal : choose this if you want the requests from the client using a private IP
address.
IP Address Type : ipv4
Listener
A listener is a process that checks for connection requests, using the protocol and port that you configured.
There can be only two listeners in the application load balancer, which are :
HTTP on port 80
HTTPS on port 443
Availability Zones
Load balancer's main job is to maintain traffic across different areas and regions. There are multiple availability zones in one region. These can be imagined as placing multiple servers in us-east These availability zones each have a separate subnet. But only one subnet can be selected for a particular zone.
You need to select at least 2 such availability zones having distinct subnets. This basically helps the load balancer to balance the load on at least 2 servers.
Step 3 : Configure Security Settings and Add Instance
Configuring security settings consists of specifying the certificates if you have selected to listen to https in the previous step. Since you selected the https listener, AWS needs to use the certificate. You can learn how to get a certificate from AWS Certificate Manager. Over here you have to select :
Certificate Type : Choose an existing certificate from AWS Certificate Manager (ACM)
Certificate Name : It pops up the certificate name in the drop down list.
Select the latest security policy
Security Policy : ELBSecurity-2016-08
Select the existing security group made for your instance.
Step 4 : Target Groups
Create a target group. Name it according to what it listens and where it targets.
You have to mention a path and a port where the listener targets the traffic to.
Step 5 : Deploy
After you review the settings, deploy and create your load balancer. This will do all the cleansing and management. It is like hiring a manager for you server traffic. You can go and meditate now for some time.
The load balancer will take almost a minute to be up and about. After the load balancer is active. Copy the DNS link of the load balancer on the main load balancer dashboard since we will need it in the next step. It will look something like this :
load-balancer-name-xxxxxxxxxx.us-east-x.xxx.amazonaws.com (A Record)
Step 6 : Map your domain name to the Load Balancer
Provides a reliable and cost-effective way to route visitors to websites by translating domain names (such as www.example.com) into the numeric IP addresses (such as 192.0.2.1) that computers use to connect to each other. AWS assigns URLs to your resources, such as load balancers. However, you might want a URL that is easy for users to remember. For example, you can map your domain name to a load balancer.
Go to Route 53 and select the hosted zone and the record set for your domain name.
You need to create a new record :
1) Leave the domain name blank.
2) Select Yes for Alias.
3) Paste the DNS link for the Load Balancer in the Alias Target.
4) Create.
This step is basically a transfer of risk. It routes the domain name to the dns of the load balancer. Hence solves our purpose of handling traffic. The rest of the job is handled by the ELB, which translates its statistics into the health reports, based on which you can create and replace more instances.
Have a great one!
Citation : https://sites.google.com/gwmail.gwu.edu/aws-tools/aws-elastic-load-balancer?authuser=0