Whenever I try to redeploy my load balanced service in aws (via copilot) I keep getting health check failures (502 bad gateway) here's the error message:
(service my-app-my-env-my-service-Service-n6SienH8zSJt) (port 3000) is unhealthy in
(target-group arn:aws:elasticloadbalancing:us-east-1:[my target grouo]) due to (reason Health checks failed).
I have a cluster (ECS) with two services (one backend service working totally fine, and then one load balanced service that's causing the issues) that each run one task (Fargate). The load balanced service is a meteor/node app which is listening on port 3000.
The Elastic Load Balancer (application) is listening on port 80 and it should be forwarding traffic to a target group for the service mentioned above which should be listening on port 3000.
This target group for the load balanced service has:
Target type: IP
IP address type: IPv4
Protocol: Port -- HTTP:3000
Protocol Version: HTTP1
The targets for this group have their own IP addresses with port 3000.
target type: ip address since I use fargate and not EC2 for my tasks. So when a task is turning on, I correctly see the private IP of the task registering into the target group.
A few notes:
The server is launching correctly. I'm receiving logs that indicate a healthy server and no errors are showing up
I have a /_health route which I set up and is working locally (getting my 200 status with a curl request to localhost:3000/_health). I'm pretty convinced that NO routes are working because i changed my app to render a static page regardless of the route and still having issues connecting. This makes me think the issues lies between the load balancer and the service
Been stuck on this for a week so if anyone knows what I'm missing that would be particularly helpful! I'm happy to share more information about my cluster if that will help! Thanks in advance :)
What is the mapping of the health check route in your copilot manifest?
By default copilot configures health checks to target '/'.
I feel pretty silly about this but pretty sure I found the solution. While I configured the port: 3000 correctly on the image in the manifest.yml, I needed an additional environment variable called PORT: 3000 in the variables for the manifest. This seemed to do the trick... like I said silly mistake!
Related
I'm trying to set-up a managed instance group. The instance group is to run an image in which a Node.js server is listening on port 8080.
Setting up the template and the instance group works. I can SSH into the machine. docker ps gives me the expected results. And curl localhost:8080/v1/status results in OK.
Now I tried to add a health check - and that doesn't work. It always results in a TIMEOUT. According to the logs, no request of the health check reaches the Node.js server.
The logs of the health checker are not really helpful.
I triple-checked the settings in the health check. Path and port number are corrected.
I have looked at other projects with a similar setup. The only difference I spotted is the port. Those are using port 80. However, I cannot see why this should be an issue. If the port the server listens to and the port the health checker checks on are equal - then there should be no issue. But maybe I'm missing something here.
How can I debug this further? How can I see why the health check is failing? Can I add some port mapping from 8080 to 80 anywhere?
you need to add a firewall rule for this ip ranges and add your port/instance group as a destination
Health check source ranges for gcp
35.191.0.0/16
130.211.0.0/22
for more information please have a look at this doc
https://cloud.google.com/load-balancing/docs/firewall-rules
I have an Application Load Balancer that is in HTTPS and will make a request to my EC2 instance. I have configured the ALB listeners to HTTP/HTTPS and then I have specified the target group (for the application load balancer) as the EC2 instance.
On this EC2 instance, I have two things running: 1) A website 2) A nodejs service (on port 4000)
In the target group I have specified the relevant EC2 instance twice. (once with port number 80 and another with port number 4000). On port 4000, the health check seems to be failing (even though the service is running on port 4000 only).
When I make the htttps request to my website, the response is fine. However, when I make the https request to my nodejs service (running on port number 4000), it is giving connection refused error.
A target group should only be forwarding to one application. So you would want a separate target group for your web site and your node js service. Then using ALB use advanced routing to determine which to load.
An example of above would be:
Path of /api/* forwards to node js target group
Default forwards to website.
Secondly your health check must pass on your node js app, its currently returning a 404 indicating that the path you are host checking against is not available.
Finally make sure you connect based on the listener configuration (http/https and port), not the port from the target group.
I am able to access my app inside the ec2 instance using elinks but when I try to do from the browser it says "This site can’t be reached".
I am trying to run a node app on port 3000
my netstat
netstat
my ec2 security group inbound rules
after going through other stackoverflow tickets I figured that there are two levels of checks
EC2 security groups
EC2 instance firewall
so although I added the port 3000 on security group inbound rules still I had to puncture the instance firewall as well. Port 80 is open by default
I'm actually doing the same thing, and in order to do so, you need to give it a public IPv4 address and also run a web server from it. At least that is the conclusion I reached this morning.
Few days ago I wanted to launch my own Agario server. I assumed not to spend any money on hosting/vps etc. After a long search I found Bluemix PaaS, I put open source Agario clone Ogar (https://github.com/OgarProject/Ogar) in them and server has already started in 1523 port, but when i try to connect to this server via agario site ( connect("ws://appname.eu-gb.mybluemix.net:1523") ) I can't connect. I tried also other ways e.g. prepared agar.io link (agar.io?ip=appname.eu-gb.mybluemix.net), but nothing worked.
Has anyone met similar problem?
Inbound traffic is only on standard HTTP/HTTPS ports 80/443. Bluemix will tell your application what port to listen on with the VCAP_APP_PORT env variable. Inbound requests are then mapped to that port. So, once you bind to the VCAP_APP_PORT port, you should be able to connect to : ws://appname.eu-gb.mybluemix.net
i am running a nodejs tcp app at my aws linux ec2 instance . the basic code is given below
var net = require('net');
net.createServer(function(socket){
socket.write('hello\n');
socket.on('data', function(data){
socket.write(data.toString().toUpperCase())
});
}).listen(8080);
and its run like charm, but i wanted to run this same app at aws beanstalk (just to get the benefit of auto scaling). ya ya i am not aws ninja. by the way to get the public ip at beanstalk i use aws VPC.
beanstalk app connect to VPC = checked.
VPC 8080 port open = checked .
change hard coded port 8080 to process.env.PORT = checked.
but if i ping anything at port 8080 it does not return 'hello' from the application. what i am missing ?
Your application is not implementing HTTP. ElasticBeanstalk by default is going to configure the Elastic Load Balancer (ELB) to act as an HTTP load balancer. This means your instance is not healthy and is not being put into service by the ELB and the ELB itself would also be rejecting the non-HTTP request.
Important note: While it would be possible to modify ElasticBeanstalk to work for your use case, you are going to be using it in a non-standard way so there will be some risks. If you are regularly creating and deleting environments using CloudFormation or the API then you will likely run into a lot of headaches.
If you are going to just create an environment and leave it running then I suggest you take the following steps.
First off, ElasticBeanstalk's nodejs configuration is going to configure an Nginx server on the EC2 instance, since you are using TCP you will want to bypass this entirely. This can be done by re-configuring the ELB and security groups. It would be easiest to just leave Nginx running, it just will not be used, just make sure it is not on the same port as nodejs.
By default the ELB configuration will look like this:
The step you missed was updating the ELB to use TCP load balancing on the appropriate ports. You can go into the EC2 web console under Load Balancers and update the load balancer configuration for the already created Beanstalk to look like this:
You will also want to modify the health check of the load balancer to be on the correct port:
Last, double check to make sure the security groups for both the load balancer and EC2 instances allow the appropriate ports to be accessed. The last thing to check, but you already mentioned you looked, is that your VPC's NACLs also allow the appropriate ports to be accessed.