I am trying to create a online cloud based IDE just like replit.com. Whenever user creates a new project, I spin a docker container on AWS ECS Fargate.
I also generate a unique URL for each project. FOr example: project1.example.com, project2.example.com. To achive this I have my own DNS that maps URL to an IP address stored in my DB.
The problem is that I have a SSL certificate (Let's Encrypt) which is valid for *.example.com (Entire Subdomain) but how to proxy the SSL certitficate.
Do I need to put the copy of SSL certificate in each docker instance or is there somthing else I can do?
Lets say my DNS is running on EC2 machine on ns1.example.com
project1.example.com --> docker instance
project2.example.com --> docker instance
How to manage single SSL certificate to be used in all docker instances that are dynamically created and destroyed.
Thanks in advance
I tried to make an nginx proxy server that accepts all requests from *.example.com and proxy_pass to docker instance. but I am not able to achive this as I can't dynamically fetch IP address in nginx.
Your can build base docker image from nginx image, which contain ssl certificats and usage this image for up docker container. Also look on 'portworx' it shared storage driver for docker and kubernetes.
Related
I have an ECS cluster in which I run a task with many container. Three of them need to be reached from the internet. These container are exposed on port 80, 8080 and 8880 of the cluster's ec2 instance.
I have a DNS name registered (say example.com), and I can create a CNAME record that points to the ec2 dns name, but if I do so, the app will be reachable as
example.com:80
example.com:8080
example.com:8880
Instead what I would like to do is to reach the three container like this:
app1.example.com (instead of example.com:80)
app2.example.com (instead of example.com:8080)
app3.example.com (instead of example.com:8880)
I can't do it with the DNS CNAME because is not possible with CNAME to redirect to specific ports.
I hope the question make sense.
Any suggestion from anyone would be appreciate.
Thanks in advance!
You will need to place an AWS Application Load Balancer in front of the ECS containers in order to accomplish this. You would have 3 different target groups (one for each container) and configure the Load Balancer to use host-based routing to send the traffic to the appropriate target group/container.
I apologize for naive question. I am using my personal docker registry. I need to disable global docker registry. How to list the configure remote registries and disable one of them?
I searched on google but did not find any solution.
Docker daemon can connect with any docker registry in the world, if the registry has a valid HTTPS certificate. There is no "list", just like there is no list of like all web pages in the world.
disable one of them?
Setup a firewall, remove the dns names, setup a https proxy that filters the traffic, etc. How to block a Docker registry?
I'm new in AWS and EC2, I created a very simple hello world node.js application and dockerized it and I'm able to access the api route from within the container in my local machine, then I deployed the docker image to the docker hub and pulled that image from an EC2 instance and ran the image the docker logs shows that container is running fine.
Then from the EC2 instance information I get the
IPv4 Public IP xx.xxx.x.xxx
From the browser I try going to http://xx.xxx.x.xxx:8080/
but it times out and can't be reached, I'm expecting to see the hello world.
wonder if I'm missing any thing?
You probably haven't whitelisted your IP! AWS resources by default will block unknown incoming requests. To allow your machine to access to the EC2 you have to add it's IP address to your EC2's security group.
AWS has come up with a new service AWS Certificate Manager. One thing I got from the description is that if we are using this service we don't have to pay for the certificate anymore.
They are providing certificates for Elastic Load Balancer (ELB) and CloudFront, but I didn't find EC2 anywhere.
Is there any way to use the certificate with EC2?
Q: Can I use certificates on Amazon EC2 instances or on my own servers?
No. At this time, certificates provided by ACM can only be used with specific AWS services.
Q: With which AWS services can I use certificates provided by ACM?
You can use ACM with the following AWS services:
• Elastic Load Balancing
• Amazon CloudFront
• AWS Elastic Beanstalk
• Amazon API Gateway
https://aws.amazon.com/certificate-manager/faqs/
You can't install the certificates created by Amazon Certificate Manager (ACM) on resources you have direct low-level access to, like EC2 or servers outside of AWS, because you aren't provided with access to the private keys. These certs can only be deployed on resources managed by the AWS infrastructure -- ELB and CloudFront -- because the AWS infrastructure holds the only copies of the private keys for the certificates that it generates, and maintains them under tight security with auditable internal access controls.
You'd have to have your EC2 machines listening behind CloudFront or ELB (or both, cascaded, would also work) in order to use these certs for content coming from EC2... because you can't install these certs directly on EC2 machines.
No, you cannot use aws certificate manager for deploying certs on EC2. The certificate manager certs can only be deployed against cloudfront and elastic load balancer. Inoredr to use it on ec2, you need to put elb on top of ec2, so that request from client to load balancer will be https protected and from elb to ec2 webserver will be on http.
If you are using AWS ACM Cert for internal purpose only then you could probably use AWS ACM Private CA to issue the certs.(I think you can use it for public/external traffic purpose as well if your root CA is publicly trusted CA).
https://docs.aws.amazon.com/acm-pca/latest/userguide/PcaGetStarted.html
During Application/EC2/Container startup, set a step to export your ACM Private CA issued Cert/Private Key to your destination and start referring that for serving the traffic.
https://docs.aws.amazon.com/cli/latest/reference/acm/export-certificate.html
One good thing is, you can control who can call export cert feature using IAM Role so not everyone can download private key of the cert.
One downside with this is, private CA is expensive AWS service($400/month).
https://aws.amazon.com/certificate-manager/pricing/
Adding to the comments above, you can't use the AWS Certificate Manager for this, but you can add a Let's Encrypt certificate to ec2 on a Windows server running IIS and it's pretty easy:
Associate an elastic ip with your ec2 instance.
Make sure you have a registered domain. You can't use ec2----------.us-east-1.compute.amazonaws.com type names that come with your instance.
Through your domain provider's DNS settings have your domain point to your Elastic IP.
Connect to your ec2 instance and add your domain name to the site bindings.
Go to https://github.com/PKISharp/win-acme/releases
Look under assets, and use the latest version (win-acme.v2.0.10.444.zip for example). This is the only assets folder that you need.
Unzip the folder, open the terminal as administrator, and cd into the unzipped folder.
Run wacs.exe and follow the prompts.
Make sure the security group assigned to your instance allows traffic (at the very least your own IP) through the HTTPS port you chose in IIS; this is port 443 by default.
I found the links below helpful when I was figuring this out. Here is a video using an earlier release if you need more help, but it's the same idea.
https://www.youtube.com/watch?v=fq5OUOjumuM
Also this article might be helpful:
https://weblog.west-wind.com/posts/2016/feb/22/using-lets-encrypt-with-iis-on-windows
My Problem is as below.
I have a domain registered with GoDaddy [ assume : mytest.com ]
I have a EC2 ALM instance at AWS
I have elastic IP associated with my Instance [ assume : 111.222.333.444]
I have developed a nodejs application and deployed in instance
That node application is running at port 8181
Application is accesable using http://111.222.333.444:8181/
I want to configure my domain so that http://mytest.com will point to http://111.222.333.444:8181/
I have few knowledge to configure using Route53 also but not able to solve the mapping with port 8181.
Need kind help to resolve this issue
Thanks in advance
Note : For time being i am not looking for AWS PaaS Elastic Beanstalk for node js deployment
Ajoy
You actually have to do several things to make your thing work.
1. Change DNS nameservers to point to your Route53 Hosted Zone
Since you registered your hostname with GoDaddy, and you want to use Route53 as your DNS, you have to do several things.
First, go to Route53 and create a hosted zone for your domain. For demonstrative purposes, I will be using stackoverflowexample.com as my domain.
After you've created your hosted zone, click into it and you should be presented with the NS and SOA records. The NS record lists 4 unique nameservers that you will need to configure with GoDaddy.
The image below is an example of the NS record set (AWS Route53 calls it the delegation set).
Then you will have to go to your registrar and follow their directions to change nameservers for your domain, using the 4 nameservers Route 53 assigned you as your custom nameserver.
Then create a new A record in route 53 pointing at your elastic IP address. In the end your Route 53 zone will look something like this, with an A record mapping your domain to your elastic IP address.
2. Hosting/proxying your application on default HTTP port
After you have set up the DNS records (and waited a while for DNS to propagate), then you should be able to hit your server on port 8181.
But you want to hit it without a port number, so how do you do that?
DNS itself doesn't care about ports, it really just provides information about IP addresses and domain names. What you need now is to set up a proxy or other mechanism to direct traffic on your server. I will provide a few solutions.
Use nginx to reverse proxy
Since you control your own instance, you could install nginx (a web/proxy server) on the instance and configure it so that when it gets requests, it knows how to direct the traffic.
Here are generic instructions for configuring nginx for reverse proxy operation.
nginx example for Debian/Ubuntu
If you are using Debian or Ubuntu, a simple set up is as follows:
# Install nginx
sudo apt-get update && sudo apt-get install nginx
# By default, nginx runs a default site on port 80 you don't care about
# This removes the symlink for the default nginx site
sudo rm /etc/nginx/sites-enabled/default
Then you will want to create your own configuration in sites-available.
sudo vim /etc/nginx/sites-available/mainsite
The contents of your file will look probably like this:
server {
listen 80;
server_name stackoverflowexample.com;
location / {
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $http_host;
# Proxy all requests to the NodeJS app on port 8181
proxy_pass http://localhost:8181;
}
}
After you're done, execute the next two steps
# symlink your new nginx config to sites-enabled, which nginx
# automatically discovers and loads.
sudo ln -s /etc/nginx/sites-available/mainsite /etc/nginx/sites-enabled/mainsite
# Reload nginx configuration
sudo service nginx reload
Now try hitting your site without the port - you should reach your application.
Use Elastic Load Balancer
AWS offers a load balancer service (at an additional cost of ~$20 USD per load balancer per month) which lets you configure an HTTP and/or HTTPS load balancer for your application, allowing you to map incoming request ports to ports on your instance.
It's also automatically provided to you if you use Elastic Beanstalk.
They have their own tutorial which should help.
Configure your application to run on port 80
I do not do this myself and this is only useful if your instance will only ever host one app that controls all its own routing.
You could run your Node app directly on port 80 without needing a reverse proxy in front. If you go down this route, I'd recommend not allowing the service to run as root and instead configuring something like authbind to allow non-privileged access to port 80.
I will defer to other answers like this SuperUser one if you want to bind your service to port 80.
Additional Notes
You'll note that I excluded configuring HTTPS. That requires more steps like provisioning an SSL cert, though AWS or LetsEncrypt provide them for free now. Configuration also differs if you're using an Elastic Loadbalancer or a server on your instance, or if you're exposing your app directly.
I suggested nginx, and I would still recommend it in almost all general cases, but there are multiple different servers you could use to act as a reverse proxy. Other popular ones are haproxy or apache httpd with mod_proxy.
You can use any AWS public gateway provider such as EC2, ELB, S3 to serve your website or server to the public domain you have on GoDaddy. The AWS usually charges about .51 USD per month to maintain this zone entry as well.
the following steps set up your domain.
Obtain the Elastic IP or EC2 IP for the website or service.
Create a Hosted Zone on Route53.
Update the domain name records on Hosted Zone.
Update name servers on GoDaddy for discovery.
Check this tutorial
For those who are only looking to route their domain to ec2 instance (have already running ec2 instance and having domain name).
1.) Go to: https://dcc.godaddy.com/domains and select your domain.
2.) Click on Manage DNS and edit Record Type A.
3.) Enter your ec2 instance public ip in Points to textbox. Save. Done.