I created a micro instance on EC2 that has my node.js based web application along with nginx (created a reverse front-end proxy so that my app can be on port 3000, and I have that routed to my localhost with nginx).
I also installed mongodb on this same (micro) instance, however, I was reading last night from the MongoDB docs on the way to deploy MongoDB on EC2 here. The difference between this method and my initial method is:
This method uses the ec2 command line tools to create new instances
When I use the ec2 command line tools to replicate the instructions, it tells me that it's ignoring one of the flags, so I think that the following command is outdated:
$ ec2-run-instances ami-05355a6c -t m1.large -g [SECURITY-GROUP] -k [KEY-PAIR] -b "/dev/sdf=:200:false:io1:1000" -b "/dev/sdg=:25:false:io1:250" -b "/dev/sdh=:10:false:io1:100" --ebs-optimized true
After using the above command, and proceeding to do: sudo mkfs.ext4 /dev/sdf, the name changed on my AMI image since it doesn't live there anymore.
After running ec2-run-instances and refreshing my Amazon EC2 dashboard, it doesn't show up in my instances, but if I do sudo fdisk -l it'll show 2 mounts.
As you can see, the guide is probably a little outdated, and I'm just wondering how in the world to deploy my mongodb to EC2 on its own instance. From there, how do I get them to talk to each other too? E.g. my new mongodb instance to talk to my node.js micro instance with nginx on it.
Try to add the volumes from EC2 panel, and then attach them to an existing instance. It works for me.
The command line is
-b "/dev/xvdf=:200:false:io1:1000" -b "/dev/xvdg=:25:false:io1:250" -b "/dev/xvdh=:10:false:io1:100"
and
"/dev/xvdf=:200:false:io1:1000"
means that
1, you have to add a Provisioned IOPS (PIOPS) EBS volume.
2, volume size should be 200, and IOPS value is 1000, the available zone should be as same as your ec2 instance.
3, And xvdf is the location you add when you attach the volume to the instance.
Related
I have an app that has been successfully running on EC2 for a few years. The system is Ubuntu 18.04.2 LTS (GNU/Linux 4.15.0-1032-aws x86_64).
It's a small and simple app with low traffic. I had never made any changes to the server itself until today. I wanted to deal with the X packages can be updated. message, so I ran:
sudo apt-get update
sudo apt-get upgrade
sudo apt-get dist-upgrade
Then I ran sudo reboot. Once rebooted, the app runs perfectly. I can access it as normal via the public URL and look at things, including db (postgresql directly on the server) data with no issues or surprises.
But, when I tried to ssh into the machine again, I couldn't. I do ssh -i "key.pem" -vvv ubuntu#<IP> and get:
debug1: Connecting to <IP> [<IP>] port 22.
debug1: connect to address <IP> port 22: Operation timed out
ssh: connect to host <IP> port 22: Operation timed out
No changes were made to security groups. Also, it's such a small project, that I never setup EC2 Instance Connect or anything like that.
I had the thought of launching a new EC2 and just switching the EBS volumes, thinking EBS would bring the app and data, while the instance itself would have configs and permissions.
I do not understand much about this (clearly), and was surprised to learn that the EBS volume itself seems to be the problem and hold all the cards.
I can switch EBS volumes back and forth between the two EC2 instances. At any given time, whichever one has the newest (and therefore blank) EBS volume attached at /dev/sda1 allows SSH but surely does not run the app. And, vice-versa: Whichever EC2 instance has the original EBS volume runs the app perfectly but keeps me locked out of ssh.
In this scenario, the question is: How can I make one of the EC2 instances bypass this EBS issue and make its own decision about allowing me to connect with ssh?
Or: What is the obvious and/or silly thing I'm missing here?
PS: I do have elastic IP going for all of this, so it doesn't seem like DNS would be the source of the problem.
With John Rotenstein's help, I was able to resolve this.
Here are the core steps:
Phase 1 - Attach and mount additional volume
Per John's comment, it's possible to boot the instance from the "good" volume and then attach and mount the "bad" volume after. This allowed me to explore files and look for issues.
AWS panel
Attach volume to EC2 instance as root by using /dev/sda1 for name
Start the EC2 instance
Attach the other volume after instance has booted
Terminal
SSH into the server
See root volume information:
~$ df -hT /dev/xvda1
Check for mounted volumes:
~$ lsblk
See additional volume information:
~$ df -hT /dev/xvdf1
Switch to root user:
~$ sudo su -
Make a directory to be the mount path:
~$ mkdir /addvol
Mount the additional volume to the path:
~$ mount /dev/xvdf1 /addvol
Check additional volume contents:
~$ ls -la /addvol/home/ubuntu
Now I could see and navigate the additional volume's contents, finding config files, looking at authorized_keys, file permissions, etc.
This article from AWS helped a lot to get me here.
After finally getting to this point, I could not find any problems with the keys, or permissions, etc. John pointed me to this article dealing with Ubuntu's firewall things.
Phase 2 - Dealing with the firewall
I ran some commands from the article and tried to understand how they worked.
Once I grasped it a little, I decided to use an existing reboot script I have on the volume to ensure the firewall was ok with SSH connections.
I updated my existing custom reboot script, adding the following lines:
sudo ufw allow ssh
sudo ufw allow 22
sudo ufw disable
sudo ufw --force enable
Basically it calls to allow for ssh twice, once by name and then by port. I'm a newbie on this stuff and just went for the overkill.
Then it disables and enables the firewall to ensure it runs with these news things configured.
Because sudo ufw enable requires an interaction, I chose to use sudo ufw --force enable.
Phase 3 - Testing and using it!
After the script update, I exited the server.
AWS panel:
Stop the EC2 instance
Detach one volume from the instance
Detach the other volume from the instance
Reattach the "bad" volume, this time as root
Start the EC2 instance
Terminal:
SSH into the instance - Voila!
NOTE: Before truly working 100%, my computer complained about the known_hosts thing. The server key must have changed on the update/upgrade and/or after all of the volume changes. I don't think having to confirm hosts is a big deal, so I just usually clear all of the contents in my local .ssh/known_hosts file. If you prefer to be specific, you can find the server's information on there specifically and delete only the relevant lines.
I'm using docker/elk image to display my data in kibana dashboard (Version 6.6.0) and It works pretty good. I started the service like using below command.
Docker Image git repo:
https://github.com/caas/docker-elk
Command:
sudo docker-compose up --detach
Expecting that it will run background, and did as expected. After two days the server up and running the and third day the kibana alone getting stopped. and Used below command to make it up and running.
sudo docker run -d <Docer_image_name>
It's up and running when I use docker ps command. But when I tried to hit the kibana server in chrome browser it says not reachable.
So I just used to below command to restart the service.
sudo docker-compose down
After that I can see kibana server in chrome browser which is up and running but I do see all my data is lost.
I used below URL in jenkins to collect the data.
`http://hostname:9200/ecdpipe_builds/extern`al
Any idea how can I resolve this issue?
I did not see the persistent storage configuration the image you mentioned in their GitHub docker-compose file.
This is common to lost data in case of docker container if you did not provide persistent storage configuration. so docker-compose down may cause to lost you data if there is no persistent configuration docker-compose file.
Persisting log data
In order to keep log data across container restarts, this image mounts
/var/lib/elasticsearch — which is the directory that Elasticsearch
stores its data in — as a volume.
You may however want to use a dedicated data volume to persist this
log data, for instance to facilitate back-up and restore operations.
One way to do this is to mount a Docker named volume using docker's -v
option, as in:
$ sudo docker run -p 5601:5601 -p 9200:9200 -p 5044:5044 \
-v elk-data:/var/lib/elasticsearch --name elk sebp/elk
This command mounts the named volume elk-data to
/var/lib/elasticsearch (and automatically creates the volume if it
doesn't exist; you could also pre-create it manually using docker
volume create elk-data).
So you can set these paths in your docker-compose file accordingly. Here is the link that you can check elk-docker-persisting-log-data
Use docker volume or file location as persistant space
I have built a node.js application in the docker, every time when I need to run it, I just run the docker and run the command node app.js.
I have set it up on Amazon EC2, but in a vanilla way by register and log into the Amazon EC2 instance, pull the docker image, then run and log into the docker and run the command node app.js.
Now, since Amazon has this EC2 container service, I was told that I can do these two things automatically:
EC2 runs the docker
Docker runs node app.js
The advantage of doing this is that whenever either docker is crashed or the app is crashed, both of them are crashed, therefore that EC2 can automatically run the command again and rescue them.
How can I set this function up?
It comes by default when you set up an ECS task. Make sure the task is marked as 'essential' in your task's container and that you have at least one task requested in your ECS service, and it will automatically restart a failed/crashed container for you.
There is any way to download a file from google managed VM docker?
we lost one that is in production version and I want to download it to my computer but I cant find the app path
It should be possible.
First, determine the GCE instance that runs your version. The name of the version should be part of the instance name. If your version has multiple instances, you may have to try all of them (or if your file was part of the application, any of them may work).
From the Cloud console, you can switch it from "Google managed" to self-managed.
Next, use gcloud compute ssh <instance name> to ssh to the instance.
Next, run docker ps to find the container running your application code. You should see a few side-car containers like nginx, but if you look through the names of the containers you should see one for your application.
Finally, you could docker exec -it <container id> -- bash to create a shell on the instance. Or instead of bash, perhaps run a cat command or whatever else you need to do to recover your file.
Im a trying to deploy my application using Docker and came across an issue that restarting named containers assigns a different IP to container. Maybe explaining what I am doing will better explain the issue:
Postgres runs inside a separate container named "postgres"
$ PG_ID=$(docker run --name postgres postgres/image)
My webapp container links to postgres container
$ APP_ID=$(docker run --link postgres:postgres webapp/image)
Linking postgres container image to webapp container inserts in webapp container a hosts file entry with the IP of the postgres container. This allows me to point to postgres db within my webapp using postgres:5432 (I am using Django btw). This all works well except if for some reason postgres crashes.
Before I manually stop postgres process to simulate postgres process crashing I verify IP of postgres container:
$ docker inspect --format "{{.NetworkSettings.IPAddress}}" $PG_ID
172.17.0.73
Now to simulate crash I stop postgres container:
$ docker stop $PG_ID
If now I restart postgres by using
$ docker start $PG_ID
the ip of the container changes:
$ docker inspect --format "{{.NetworkSettings.IPAddress}}" $PG_ID
172.17.0.74
Therefore the IP which points to postgres container in webapp container is no longer correct. I though that by naming container docker assigns a name to it with specific configs so that you can reliably link between containers (both network and volumes). If the IP changes this seems to defeat the purpose.
If I have to restart my webapp process each time I postgres restarts, this does not seem any better than just using a single container to run both processes. Then I can use supervisor or something similar to keep both of them running and use localhost to link between processes.
I am still new to Docker so am I doing something wrong or is this a bug in docker?
2nd UPDATE: maybe you already discovered this, but as workaround, I plan to map the service to share the database to the host interface (ej: with -p 5432:5432), and connect the webapps to the host IP (the IP of the docker0 interface: in my Ubuntu and CentOS, the IP is 172.17.42.1). If you restart the postgres container, the conteiner's IP will change, but I wil be accesible using 172.17.42.1:5432. The downside is that you are exposing that port to all the containers, and loose the fine-grained mapping that --link gives you.
--- OLD UPDATES:
CORRECTION: Docker will map 'postgres' to the container's IP in the /etc/hosts files, on the webapp container. So, in the webapp container, you can ping 'postgres', and it will be mapped to the IP.
1st UPDATE: I've seen that Docker generates and mounts /etc/hosts, /etc/resolv.conf, etc. to have always the correct information, but this does not apply when the linked container is restarted. So, I've assumed (wrongly) that Docker would update the hosts files.
-- ORIGINAL (wrong) response:
Add --hostname=postgres-db (you can use anythin, I'm using something different than 'postgres' to avoid confussion with the container name):
$ docker run --name postgres --hostname postgres-db postgres/image
Docker will map 'postgres-db' to the container's IP (check the contents of /etc/hosts on the webapp container).
This will allow you run 'ping postgres-db' from the webapp container. If the IP changes, Dockers will update /etc/hosts for you.
In the Django app, use 'postgres-db' instead of the IP (or whatever you use for --hostname of the container with PostgreSql).
Bye!
Horacio
According to https://docs.docker.com/engine/reference/commandline/run/, it should be possible to assign a static IP for your container -- at the time of container creation -- using the --ip option:
Example:
docker run -itd --ip 172.30.100.104 --name postgres postgres/image
....where 172.30.100.104 is a free IP address on a custom bridge/overlay network.
This should then retain the same IP address even if postgres container crashes/restarts.
Looks like this was released in Docker Engine v 1.10 or greater, therefore if you have a lower version, you have to upgrade first.
As of Docker 1.0 they implemented a stronger sense of linked containers. Now you can use the container instance name as if it were the host name.
Here is a link
I found a link that better describes your problem. And while that question was answered I wonder whether or not this ambassador pattern might not solve the problem... this assumes that the ambassador is more reliable than the services that link.