How to get EC2 instance private IP in a private subnet? - python-3.x

I am running a python application which is running inside a docker container using Beanstalk in a private subnet, and I want to get the Private/Local IP of the EC2 instance. Is it possible to get the Local IP address without using curl http://169.254.169.254/latest/meta-data/local-ipv4 inside a docker container.
Although I tried docker run --net=host <image_name> but still its not accessible.

You can get the local ip of a Linux instance with this command:
hostname -I | awk '{print $1}'
For EB, use .ebexentions and write a bash script in a script_name.config to run: export HOST_IP=$(hostname -I | awk '{print $1}')

Related

Docker doesn't apply changes to daemon.json

I have created the file /etc/docker/daemon.json on ubuntu with the following contents:
{
"ipv6": false
}
Afterward I rebooted the machine and docker is still looking for ipv6 addresses, giving me the following error on docker swarm init --advertise address enp0s3:
Error response from daemon: interface enp0s3 has more than one IPv6 address (2a00:c98:2060:a000:1:0:1d1e:ca75 and fe80::a00:27ff:fe7e:d9c4)
¿How do I apply the changes to the daemon so I stop encountering this error? I can't advertise an specific ip address since the machine is using dhcp.
Thanks.
The problem was solved using the following command:
sudo docker swarm init --advertise-address "$(ip addr show $MAIN_ETH_INTERFACE | grep "inet\b" | awk '{print $2}' | cut -d/ -f1)"
This way I don't need to specify an ipv4 address.

How can I use the internal ip address of a container as an environment variable in Docker

I'm trying to get the IP address of my docker container as an environment variable within the container. Here is what I've tried:
When starting the container
docker run -dPi -e ip=`hostname -i` myDockerImage
When the container is already booted up
docker exec -it myDockerImage bash -c "export ip=`hostname -i`"
The problem with these two methods is that it uses the ip address of the host running the commands, not the docker container it's being run on.
So then I created a script inside the docker container that looks like this:
#!/bin/bash
export ip=`hostname -i`
echo $ip
And then run this with
docker exec -it myDockerImage bash -c ". ipVariableScript.sh"
When I add my_cmd which in my case is bash to the end of the script, it works in that one session of bash. I can't use it later in the files I need it in. I need to set it as an environment variable, not as a variable for one session.
So I already sourced it with the '.'. But it still won't echo when I'm in the container. If I put an echo $ip in the script, it will give me the correct IP address. But can only be used from within the script it's being set in.
Service names in Docker are more reliable and easier to use. However, here's
How to assign Docker guest IP to environment var inside guest
$ docker run -it ubuntu bash -c 'IP=$(hostname -i); echo ip=$IP'
ip=172.17.0.76
So, this is an old question but I ended up with the same question yesterday and my solution is this: use the docker internal option.
My containers were working fine but at some point the ip changed and I needed to change it on my docker-compose. Of course I can use the "docker network inspect my-container_default" and get my internal IP from that, but this also means changing my docker-compose every time the ip changes (and I'm still not that familiar with docker in order to detect IP changes automatically or make a more sofisticated config). So, I use the "host.docker.internal" flag. Now I no more need to check what's my IP from docker and everything is always connected.
Here an example of a node app which uses elastic search and needs to connect.
version: '3.7'
services:
api:
...configs...
depends_on:
- 'elasticsearch'
volumes:
- ./:/usr/local/api
ports:
- '3000:80'
links:
- elasticsearch:els
environment:
- PORT=80
- ELASTIC_NODE=http://host.docker.internal:9200
elasticsearch:
container_name: 'els'
image: docker.elastic.co/elasticsearch/elasticsearch:7.13.4
...elastic search container configs...
ports:
- '9200:9200'
expose:
- 9200
networks:
- elastic
networks:
elastic:
driver: bridge
Note the "ELASTIC_NODE=http://host.docker.internal:9200" on api environments and the "network" that the elastic search container is using (on bridge mode)
This way you don't need to worry about knowing your IP.
The container name is postgres in this example. It is a bit clumsy, but it delivers.
container_ip=$(docker inspect postgres -f "{{json .NetworkSettings.Networks }}" \
| awk -v FS=: '{print $9}' \
| cut -f1 -d\, \
| echo "${container_ip//\"}")
Make a function out of it:
#!/usr/bin/env bash
set -o errexit
set -o nounset
set -eu -o pipefail
#set -x
#trap read debug
#assign container ip address to variable
function get_container_ip () {
container_ip=$(docker inspect "$1" -f "{{json .NetworkSettings.Networks }}" \
| awk -v FS=: '{print $9}' \
| cut -f1 -d\,)
container_ip=$(echo "${container_ip//\"}")
}
get_container_ip $1
echo "$container_ip"

Host port with DB to Docker container

I have host with PostgreSQL and Docker container. PostgreSQL work on 5432 port. Docker container must connect to database. How to connect container with database through Dockerfile or run command? EXPOSE 5432 and docker run -p 5432:5432 ... did not help.
From the documentation page:
Sometimes you need to connect to the Docker host from within your
container. To enable this, pass the Docker host’s IP address to the
container using the --add-host flag. To find the host’s address, use
the ip addr show command.
$ HOSTIP=`ip -4 addr show scope global dev eth0 | grep inet | awk '{print \$2}' | cut -d / -f 1`
$ docker run --add-host=docker:${HOSTIP} --rm -it busybox telnet docker 5432
EXPOSE or -p flag work the other way around e.g. publish container ports to host which you don't want here.

How to pass host ip address to docker in mac OS?

I am working on Linux machine and I wrote an script to pass local host IP address to docker container by passing an parameter It works fine for ubuntu.
will the same script run on mac OS and work as expected (pass IP address of local host to docker container)?
docker run -t -i -e "DOCKER_HOST=$(ip -4 addr show eth0 | grep -Po 'inet \K[\d.]+')" $IMAGE_NAME
On OSX use this command line:
docker run -it -e "DOCKER_HOST=$(ifconfig en0 | awk '/ *inet /{print $2}')" $IMAGE_NAME
On mac, you will be using a VM, so you might want to pass the IP of the docker machine you have declared:
(image from "docker on Mac OS X")
eval $(docker-machine env default)

How can I get Docker Linux container information from within the container itself?

I would like to make my docker containers aware of their configuration, the same way you can get information about EC2 instances through metadata.
I can use (provided docker is listening on port 4243)
curl http://172.17.42.1:4243/containers/$HOSTNAME/json
to get some of its data, but would like to know if there is a better way at least the get the full ID of the container, because HOSTNAME is actually shortened to 12 characters and docker seems to perform a "best match" on it.
Also, how can I get the external IP of the docker host (other than accessing the EC2 metadata, which is specific to AWS)
Unless overridden, the hostname seems to be the short container id in Docker 1.12
root#d2258e6dec11:/project# cat /etc/hostname
d2258e6dec11
Externally
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
d2258e6dec11 300518d26271 "bash" 5 minutes ago
$ docker -v
Docker version 1.12.0, build 8eab29e, experimental
I've found out that the container id can be found in /proc/self/cgroup
So you can get the id with :
cat /proc/self/cgroup | grep -o -e "docker-.*.scope" | head -n 1 | sed "s/docker-\(.*\).scope/\\1/"
A comment by madeddie looks most elegant to me:
CID=$(basename $(cat /proc/1/cpuset))
You can communicate with docker from inside of a container using unix socket via Docker Remote API:
https://docs.docker.com/engine/reference/api/docker_remote_api/
In a container, you can find out a shortedned docker id by examining $HOSTNAME env var.
According to doc, there is a small chance of collision, I think that for small number of container, you do not have to worry about it. I don't know how to get full id directly.
You can inspect container similar way as outlined in banyan answer:
GET /containers/4abbef615af7/json HTTP/1.1
Response:
HTTP/1.1 200 OK
Content-Type: application/json
{
"Id": "4abbef615af7...... ",
"Created": "2013.....",
...
}
Alternatively, you can transfer docker id to the container in a file.
The file is located on "mounted volume" so it is transfered to container:
docker run -t -i -cidfile /mydir/host1.txt -v /mydir:/mydir ubuntu /bin/bash
The docker id (shortened) will be in file /mydir/host1.txt in the container.
This will get the full container id from within a container:
cat /proc/self/cgroup | grep "cpu:/" | sed 's/\([0-9]\):cpu:\/docker\///g'
WARNING: You should understand the security risks of this method before you consider it. John's summary of the risk:
By giving the container access to /var/run/docker.sock, it is [trivially easy] to break out of the containment provided by docker and gain access to the host machine. Obviously this is potentially dangerous.
Inside the container, the dockerId is your hostname.
So, you could:
install the docker-io package in your container with the same version as the host
start it with --volume
/var/run/docker.sock:/var/run/docker.sock --privileged
finally, run: docker inspect $(hostname) inside the container
Avoid this. Only do it if you understand the risks and have a clear mitigation for the risks.
To make it simple,
Container ID is your host name inside docker
Container information is available inside /proc/self/cgroup
To get host name,
hostname
or
uname -n
or
cat /etc/host
Output can be redirected to any file & read back from application
E.g.: # hostname > /usr/src//hostname.txt
I've found that in 17.09 there is a simplest way to do it within docker container:
$ cat /proc/self/cgroup | head -n 1 | cut -d '/' -f3
4de1c09d3f1979147cd5672571b69abec03d606afcc7bdc54ddb2b69dec3861c
Or like it has already been told, a shorter version with
$ cat /etc/hostname
4de1c09d3f19
Or simply:
$ hostname
4de1c09d3f19
Docker sets the hostname to the container ID by default, but users can override this with --hostname. Instead, inspect /proc:
$ more /proc/self/cgroup
14:name=systemd:/docker/7be92808767a667f35c8505cbf40d14e931ef6db5b0210329cf193b15ba9d605
13:pids:/docker/7be92808767a667f35c8505cbf40d14e931ef6db5b0210329cf193b15ba9d605
12:hugetlb:/docker/7be92808767a667f35c8505cbf40d14e931ef6db5b0210329cf193b15ba9d605
11:net_prio:/docker/7be92808767a667f35c8505cbf40d14e931ef6db5b0210329cf193b15ba9d605
10:perf_event:/docker/7be92808767a667f35c8505cbf40d14e931ef6db5b0210329cf193b15ba9d605
9:net_cls:/docker/7be92808767a667f35c8505cbf40d14e931ef6db5b0210329cf193b15ba9d605
8:freezer:/docker/7be92808767a667f35c8505cbf40d14e931ef6db5b0210329cf193b15ba9d605
7:devices:/docker/7be92808767a667f35c8505cbf40d14e931ef6db5b0210329cf193b15ba9d605
6:memory:/docker/7be92808767a667f35c8505cbf40d14e931ef6db5b0210329cf193b15ba9d605
5:blkio:/docker/7be92808767a667f35c8505cbf40d14e931ef6db5b0210329cf193b15ba9d605
4:cpuacct:/docker/7be92808767a667f35c8505cbf40d14e931ef6db5b0210329cf193b15ba9d605
3:cpu:/docker/7be92808767a667f35c8505cbf40d14e931ef6db5b0210329cf193b15ba9d605
2:cpuset:/docker/7be92808767a667f35c8505cbf40d14e931ef6db5b0210329cf193b15ba9d605
1:name=openrc:/docker
Here's a handy one-liner to extract the container ID:
$ grep "memory:/" < /proc/self/cgroup | sed 's|.*/||'
7be92808767a667f35c8505cbf40d14e931ef6db5b0210329cf193b15ba9d605
Some posted solutions have stopped working due to changes in the format of /proc/self/cgroup. Here is a single GNU grep command that should be a bit more robust to format changes:
grep -o -P -m1 'docker.*\K[0-9a-f]{64,}' /proc/self/cgroup
For reference, here are snippits of /proc/self/cgroup from inside docker containers that have been tested with this command:
Linux 4.4:
11:pids:/system.slice/docker-cde7c2bab394630a42d73dc610b9c57415dced996106665d427f6d0566594411.scope
...
1:name=systemd:/system.slice/docker-cde7c2bab394630a42d73dc610b9c57415dced996106665d427f6d0566594411.scope
Linux 4.8 - 4.13:
11:hugetlb:/docker/afe96d48db6d2c19585572f986fc310c92421a3dac28310e847566fb82166013
...
1:name=systemd:/docker/afe96d48db6d2c19585572f986fc310c92421a3dac28310e847566fb82166013
I believe that the "problem" with all of the above is that it depends upon a certain implementation convention either of docker itself or its implementation and how that interacts with cgroups and /proc, and not via a committed, public, API, protocol or convention as part of the OCI specs.
Hence these solutions are "brittle" and likely to break when least expected when implementations change, or conventions are overridden by user configuration.
container and image ids should be injected into the r/t environment by the component that initiated the container instance, if for no other reason than
to permit code running therein to use that information to uniquely identify themselves for logging/tracing etc...
just my $0.02, YMMV...
There are 3 places that I see that might work so far, each have advantages & disadvantages:
echo $HOSTNAME or hostname
cat /proc/self/cgroup
cat /proc/self/mountinfo
$HOSTNAME is easy, but it is partial, and it will also be overwritten to pod name by K8s.
/proc/self/cgroup seems working with cgroupV1 but won't be there hosted in cgroupV2.
/proc/self/mountinfo will still have the container id for cgroupV2, however, the mount point will have different values by different container runtimes.
For example, in docker engine, the value looks like:
678 655 254:1 /docker/containers/7a0144cee1256c539fab790199527b7051aff1b603ebcf7ed3fd436440ef3b3a/resolv.conf /etc/resolv.conf rw,relatime - ext4 /dev/vda1 rw
In ContainerD (K8s default engine lately), it looks like:
1733 1729 0:35 /kubepods/besteffort/pod3272f253-be44-4a82-a541-9083e68cf99f/7a0144cee1256c539fab790199527b7051aff1b603ebcf7ed3fd436440ef3b3a /sys/fs/cgroup/blkio ro,nosuid,nodev,noexec,relatime master:17 - cgroup cgroup rw,blkio
Also, the biggest problem for all above is that they are all implementations, there's no abstraction and they all could be changed over time.
There is an effort to make it standard and I think it is worth watching:
https://github.com/opencontainers/runtime-spec/issues/1105
You can use this command line to identify the current container ID (tested with docker 1.9).
awk -F"-|/." '/1:/ {print $3}' /proc/self/cgroup
Then, a little request to Docker API (you can share /var/run/docker.sock) to retrieve all informations.
awk -F'[:/]' '(($4 == "docker") && (lastId != $NF)) { lastId = $NF; print $NF; }' /proc/self/cgroup
As an aside, if you have the pid of the container and want to get the docker id of that container, a good way is to use nsenter in combination with the sed magic above:
nsenter -n -m -t pid -- cat /proc/1/cgroup | grep -o -e "docker-.*.scope" | head -n 1 | sed "s/docker-\(.*\).scope/\\1/"
Use docker inspect.
$ docker ps # get conteiner id
$ docker inspect 4abbef615af7
[{
"ID": "4abbef615af780f24991ccdca946cd50d2422e75f53fb15f578e14167c365989",
"Created": "2014-01-08T07:13:32.765612597Z",
"Path": "/bin/bash",
"Args": [
"-c",
"/start web"
],
"Config": {
"Hostname": "4abbef615af7",
...
Can get ip as follows.
$ docker inspect -format="{{ .NetworkSettings.IPAddress }}" 2a5624c52119
172.17.0.24

Resources