I am not able to connect to my mongoDB instance through the client such as Robot 3T. It doesn't even give a chance to authenticate.
Here is my docker-compose file:
version: '3'
services:
frontend:
build:
context: panel_front_end/
ports:
- 8080:80
environment:
- NODE_ENV=local
restart: always
networks:
- nginx_network
backend:
build:
context: panel_back_end/
args:
- SSH_PRIVATE_KEY
environment:
- NODE_ENV=local
- HTTPS=true
ports:
- "3000:3000"
- "8443:8443"
volumes:
- ./panel_back_end/ssl:/usr/src/app/ssl
restart: always
networks:
- nginx_network
depends_on:
- mongodb
links:
- mongodb
mongodb:
build:
context: mongo/
ports:
- "27017:27017"
expose:
- 27017
volumes:
- ./data/db:/data/db
- ./backup:/backup
environment:
# provide your credentials here
- MONGODB_ADMIN_USER=${MONGODB_ADMIN_USER}
- MONGODB_ADMIN_PASS=${MONGODB_ADMIN_PASS}
- MONGODB_APPLICATION_DATABASE=${MONGODB_APPLICATION_DATABASE}
- MONGODB_APPLICATION_USER=${MONGODB_APPLICATION_USER}
- MONGODB_APPLICATION_PASS=${MONGODB_APPLICATION_PASS}
networks:
nginx_network:
external: true
I dodocker ps and this is the output:
ef695029ff3f control-panel_backend "/bin/sh -c ./entryp…" 5 minutes ago Restarting (1) 22 seconds ago control-panel_backend_1
d397eac69a70 control-panel_frontend "/bin/sh -c ./entryp…" 5 minutes ago Up 5 minutes 0.0.0.0:8080->80/tcp control-panel_frontend_1
48ee29caee6a control-panel_mongodb "docker-entrypoint.s…" 5 minutes ago Up 5 minutes 0.0.0.0:27017->27017/tcp control-panel_mongodb_1
5514f5e97a65 mysql/mysql-server:5.7 "/entrypoint.sh mysq…" 2 days ago Up 2 days (healthy) 0.0.0.0:3306->3306/tcp, 33060/tcp mysql-migration
The client gives me this:
It doesn't even gives the chance to authenticate the credentials. And i see the port being open locally and in docker container.
I tried:
sudo lsof -i tcp:27017
and it get:
com.docke 31591 jgonz 23u IPv4 0xaa557204f5a42733 0t0 TCP *:27017 (LISTEN)
com.docke 31591 jgonz 26u IPv6 0xaa557204e73407f3 0t0 TCP localhost:27017 (LISTEN)
I not exactly sure what i am missing in my configuration. Also, at the folder level of the docker-compose I have a .env file with all the environment variables. My mongodb Dockerfile run two scripts, by following this article:
http://blog.bejanalex.com/2017/03/running-mongodb-in-a-docker-container-with-authentication/
Related
I have a compose file that looks as follows:
version: "3.9"
services:
postgresdb:
image: postgres:14.4-alpine
container_name: pgsql
restart: always
volumes:
- ./db/init.sql:/docker-entrypoint-initdb.d/0_init.sql
# - $HOME/database:/var/lib/postgresql/data
ports:
- "8081:5432"
expose:
- "8081"
environment:
POSTGRES_DB: todos
POSTGRES_USER: admin
POSTGRES_PASSWORD: password
SERVICE_TAGS: dev
SERVICE_NAME: postgresdb
networks:
- internalnet
nodeapp:
container_name: server
build: .
image: nodeapp:1.0
ports:
- "3002:3002"
expose:
- "3002"
environment:
DB_HOST: postgresdb
DB_PORT: 8081
DB_USER: "admin"
DB_PASSWORD: "password"
DB_NAME: todos
DB_CONNECTION_LIMIT: 20
SERVICE_TAGS: dev
SERVICE_NAME: nodeapp
PORT: 3002
depends_on:
- postgresdb
networks:
- internalnet
networks:
internalnet:
driver: bridge
When i run the compose up -d command the ps command here is what i am getting:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
53a12ab5a9ae nodeapp:1.0 "docker-entrypoint.s…" 12 seconds ago Up 5 seconds 0.0.0.0:3002->3002/tcp server
98cda5442cb0 postgres:14.4-alpine "docker-entrypoint.s…" 14 seconds ago Up 8 seconds 8081/tcp, 0.0.0.0:8081->5432/tcp pgsql
To me this looks okay. But now when i try to send request to the server I'm getting an error saying:
"connect ECONNREFUSED 192.*.*.*:8081"
I don't know why my application is failing to connect to postgres of which they are on the same network.
I am trying to set up multiservices architecture in my Node.js backend with docker. I have currently two services with two separate databases
version: "3"
services:
server-api-getaway:
build:
context: "."
dockerfile: "./server-api-gateway/Dockerfile"
depends_on:
- db
- redis
ports:
- "7100:7100"
volumes:
- ./server-api-gateway:/usr/src/app/server-api-gateway
environment:
- CHOKIDAR_USEPOLLING=true
- REDIS_URL=redis://cache
- DATABASE_URL=postgres://postgres:postgres#db:3336/dbname
- DATABASE_PORT=3336
- DATABASE_HOST=host.docker.internal
- POSTGRES_DB=dbname
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- PORT=7100
db:
image: postgres
restart: always
environment:
- POSTGRES_DB=dbname
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
ports:
- "3336:3306"
server-user-service:
build:
context: "."
dockerfile: "./server-user-service/Dockerfile"
depends_on:
- user-service-db
ports:
- "7000:7000"
volumes:
- ./server-user-service:/usr/src/app/server-user-service
environment:
- CHOKIDAR_USEPOLLING=true
- REDIS_URL=redis://cache
- DATABASE_URL=postgres://postgres:postgres#user-service-db:3307/dbname
- DATABASE_PORT=3307
- DATABASE_HOST=host.docker.internal
- POSTGRES_DB=dbname
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- PORT=7000
user-service-db:
image: postgres
restart: always
environment:
- POSTGRES_DB=dbname
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
ports:
- "3307:3306"
redis:
image: redis
container_name: cache
restart: always
ports:
- 6379:6379
volumes:
pgdata:
driver: local
pgconf:
driver: local
pglog:
driver: local
I expect both databases to run on different ports, but when I run docker-compose up they are exposed to 5432 port
2021-03-05 07:12:10.111 UTC [1] LOG: listening on IPv4 address
"0.0.0.0", port 5432
When I connect my service to port 5432 everything works fine. But I need two databases to be exposed to different ports. How can I achieve it?
Update
I've changed port mapping as was suggested in comments.
user-service-db:
image: postgres
restart: always
environment:
- POSTGRES_DB=dbname
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
ports:
- "5432:3336"
And made a fresh start after docker-compose down
Result remained the same
user-service-db_1 | 2021-03-05 10:01:23.023 UTC [1] LOG:
listening on IPv4 address "0.0.0.0", port 5432
probably there is a detail I am missing
As an implementation detail, containers have their own private IP addresses. You can't usually access these addresses from outside Docker, but when your container tries to connect to the host name db, that maps to one of these internal addresses. This means that it's okay for processes in different containers to listen on the same port, since each process is in its own isolated network space.
That means it's standard for a packaged process to listen on its "normal" port. The output you're seeing is the output of the PostgreSQL startup; this will always listen on 0.0.0.0:5432 (unless you make tricky modifications to its configuration file).
Connections between containers also always connect to the "normal" port:
# Use ordinary port, ports: are ignored vvvv
- DATABASE_URL=postgres://postgres:postgres#db:5432/dbname
Publishing a port makes it accessible from outside Docker, using the host's IP address (and DNS name). The second ports: number needs to match the port number inside the container, the first one can be anything that's not already in use. Connections between containers never use this.
ports:
# vvvv the ordinary port, fixed by the image
- "3336:5432"
"Expose" as a verb means almost nothing in modern Docker. EXPOSE in a Dockerfile is mostly documentation and there's no reason to use the Compose expose: setting.
You need to expose the correct port.
Postgres default port is 5432: i.e. the postgres service in your docker container will listen to port 5432.
I guess, on the host, you want to listen 3336 (or 3306?).
in this case the mapping must be:
ports:
- "3336:5432"
i.e. the format is HOST:CONTAINER
see also:
https://docs.docker.com/compose/compose-file/compose-file-v3/#ports
https://www.whitesourcesoftware.com/free-developer-tools/blog/docker-expose-port/
I've built two NEST app which shares a common database. I've been trying to figure out a way to make these two apps to get data from the Mongo db container. I'm stuck with this process.
docker-compose.yml with app 1
version: '3'
services:
backend:
image: 'example-image_1:1.0.0'
working_dir: /app/example-app-1/backend/example-app-1-api
environment:
- DB_URL=mongodb://0.0.0.0:27017/example_app_1
- BACKEND_PORT=3333
- BACKEND_IP=0.0.0.0
restart: always
network_mode: "host"
ports:
- '3333:3333'
command: ['node', 'main.js']
depends_on:
- mongodb
expose:
- 3333
mongodb:
image: 'mongo:latest'
environment:
- 'MONGODB_DATABASE="example_app_1"'
ports:
- '27017:27017'
expose:
- 27017
docker-compose.yml with app-2
version: '3'
services:
backend:
image: 'example_app_2:1.0.0'
working_dir: /app/example_app_2/backend/example-app-2-api
environment:
- DB_URL=mongodb://0.0.0.0:27017/example_app_2
- BACKEND_PORT=8888
- BACKEND_IP=0.0.0.0
restart: always
ports:
- '8888:8888'
command: ['node', 'main.js']
expose:
- 8888
mongodb:
image: 'mongo:latest'
environment:
- 'MONGODB_DATABASE="example_app_2"'
ports:
- '27017:27017'
expose:
- 27017
I need help in making these app communicate with common container - mongodb
Click link for Architecture Setup
Since you're using 2 different docker-compose.yml you're actually running 2 backend and 2 mongodb on 2 docker-networks
One of the 2 mongo won't start cause the port is already occupied.
Option 1 (nicer):
services:
backend_1:
...
ports:
- '8888:8888'
backend_2:
...
ports:
- '8899:8899'
mongodb:
ports:
- '27017:27017'
This setup provides 3 container on the same network.
Now you can access at mongo from both backends at <mongo_ip>:27017
Option 2 (ugly):
services:
backend:
...
ports:
- '8888:8888'
mongodb:
ports:
- '27017:27017'
And in another docker-compose
services:
backend:
...
ports:
- '8888:8888'
This setup provides 3 container on 2 different network.
In this setup each docker-compose.yml has it's own docker network, so from the second backend service you have to connect to another docker network to access the container.
I have docker compose use Mongo, Redis and Node js.
Mongo and Redis running good the problem is node js cant connect to Redis.
When i test on my ubuntu laptop with docker is work fine. but not when i run docker in server (Centos 7).
i sure my redis work because when i do SSH Port Forwarding its work. i can access from my ubuntu.
version: '2.1'
services:
aqua-server:
image: aqua-server
build: .
command: pm2-runtime process.yml
container_name: "aqua-server"
working_dir: /app
environment:
NODE_ENV: production
AQUA_MONGO_IP: 172.20.0.1
AQUA_MONGO_PORT: 3002
AQUA_MONGO_DATABASE: aqua-server
AQUA_EXPRESS_PORT: 3000
AQUA_REDIS_PORT: 3003
AQUA_REDIS_HOST: 172.20.0.1
volumes:
- .:/app/
ports:
- 3001:3000
links:
- mongodb
- redis
depends_on:
- mongodb
- redis
networks:
frontend:
ipv4_address: 172.20.0.2
mongodb:
image: mongo:4.2.0-bionic
container_name: "aqua-mongo"
environment:
MONGO_LOG_DIR: /dev/null
volumes:
- /home/sagara/DockerVolume/MongoLocal:/data/db
ports:
- 3002:27017
networks:
frontend:
ipv4_address: 172.20.0.3
redis:
image: redis:5.0.5-alpine
container_name: "aqua-redis"
ports:
- 3003:6379
networks:
frontend:
ipv4_address: 172.20.0.4
networks:
frontend:
driver: bridge
ipam:
config:
- subnet: 172.20.0.0/24
This my DockerFile on Node js
FROM node:10.16.0-alpine
WORKDIR /app
COPY . .
RUN npm install
RUN npm install pm2 -g
EXPOSE 3000
CMD pm2-runtime process.yml
this proccess.yml
apps:
- script: index.js
instances: 1
exec_mode: cluster
name: 'aqua-server'
this my logs docker
aqua-server | Error: Redis connection to 172.20.0.1:3003 failed - connect EHOSTUNREACH 172.20.0.1:3003
aqua-server | at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1106:14)
aqua-server | 2019-09-06T07:48:13: PM2 log: App name:aqua-server id:0 disconnected
aqua-server | 2019-09-06T07:48:13: PM2 log: App [aqua-server:0] exited with code [0] via signal [SIGINT]
aqua-server | 2019-09-06T07:48:13: PM2 log: App [aqua-server:0] starting in -cluster mode-
aqua-server | 2019-09-06T07:48:13: PM2 log: App [aqua-server:0] online
Redis Logs Docker
aqua-redis | 1:M 06 Sep 2019 07:47:12.512 * DB loaded from disk: 0.000 seconds
aqua-redis | 1:M 06 Sep 2019 07:47:12.512 * Ready to accept connections
Mongo Logs Docker
aqua-mongo | 2019-09-06T07:47:15.895+0000 I NETWORK [initandlisten] Listening on /tmp/mongodb-27017.sock
aqua-mongo | 2019-09-06T07:47:15.895+0000 I NETWORK [initandlisten] Listening on 0.0.0.0
aqua-mongo | 2019-09-06T07:47:15.895+0000 I NETWORK [initandlisten] waiting for connections on port 27017
That problem occur because you're trying to access from your Ubuntu, outside of Docker Network. And your Node container is inside that network so the way they communicate need to modify a bit to make it work. So I will refer my best practice way:
version: '2.1'
services:
aqua-server:
image: aqua-server
build: .
command: pm2-runtime process.yml
container_name: "aqua-server"
working_dir: /app
environment:
NODE_ENV: production
AQUA_MONGO_IP: mongodb
AQUA_MONGO_PORT: 27017
AQUA_MONGO_DATABASE: aqua-server
AQUA_EXPRESS_PORT: 3000
AQUA_REDIS_PORT: 6379
AQUA_REDIS_HOST: redis
volumes:
- .:/app/
ports:
- 3001:3000
links:
- mongodb
- redis
depends_on:
- mongodb
- redis
networks:
frontend:
ipv4_address: 172.20.0.2
mongodb:
image: mongo:4.2.0-bionic
container_name: "aqua-mongo"
environment:
MONGO_LOG_DIR: /dev/null
volumes:
- /home/sagara/DockerVolume/MongoLocal:/data/db
ports:
- 3002:27017
networks:
frontend:
ipv4_address: 172.20.0.3
redis:
image: redis:5.0.5-alpine
container_name: "aqua-redis"
ports:
- 3003:6379
networks:
frontend:
ipv4_address: 172.20.0.4
networks:
frontend:
driver: bridge
ipam:
config:
- subnet: 172.20.0.0/24
This way works because Docker handle the DNS for you and the domain names are the service name so that you can see in the docker-compose.yml above, I've changed AQUA_MONGO_IP to mongodb and AQUA_REDIS_HOST to redis. And I've changed the PORT too, Docker containers created by that docker-compose.yml will stay inside a network and they communicate by the PORT they exposed on their Dockerfile. So in this case other services can communicate with aqua-server via PORT=3000, aqua-server can communicate with mongodb service via PORT=27017 (this port is exposed inside Dockerfile of mongodb, EXPOSE 27017)
Hope that helps!
I have a running service within a container on my localhost machine
I Can't establish a connection with it even though I specified the port mapping within my docker-compose.yml here it is
version: '2.1'
services:
users-db:
container_name: users-db
build: ./services/users-service/src/db
ports:
- '27017:27017'
volumes:
- './services/users-service/src/db/:/data/db'
users-service:
container_name: users-service
build: './services/users-service/'
volumes:
- './services/users-service:/usr/src/app'
- './services/users-service/package.json:/usr/src/package.json'
ports:
- '3000:3000'
environment:
- NODE_ENV=test
- JWT_SECRET=fuckOffChinese
depends_on:
users-db:
condition: service_started
presence_db:
image: redis
presense_service:
container_name: presense_service
build: './services/presence-service/'
ports:
- "8081:8081"
environment:
- JWT_SECRET=thirdEyeSecret
- PORT=8081
volumes:
- './services/presence-service:/usr/src/app'
- './services/presence-service/package.json:/usr/src/package.json'
depends_on:
- presence_db
this is the command that I use to run this service
docker-compose run presense_service
and this is what I get every time I try to ping it from the terminal by simply doing an HTTP GET request
http: error: ConnectionError: HTTPConnectionPool(host='localhost', port=8081): Max retries exceeded with url: / (Caused by
NewConnectionError(': Failed to establish a new connection: [Errno
61] Connection refused',)) while doing GET request to URL:
http://localhost:8081/
I'm running macOS 10.13.5 and the server start noramally and here is the logs of it
> users-service#1.0.0 start /usr/src
> gulp --gulpfile app/gulpfile.js
[10:12:46] Working directory changed to /usr/src/app
[10:12:52] Using gulpfile /usr/src/app/gulpfile.js
[10:12:52] Starting 'start'...
[10:12:56] Finished 'start' after 3.99 s
[10:12:56] Starting 'lint'...
[10:12:57] Finished 'lint' after 239 ms
[10:12:57] Starting 'default'...
[10:12:57] Finished 'default' after 131 μs
[10:12:57] [nodemon] 1.18.2
[10:12:57] [nodemon] to restart at any time, enter `rs`
[10:12:57] [nodemon] watching: *.*
[10:12:57] [nodemon] starting `node ./index.js`
Server listening on: http://localhost:8081
Redis client connected
Try docker-compose run presense_service --service-ports, or better, use docker-compose up.
docker-compose run specifically doesn't apply the ports from your Compose file to "prevent port collisions with already-open ports" [1] - so you have to add this option, or specify them manually with the same options you would pass to docker run.
Ideally, use docker-compose up -d and then docker-compose logs -f presense_service to get logs. Shut your application down with docker-compose down.
If you really need to, you can comment services out of your docker-compose.yml file that you don't want started.
If you didn't know, the latest version of the compose format is 3.6 - 2.1 is over two years old (released with Docker v1.12.0 on 201607/28 [2]).
Proving this is easy - since I didn't have your code, I replaced all the image/build lines with image: nginx (and took the host path off any volumes).
Example modified compose file (just for reference):
version: '2.1'
services:
users-db:
container_name: users-db
image: nginx
ports:
- '27017:27017'
volumes:
- './services/users-service/src/db/:/data/db'
users-service:
container_name: users-service
image: nginx
volumes:
- '/usr/src/app'
- '/usr/src/package.json'
ports:
- '3000:3000'
environment:
- NODE_ENV=test
- JWT_SECRET=fuckOffChinese
depends_on:
users-db:
condition: service_started
presence_db:
image: redis
presense_service:
container_name: presense_service
image: nginx
ports:
- "8081:8081"
environment:
- JWT_SECRET=thirdEyeSecret
- PORT=8081
volumes:
- '/usr/src/app'
- '/usr/src/package.json'
depends_on:
- presence_db
Running docker ps after docker-compose up -d gives this (pay attention to the PORTS column):
my-machine$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f1b89cf3d6c2 nginx "nginx -g 'daemon of…" 19 seconds ago Up 14 seconds 80/tcp, 0.0.0.0:3000->3000/tcp users-service
be0e9b2bb005 nginx "nginx -g 'daemon of…" 19 seconds ago Up 13 seconds 80/tcp, 0.0.0.0:8081->8081/tcp presense_service
2efed2546926 nginx "nginx -g 'daemon of…" 20 seconds ago Up 14 seconds 80/tcp, 0.0.0.0:27017->27017/tcp users-db
c7a88a84f422 redis "docker-entrypoint.s…" 20 seconds ago Up 14 seconds 6379/tcp test_presence_db_1
...so there's nothing wrong with your port configuration. docker ps after docker-compose run presense_service, then, shows:
my-machine$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
89e4a4f90a75 nginx "nginx -g 'daemon of…" 28 seconds ago Up 21 seconds 80/tcp test_presense_service_run_1
2c91fcb5091d redis "docker-entrypoint.s…" 29 seconds ago Up 24 seconds 6379/tcp test_presence_db_1
...and therefore it was your command causing the problem. Happy to help as I've learnt something new :)
[1] https://docs.docker.com/compose/reference/run/
[2] https://docs.docker.com/release-notes/docker-engine/#1120-2016-07-28
Try this to find out if port 8081 is open/occupied:
netstat -l | grep 8081
When your server logs say
Server listening on: http://localhost:8081
The server will be inaccessible outside the Docker container. It looks like this is the default behavior of the Express JavaScript Web server, and many other frameworks when run in “developer” mode. You need to set the server to listen on all IP addresses, probably by passing 0.0.0.0 as the “bind” address. (Note that you cannot connect to 0.0.0.0, and saying “listen to 0.0.0.0” means “accept connections from anywhere”.
If your server is in fact based on Express, https://superuser.com/questions/582624/how-to-access-nodejs-server-on-lan might be informative to you.