I'm experimenting with docker and reflected a very slow connection from the nodejs (4.2.3) container to mongodb (3.2) container.
My setup, very basic, is this (docker-compose):
version: '2'
services:
web:
build: ./app
volumes:
- "./app:/src/app"
ports:
- "80:3000"
links:
- "db_cache:redis"
- "db:mongodb"
command: nodemon -L app/bin/www
db_cache:
image: redis
db:
image: mongo
My s.o. is OSX 10.10 and the docker version is 1.10.2.
The strange thing is that the connection time to the db is always 30 seconds.
Is there any automatic delay?
EDIT:
if I set ip address of mongodb container intead a "dns" (mongodb), the delay disappears!
Any ideas?
This does not completely solve the problem, but it allows you to restore the normal behavior.
The cause of this seems to be the version 2 of the docker-compose.yml.
If I remove the version 2 is completely eliminated the 30-second delay when connecting to mongodb:
web:
build: ./app
volumes:
- "./app:/src/app"
ports:
- "80:3000"
links:
- "db_cache:redis"
- "db:mongodb"
command: nodemon -L app/bin/www
db_cache:
image: redis
db:
image: mongo
I opened an issue here.
Related
I'm running an instance of a web application in my Docker container and am also running a MongoDB container so when I launch the web app I can easily connect to the DB on the app's connection page.
The issue is that I'm not sure how to reach the Mongo container from my web app and am not sure if my host/port connection info is correct.
My Docker Setup
As you can see the container is up and running with both mongo and web app services running without errors
I build the two through docker-compose.yml
version: "3.3"
services:
web:
image: grafana-asw-v3
container_name: grafana-asw-v3
restart: always
build: .
ports:
- "13000:3000"
volumes:
- grafana-storage:/var/lib/grafana
stdin_open: true
tty: true
db:
container_name: mongo
image: mongo
environment:
MONGO_INITDB_ROOT_USERNAME: root
MONGO_INITDB_ROOT_PASSWORD: example
volumes:
- grafana-mongo-db:/var/lib/mongo
ports:
- "27018:27017"
volumes:
grafana-mongo-db: {}
grafana-storage: {}
Issue
With everything up and running I'm attempting to connect through the web app, but I seem to be using the wrong connection info...
I assumed to use "hostMachine:port" (roxane:27018), but it's not connecting. Is there something I overlooked here?
There were two changes I had to make to fix this issue:
Modify the bind_ip in mongod.conf via making this change to my docker-compose file
db:
container_name: mongo
image: mongo
environment:
MONGO_INITDB_ROOT_USERNAME: root
MONGO_INITDB_ROOT_PASSWORD: example
volumes:
- grafana-mongo-db:/var/lib/mongo
ports:
- "27018:27017"
command: mongod --bind_ip 0.0.0.0
I needed to refer to the IP address instead of the hostname in the cli in my we application. (Thanks to this answer for help with this one)
Short answer
db service is in the same network than web service not in host network.
As you named your services via container_name you shoud be able to use the connection string mongodb://mongo:27017
Explanation
By default, docker containers run under a bridge network allowing them to communicate without viewing your host network.
When using ports in a compose file, you define that you want to map an internal port of the container to the host port
"27018:27017" => I want to expose the container port number 27017 to the host port number 27018.
As a result, you could expose your web frontend without exposing your mongo service :
version: "3.3"
services:
web:
image: grafana-asw-v3
container_name: grafana-asw-v3
restart: always
build: .
ports:
- "13000:3000"
volumes:
- grafana-storage:/var/lib/grafana
stdin_open: true
tty: true
db:
container_name: mongo
image: mongo
environment:
MONGO_INITDB_ROOT_USERNAME: root
MONGO_INITDB_ROOT_PASSWORD: example
volumes:
- grafana-mongo-db:/var/lib/mongo
volumes:
grafana-mongo-db: {}
grafana-storage: {}
I'm running a NodeJS app and its related services (Redis, Postgres) through docker-compose. My NodeJS app can reach Redis just fine using its name & port from my docker-compose file, but for some reason I can't seem to reach Postgres:
Error: getaddrinfo EAI_AGAIN postgres
at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:66:26)
My docker-compose file:
services:
api:
build:
context: ./
dockerfile: Dockerfile
ports:
- "3001:3001"
depends_on:
- postgres
- redis
postgres:
image: postgres:11.1
ports:
- "5432:5432"
expose:
- "5432"
hostname: postgres
environment:
POSTGRES_USER: root
POSTGRES_PASSWORD: root
POSTGRES_DB: test
restart: on-failure
networks:
- integration-tests
redis:
image: 'docker.io/bitnami/redis:6.0-debian-10'
environment:
# ALLOW_EMPTY_PASSWORD is recommended only for development.
- ALLOW_EMPTY_PASSWORD=yes
- REDIS_DISABLE_COMMANDS=FLUSHDB,FLUSHALL
ports:
- '6379:6379'
hostname: redis
volumes:
- 'redis_data:/bitnami/redis/data'
I've tried both normal lts and lts-alpine base images for my NodeJS app. I'm using knex, which delegates connecting to the pg library... Anybody have any idea why it won't even connect? I've tried both running directly through docker-compose and through tilt.
By adding :
networks:
- integration-tests
Only for postgres, you create a separate network only for postgres.
By default, docker-compose create a network for all your container inside the same file with the name: <project-name>_default. It's why, when using docker-compose all the containers in the same file could communicate using their name.
By specifying a network for postgres, you "ask" to docker-compose to not use the default network for it.
You have 2 solutions:
- Remove the instruction to failback to the default network
- Add the networks instruction to all other containers in your project / or only those who need it
Note: By default, docker-compose will prefixe all your object (container, networks, volume) with the project name. The default project name is the name of the current directory.
I have an application running on several containers and I am using docker-compose to run these services inside containers. I am using NodeJS with postgresql using node-postgresql library(pg).
When I run the application in my local machine it works perfectly fine. And also when I build the application in a aws e2 ubuntu server , it builds without any error. And when I try to check the endpoints it works pretty much well. But when I try an endpoint that involves a database call, no response is received. I tried to debug the application by adding some console logs and I think the problem is with the pg library. When the code reaches pool.query it just hangs there without giving any error. I tried to console.log , try catch errors but everytime the code stops at a pool.query() and it hangs. Any idea why this is happening ? Tried for hours to find a solution online but no luck.
I found another similar issue here https://github.com/brianc/node-postgres/issues/2069
My docker-compose.yml
version: '3'
services:
postgres:
container_name: postgres
image: mdillon/postgis
environment:
POSTGRES_MULTIPLE_DATABASES: user_data,gov_auth
POSTGRES_USER: commhawk
POSTGRES_PASSWORD: password
volumes:
- postgres:/data/postgres
- ./database:/docker-entrypoint-initdb.d
ports:
- "5433:5432"
restart: unless-stopped
rethinkdb:
container_name: rethinkdb
image: rethinkdb:latest
ports:
- "3004:8080"
- "29015:29015"
- "28015:28015"
volumes:
- rethinkdb:/data
user_data_service:
build: './user_data_service'
container_name: uds
ports:
- "3002:3000"
depends_on:
- postgres
environment:
DATABASE_URL: postgres://commhawk:password#postgres:5432/user_data
# volumes:
# - ./user_data_service:/src
# - container_node_modules:/src/node_modules
gov_authority_service:
build: './gov_authority_service'
container_name: gov_auth
ports:
- "3001:3000"
depends_on:
- postgres
environment:
DATABASE_URL: postgres://commhawk:password#postgres:5432/gov_auth
# volumes:
# - ./gov_authority_service:/src
socket_service:
build: './socket_service'
container_name: socket
ports:
- "3003:3000"
depends_on:
- rethinkdb
# volumes:
# - ./socket_service:/src
api_gateway:
image: express-gateway:latest
ports:
- "8080:8080"
depends_on:
- user_data_service
- socket_service
- gov_authority_service
volumes:
- ./api_gateway/gateway.config.yml:/var/lib/eg/gateway.config.yml
volumes:
postgres:
rethinkdb:
# container_node_modules:
Simply when I run like following, it stops without any result. When I include that inside try catch, no error is shown also. No way to deubug
let result = await pool.query("SELECT NOW()")
console.log(result)
Node-postgresql connection
const {Pool} = require("pg");
const pool = new Pool({
user: "commhawk",
host: "postgres",
database: "gov_auth",
password: "password",
port: "5432"
});
module.exports = pool;
Tried it with connectionString also. Still no luck !!
Can provide more info if needed. Thanks in advance
UPDATE (SOLUTION)
Okay so I figured out the error. When I built it on my local machine, it is using nodejs 13 and pg 8.0.2. But when I tried to built it on my server(like 1 month after building on the local), since I am using node:latest in my Dockerfile, it pulls the latest version which is nodejs 14. Nodejs 14 is incompatible with pg 8.0.2. So that is why the local versions runs well and the server version fails.
Solutions:
Downgrade nodejs to 13 to use with pg 8.0.2
Upgrade pg to 8.0.3 to use with nodejs 14
I am working with 3 Services:
api-knotain [main api service]
api-mongo [mongo db for api service]
api-redis [redis for api service]
The Dockerfile for api-knotain looks as follows
FROM node:latest
# Create app directory
WORKDIR /usr/src/app
# Install app dependencies
# A wildcard is used to ensure both package.json AND package-lock.json are copied
# where available (npm#5+)
COPY package*.json ./
RUN npm install
# If you are building your code for production
# RUN npm ci --only=production
# Bundle app source
COPY . .
CMD [ "npm", "start" ]
my docker-compose file as such:
version: '3.3'
services:
api-knotain:
container_name: api-knotain
restart: always
build: ../notify.apiV2/src
ports:
- "7777:7777"
links:
- api-mongo
- api-redis
environment:
- REDIS_URI=api-redis
- REDIS_PORT=32770
- MONGO_URI=api-mongo
- MONGO_PORT=27017
- RESEED=true
- NODE_TLS_REJECT_UNAUTHORIZED=0
api-mongo:
container_name: api-mongo
image: mongo
volumes:
- ./data:/data/db
ports:
- "27017:27017"
api-redis:
container_name: api-redis
image: "redis:alpine"
ports:
- "32770:32770"
runnin
docker-compose build
docker-compose up
output:
api-knotain | connecting mongo ...: mongodb://api-mongo:27017/notify
api-knotain | Redis error: Error: Redis connection to api-redis:32770 failed - connect ECONNREFUSED 172.21.0.2:32770
api-knotain | mongo error:MongoNetworkError: failed to connect to server [api-mongo:27017] on first connect [MongoNetworkError: connect ECONNREFUSED 172.21.0.3:27017]
api-knotain | Example app listening on port 7777!
neither mongo nor redis can be connected.
I tried the following things:
use localhost instead of container name
use different ports
use expose vs port
always with the same result
note:
i can connect without issue to both mongo & redis through local cli 'localhost:port'
what am I missing?
Probably redis and mongo containers start later than you application and therefore your app will not see those. To counter this you must wait for those services to be ready.
Also links is a legacy feature of Docker. You should use depends_on to control startup order and user-defined networks if you want to isolate your database and redis containers from external network.
version: '3.3'
services:
api-knotain:
depends_on:
- api-mongo
- api-redis
container_name: api-knotain
restart: always
build: ../notify.apiV2/src
ports:
- "7777:7777"
links:
- api-mongo
- api-redis
environment:
- REDIS_URI=api-redis
- REDIS_PORT=32770
- MONGO_URI=api-mongo
- MONGO_PORT=27017
- RESEED=true
- NODE_TLS_REJECT_UNAUTHORIZED=0
api-mongo:
container_name: api-mongo
image: mongo
volumes:
- ./data:/data/db
ports:
- "27017:27017"
api-redis:
container_name: api-redis
image: "redis:alpine"
ports:
- "32770:32770"
It looks like depend_on did not properly work in 3.3 version of docker compose. after updating the version to 3.7 all works perfectly without any changes to the compose file.
i want launch three containers for my web application.
The containers are: frontend, backend and mongo database.
To do this i write the following docker-compose.yml
version: '3.7'
services:
web:
image: node
container_name: web
ports:
- "3000:3000"
working_dir: /node/client
volumes:
- ./client:/node/client
links:
- api
depends_on:
- api
command: npm start
api:
image: node
container_name: api
ports:
- "3001:3001"
working_dir: /node/api
volumes:
- ./server:/node/api
links:
- mongodb
depends_on:
- mongodb
command: npm start
mongodb:
restart: always
image: mongo
container_name: mongodb
ports:
- "27017:27017"
volumes:
- ./database/data:/data/db
- ./database/config:/data/configdb
and update connection string on my .env file
MONGO_URI = 'mongodb://mongodb:27017/test'
I run it with docker-compose up -d and all go on.
The problem is when i run docker logs api -f for monitoring the backend status: i have MongoNetworkError: failed to connect to server [mongodb:27017] on first connect error, because my mongodb container is up but not in waiting connections (he goes up after backend try to connect).
How can i check if mongodb is in waiting connections status before run api container?
Thanks in advance
Several possible solutions in order of preference:
Configure your application to retry after a short delay and eventually timeout after too many connection failures. This is an ideal solution for portability and can also be used to handle the database restarting after your application is already running and connected.
Use an entrypoint that waits for mongo to become available. You can attempt a full mongo client connect + login, or a simple tcp port check with a script like wait-for-it. Once that check finishes (or times out and fails) you can continue the entrypoint to launching your application.
Configure docker to retry starting your application with a restart policy, or deploy it with orchestration that automatically recovers when the application crashes. This is a less than ideal solution, but extremely easy to implement.
Here's an example of option 3:
api:
image: node
deploy:
restart_policy:
condition: unless-stopped
Note, looking at your compose file, you have a mix of v2 and v3 syntax in your compose file, and many options like depends_on, links, and container_name, are not valid with swarm mode. You are also defining settings like working_dir, which should really be done in your Dockerfile instead.