Seeding data in a Mongo db within a linked Docker container - node.js

I'm using fig to deploy my Node.js app.
fig.yml
web:
build: .
command: node app.js
links:
- db
ports:
- "1337:1337"
db:
image: dockerfile/mongodb
Running fig run db env gives me the following environment vars:
DB_PORT=tcp://172.17.0.29:27017
DB_PORT_27017_TCP=tcp://172.17.0.29:27017
DB_PORT_27017_TCP_ADDR=172.17.0.29
DB_PORT_27017_TCP_PORT=27017
DB_PORT_27017_TCP_PROTO=tcp
DB_PORT_28017_TCP=tcp://172.17.0.29:28017
DB_PORT_28017_TCP_ADDR=172.17.0.29
DB_PORT_28017_TCP_PORT=28017
DB_PORT_28017_TCP_PROTO=tcp
DB_NAME=/pos_db_run_3/db
My app's Dockerfile looks like this
# Pull base image.
FROM dockerfile/nodejs
# install mongo client
RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10
RUN echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen' | tee /etc/apt/sources.list.d/mongodb.list
RUN apt-get update
RUN apt-get install -y dnsutils
RUN apt-get install -y mongodb-org-shell
# copy the source files into the image
COPY . /data/myapp
# Define working directory.
WORKDIR /data/myapp
# Install dependencies
RUN npm install
# Create default database and user
RUN mongo $DB_PORT < seed-mongo
RUN node/seed.js
EXPOSE 1337
# Define default command.
CMD ["bash"]
However the build is failing at RUN mongo $DB_PORT < seed-mongo
MongoDB shell version: 2.6.5
connecting to: test
2014-11-20T20:47:59.193+0000 warning: Failed to connect to 127.0.0.1:27017, reason: errno:111 Connection refused
2014-11-20T20:47:59.195+0000 Error: couldn't connect to server 127.0.0.1:27017 (127.0.0.1), connection attempt failed at src/mongo/shell/mongo.js:146
exception: connect failed
Service 'web' failed to build: The command [/bin/sh -c mongo $DB_PORT < seed-mongo] returned a non-zero code: 1
According to the fig docs I'd be better off referring to the database simply as db so I tried
RUN mongo db < seed-mongo
But this gave me
MongoDB shell version: 2.6.5
connecting to: db
2014-11-20T21:08:11.154+0000 warning: Failed to connect to 127.0.0.1:27017, reason: errno:111 Connection refused
2014-11-20T21:08:11.156+0000 Error: couldn't connect to server 127.0.0.1:27017 (127.0.0.1), connection attempt failed at src/mongo/shell/mongo.js:146
exception: connect failed
So I injected a RUN host db in there just to see if db really is a host name. And no, despite the docs above it's not.
Step 10 : RUN host db
---> Running in 73b4dc787c79
Host db not found: 3(NXDOMAIN)
Service 'web' failed to build: The command [/bin/sh -c host db] returned a non-zero code: 1
So I'm stumped.
How am I supposed to talk to the mongo instance running in my linked Docker container?

Your mongo container has ip address $DB_PORT_27017_TCP_ADDR
So try with mongo --host $DB_PORT_27017_TCP_ADDR < seed-mongo
You can't use it in a RUN statement I think, because the environment variables are not available at build-time. You could try using it in a CMD I guess.

My solution was a combination of insights from above, but mainly I stopped trying to seed a specific database and db user in the build phase and instead use --host db_1 with no username or password. If I really need that functionality later I'll work it out.
I could then build the project and connect it to mongo in the other image.
In my config/connections.js I included:
development: {
adapter : 'sails-mongo',
host : 'db_1',
port : 27017,
user : '',
password : '',
database : process.env.DB_NAME
},
and I changed the CMD to be "node data/seeds.js && node app"

You never exposed your Mongo port.
web:
build: .
command: node app.js
links:
- db
ports:
- "1337:1337"
db:
image: dockerfile/mongodb
ports:
- "27017:27017"

Related

Docker MongoDB immadiately closes connection after receiving metadata

I'm trying to compose docker app with two containers:
mongo
app
Mongo container works just fine, meanwhile app cannot connect to mongo. Neither node.js app nor mongostat can. The weird part is, I tried to run this project on both computers with Win10 and it works normally on the other one.
These are logs from mongo container when I run node app.js or mongostat --uri "mongodb://mongo:27017/project" from app container:
2019-05-22T09:33:52.225+0000 I NETWORK [conn17] received client metadata from 192.168.96.2:42916 conn17: { driver: { name: "nodejs", version: "3.1.10" }, os: { type: "Linux", name: "linux", architecture: "x64", version: "4.9.125-linuxkit" }, platform: "Node.js v10.15.3, LE, mongodb-core: 3.1.9" }
2019-05-22T09:33:52.231+0000 I NETWORK [conn17] end connection 192.168.96.2:42916 (0 connections now open)
This means both containers can see each other so .yml file should be fine. If the problem was with code then it shouldn't work on both computers.
Dockerfile:
FROM node:10.15.3-alpine
RUN apk update && apk --no-cache --virtual build-dependencies add python make g++ && apk del build-dependencies
RUN mkdir -p /home/node/app && chown -R node:node /home/node/app
WORKDIR /home/node/app
COPY package*.json ./
USER node
COPY --chown=node:node . .
ENV NPM_CONFIG_PREFIX=/home/node/.npm-global
RUN npm install
EXPOSE 3000
CMD ["node", "app.js"]
docker-compose.yml:
version: "3.5"
services:
app:
container_name: app
restart: always
build: .
ports:
- "3000:3000"
networks:
- mongo
mongo:
restart: always
container_name: mongo
image: mongo
expose:
- 27017
volumes:
- mongodata:/data/db
ports:
- '27017:27017'
networks:
- mongo
volumes:
mongodata:
networks:
mongo:
external: true
snippet from app.js:
MongoClient.connect('mongodb://mongo:27017/project', {useNewUrlParser: true}, (err, client) => {
if (err) throw err; //throws MongoNetworkError: failed to connect to server [mongo:27017] on first connect [MongoNetworkError: getaddrinfo ENOTFOUND mongo mongo:27017]
console.log("connected");
client.close();//at the moment this line is not being reached because of throw err;
});```
Does it help if you insert a "sleep 10" in your application, before connecting to the mongo db? If so, adding something like wiatforit (https://github.com/maxcnunes/waitforit) might help.
Since you are getting a getaddrinfo ENOTFOUND error, the mongo hostname isn't resolving. Usually, that happens for one of two reasons: 1) your containers aren't on the same network or 2) the other container isn't up and running yet. Seeing that they are on the same network, it sounds like it's something with the container being up.
To troubleshoot, I would start another container, put it on the network, and validate the mongo hostname resolves.
docker container run --rm -ti --network mongo ubuntu
$ apt update && apt install -y dnsutils
$ dig mongo
At this point, you should see the A record resolve to the database. If not, validate the mongo database container is up and running.
You can also try doing this within your app container as well. If that's working, then using something like waitforit should work. This is a common issue, as apps may start up before the database is either running or ready to accept connections.
As one other item of feedback, you don't need to expose the mongo port. This is making it accessible to the world, which most likely isn't what you want. You can still do container-to-container communication without exposing the port.
After hours of trying multiple things I have found solution: turn off Windows Firewall. That's it.
Thanks, I appreciate your help.

How to run a nodejs app in a mongodb docker image?

i am getting this error when i try to run the commande "mongo" in the container bash:
Error: couldn't connect to server 127.0.0.1:27017, connection attempt failed: SocketException: Error connecting to 127.0.0.1:27017 :: caused by :: Connection refused :connect#src/mongo/shell/mongo.js:328:13 #(connect):1:6exception: connect failed
i'm trying to set up a new nodejs app in a mongo docker image. the image is created fine with dockerfile in docker hub and i pull it, create a container and every thing is good but when i try to tape "mongo" commande in the bash a get the error.
this is my dockerfile
FROM mongo:4
RUN apt-get -y update
RUN apt-get install -y nodejs npm
RUN apt-get install -y curl python-software-properties
RUN curl -sL https://deb.nodesource.com/setup_11.x | bash -
RUN apt-get install -y nodejs
RUN node -v
RUN npm --version
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
CMD [ "npm", "start"]
EXPOSE 3000
When your Dockerfile ends with CMD ["npm", "start"], it is building an image that runs your application instead of running the database.
Running two things in one container is slightly tricky and usually isn't considered a best practice. (You change your application code so you build a new image and delete and recreate your existing container; do you actually want to stop and delete your database at the same time?) You should run this as two separate containers, one running the standard mongo image and a second one based on a Dockerfile similar to this but FROM node. You might look into Docker Compose as a simple orchestration tool that can manage both containers together.
The one other thing that's missing in your example is any configuration that tells the application where its database is. In Docker this is almost never localhost ("this container", not "this physical host somewhere"). You should add a control to pass that host name in as an environment variable. In Docker Compose you'd set it to the name of the services: block running the database.
version: '3'
services:
mongodb:
image: mongodb:4
volumes:
- './mongodb:/data/db'
app:
build: .
ports: '3000:3000'
env:
MONGODB_HOST: mongodb
(https://hub.docker.com/_/mongo is worth reading in detail.)

I can't connect to my mongo docker container by its name

I'm pretty new to docker but I'm having some issues getting a node app to connect to a mongo database running on a separate container.
I'm using the official mongo image
I run it using:
docker run --name some-mongo --network-alias some-mongo -d mongo
It's running on port 27017 by default. I can connect to it using the mongo shell:
mongo --host mongodb://172.17.0.2:27017
But I can't connect to it by name
mongo --host mongodb://some-mongo:27017
MongoDB shell version: 3.2.19
connecting to: mongodb://some-mongo:27017/test
2018-05-07T17:23:20.813-0400 I NETWORK [thread1] getaddrinfo("some-mongo") failed: Name or service not known
2018-05-07T17:23:20.813-0400 E QUERY [thread1] Error: couldn't initialize connection to host some-mongo, address is invalid :
connect#src/mongo/shell/mongo.js:223:14
#(connect):1:6
exception: connect failed
Instead I get an error message about how I can't connect to the mongo host:
I'm trying some docker-compose tutorials but either they're too simple or they don't seem to work for me. I just want to connect a custom node app, (not the official node) to mongodb and some other dependencies.
Your approach is not altering your host's system configuration, so that the mongo service will not be available just like that. Agreeing with #unm4sk, you should compose you application's services into a single compose file like this:
version: '2'
services:
mongo:
image: mongo
expose:
- "27017"
[...]
service_utilizing_mongo:
[...]
links:
- mongo:mongo
Then, your service_utilizing_mongo would have a DNS entry that'd make this service capable of accessing your mongo service via a alias mongo on a default 27017 port.
You have to run your container with passing ports to your host machine:
docker run -p 27017:27017 --name some-mongo --network-alias some-mongo -d mongo
Then you can connect to MongoDB from your host machine:
If you don't want to do this you can connect to mongo through docker container command
docker exec -it some-mongo mongo

mongoose connection failure using docker-compose

I have a Node express server consuming a Mongo database.
I'm trying to create a container for each of them using docker-compose.
Here's my docker-compose.yml file:
version: "2"
services:
server:
container_name: server
restart: always
build: .
ports:
- "3000:3000"
depends_on:
- db
db:
container_name: db
image: mongo
volumes:
- /var/lib/mongodb:/data/db
ports:
- "27017:27017"
And my Dockerfile:
FROM node:latest
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
COPY package.json /usr/src/app
RUN npm install
COPY . /usr/src/app
RUN npm run build-run
EXPOSE 3000
I saw on many tutorials that, when using Docker to create a Mongo container, the connection string should be updated in mongoose.connect to use Docker containers naming resolution.
So I changed my connection string according to my docker-compose file:
private readonly CONNECTION_STRING: String = 'mongodb://db/search-people-db'
public connect(): void {
mongoose.connect(this.CONNECTION_STRING)
this._db.on('error', (err) => {
console.log(`mongoose server failed to start: ${err}`)
})
this._db.once('open', () => {
console.log(`mongoose server running using ${this.CONNECTION_STRING}`)
})
}'
However, when running sudo docker-compose up, I keep getting the following error:
Mongoose server failed to start: MongoNetworkError: failed to connect to server [db:27017] on first connect [MongoNetworkError: getaddrinfo ENOTFOUND db db:27017]
What am I doing wrong ? Thanks in advance
MongoDB's container boots up but MongoDB itself needs more time start. so your application will not connect to it until it's fully started.
as Docker's documents suggested, you should set a wait time for your application and then run your code.
I suggest to make mongoose try to reconnect if couldn't connect at the first time or let the application crash if it couldn't connect. Docker will run your container again.
Replace depends_on with links in your docker-compose.yml and try to run command again.

Connection nodejs and mongo with docker

This is my docker-compose-file:
version: "2"
services:
rest-api:
build: .
ports:
- "3007:3007"
links:
- mongo
mongo:
image: mongo
volumes:
- /data/mongodb/db:/data/db
ports:
- "27017:27017"
This is my Dockerfile:
FROM mhart/alpine-node:latest
RUN rm -rf /tmp/node_modules
ADD package.json /tmp/package.json
RUN cd /tmp && npm install
RUN mkdir -p /opt/app && cp -a /tmp/node_modules /opt/app/
WORKDIR /opt/app
ADD . /opt/app
EXPOSE 3007
CMD ["node", "index.js"]
When I run the image, node is not able to connect to mongo:
MongoError: failed to connect to server [localhost:27017] on first connect
What can be wrong? How can I debug this?
Update:
In my node app I have:
let connection = mongoose.connect('mongodb://localhost/myapp');
Node is trying to connect to Mongo on localhost, which fails because it is not running in the same container as Node. Instead, it runs in a different container, i.e. on a different host.
You probably have a line like this in your application:
var url = 'mongodb://localhost:27017/myproject';
It defines where the MongoDB is located. You need to change localhost to the actual host, which in your case would be mongo.
In your node app you need to reference your container instead of localhost.
In your connection string to mongo use the name mongo as the host you are probably just using the default at the minute
https://docs.mongodb.com/manual/reference/connection-string/
e.g.
mongodb://mongo:27017/
You need to do this because the mongo database is in a separate container (with a different IP address) they are linked via TCP so localhost relates to the container node is in.

Resources