Contenerized expressjs app can't access mongo container - node.js

So, I have created an app using Express.js + Typescript + mongoose and this docker-compose for development:
version: "3.8"
services:
web:
container_name: selfapi
restart: unless-stopped
depends_on:
- db
build:
context: .
dockerfile: Dockerfile
target: development
ports:
- ${PORT}:${PORT}
environment:
MONGODB_URL: ${MONGODB_URL}
NODE_ENV: development
AUTH_TOKEN: ${AUTH_TOKEN}
volumes:
- ./src/:/home/node/app/src
- ./tests/:/home/node/app/tests
- web_nodemodules:/home/node/app/node_modules
db:
container_name: mongodb
image: mongo:5.0.4
restart: always
ports:
- 27017:27017
environment:
MONGO_INITDB_ROOT_USERNAME: ${MONGODB_ROOT_USERNAME}
MONGO_INITDB_ROOT_PASSWORD: ${MONGODB_ROOT_PASSWORD}
volumes:
- mongodb_data:/data/db/
- mongodb_configdb:/data/configdb/
mongo-express:
container_name: mongo-express
image: mongo-express:0.54.0
restart: always
depends_on:
- db
ports:
- 8081:8081
environment:
ME_CONFIG_MONGODB_ADMINUSERNAME: ${MONGODB_ROOT_USERNAME}
ME_CONFIG_MONGODB_ADMINPASSWORD: ${MONGODB_ROOT_PASSWORD}
ME_CONFIG_MONGODB_SERVER: db
ME_CONFIG_BASICAUTH_USERNAME: ${MONGO_EXPRESS_USERNAME}
ME_CONFIG_BASICAUTH_PASSWORD: ${MONGO_EXPRESS_PASSWORD}
volumes:
web_nodemodules:
mongodb_data:
mongodb_configdb:
Following that and assuming that both username and password for mongodb are root, we can get to the database url mongodb://root:root#db:27017 for any service within the compose, and mongodb://root:root#localhost:27017 for the host machine.
In my app, I connect to database this way:
import mongoose from 'mongoose';
export const connectMongoDb = async () => {
try {
return await mongoose.connect(process.env.MONGODB_URL as string);
} catch {
return process.exit(1);
}
};
But it only works if I don't add any database name to the url, so the url can't be something like mongodb://root:root#db:27017/myDatabase.
When I put only the raw mongodb url, and use the command ts-node-dev --poll --respawn --transpile-only --inspect -- ./src/index.ts, everything works fine as you will see below:
API:
But when I add something to my mongodb using the express app, it goes to a database called test, as you can see here in the mongo-express:
Now, let's say I don't want that to happen, intead, I want that the collection projects goes to a database calles MyApp, so I'd put it in the db url, becoming: mongodb://root:root#db:27017/MyApp. Following mongodb docs, if the database doesn't exist, it creates on first insertion, but this is what happens:
.env
API:
Apparently it is working, right? But no, take a look how it is on browser when I access the API:
"This page is not working"
"No data was sent by localhost"
That is happening just because I added the name I wanted that my database to have.
How can I fix that, changing the name, because it's REALLY annoying that all my data goes to a database calles TEST...
PS.: When my app is in a production environment, the url WITH the database name works, but the url is from mongodb Atlas, so it is something like this: mongodb+srv://test:test#db1.mongodb.net/GithubProjects?retryWrites=true&w=majority, as you can see, the database name is GithubProjects, and it works.

Related

MongoServerError: not authorized on app to execute command on docker container

I am using docker to connect node and mongo and I am trying to insert data in a database. All the containers are up and running. And it is running perfectly on my local machine but in the server I get the following error.
MongoServerError: not authorized on app to execute command { insert: "users", documents: [ { username: "riwaj", password: "$2a$12$C3hpChig42coIoMEbtegsepw7tJeflHqpW7x.0/jPseX6G5KUXWO.", _id: ObjectId('63d41dc11d038db2b950a744'), __v: 0 } ], ordered: true, lsid:....
This clearly states that the user riwaj is not allowed to perform insert operation on the database. However, I have defined the necessary attributes required for mongo container as mentioned in the documentation in my docker-compose file which are:
MONGO_INITDB_ROOT_USERNAME=riwaj
MONGO_INITDB_ROOT_PASSWORD=dummypasswordxx
The users are created as per the credentials but I checked it by going into the interactive shell of the container and executing the following command
mongosh -u riwaj -password
However even here if I try to insert data into a database using the mongo insert() function I get a similar error related to authorization.
For more reference here is my docker-compose files:
docker-compose.yml
version: "3"
services:
nginx:
image: nginx:stable-alpine
volumes:
- ./nginx/default.conf:/etc/nginx/conf.d/default.conf:ro
node-app:
build: .
environment:
- PORT=3000
depends_on:
- mongo
#adding mango container
mongo: #service name for mongo
image: mongo
environment:
- MONGO_INITDB_ROOT_USERNAME=riwaj
- MONGO_INITDB_ROOT_PASSWORD=mypassword
volumes:
- mongo-db:/data/db #Named volume for data persistance
#adding redis container
redis:
image: redis
volumes:
mongo-db:
docker-compose.prod.yml
version: "3"
services:
nginx:
ports:
- "80:80"
node-app:
build:
context: .
args:
NODE_ENV: production
environment:
- NODE_ENV=production
- MONGO_USER=${MONGO_USER}
- MONGO_PASSWORD=${MONGO_PASSWORD}
- SESSION_SECRET=${SESSION_SECRET}
command: node index.js
mongo:
environment:
- MONGO_INITDB_ROOT_USERNAME=${MONGO_INITDB_ROOT_USERNAME}
- MONGO_INITDB_ROOT_PASSWORD=${MONGO_INITDB_ROOT_PASSWORD}
- MONGO_INITDB_DATABASE= app
It is clear the issue is with authorization but shouldn't the root user have all the authorization to do all read-write operations? Here is the link to the repo https://github.com/Riwajchalise/node-docker where the project is pushed the endpoint for the signup api is mentioned in the Readme file
Would be really helpful if you can contribute in any way

Problems in connection between Node.js and MongoDB container using environment var. in Docker Compose

I am creating a simple backend service. I am trying to connect my Node.js container with MongoDB using env file.
This is my docker-compose file
version: '3.7'
services:
web-admin:
build:
context: .
dockerfile: dockerfile
image: node:14.16.0-alpine3.10
container_name: admin-service
restart: unless-stopped
env_file: .env
environment:
- MONGO_HOSTNAME=productDB
- MONGO_PORT=27017
- MONGO_DB=product
volumes:
- /home/dd/experiment/adminservice:/app
- /home/dd/experiment/adminservice/node_modules:/app/node_modules
depends_on:
- productDB
networks:
- admin-network
productDB:
image: mongo
ports:
- "27017:27017"
container_name: productDB
restart: unless-stopped
volumes:
- /productdata:/data/db
networks:
- admin-network
volumes:
productdata:
node_modules:
networks:
admin-network:
driver: bridge
and this is my .env file
MONGO_DB=product
MONGO_PORT=27017
and this is the code-block my index.js file.
also retry logic is working perfectly no error in it.
const {
MONGO_HOSTNAME,
MONGO_PORT,
MONGO_DB
} = process.env
//db retry logic
retries = 10;
while(retries)
{
try{
mongoose.connect('mongodb://${MONGO_HOSTNAME}:${MONGO_PORT}/${MONGO_DB}',
{
useNewUrlParser: true ,
keepAlive : true ,
}).then( function()
{
console.log('mongoDB is connected')
})
break;
}
catch(err)
{
console.log(err)
retries -= 1;
console.log('retries left: ${retries}');
}
}
now when I try to up the docker-compose using docker-compose up -d command my MongoDB container sets up nicely but the Node.js container is restarting every back and forth second. So I know that there is some error which is stopping the container but I'm not able to trace the mistake.
Please help me.
This is not how you use env vars with nodejs. You need to do
const MONGO_HOSTNAME = process.env.MONGO_HOSTNAME first.
This is not how you do retry logic with promises. Also, with promises you need to catch errors with .error() , not with try/catch
"docker logs [container id]" will help you see the error, so you might figure out whats wrong.
MONGO_HOSTNAME - try giving it as host.docker.internal instead of localhost if you using windows host machine.

Node-postgres hangs inside docker container when running in ubuntu server

I have an application running on several containers and I am using docker-compose to run these services inside containers. I am using NodeJS with postgresql using node-postgresql library(pg).
When I run the application in my local machine it works perfectly fine. And also when I build the application in a aws e2 ubuntu server , it builds without any error. And when I try to check the endpoints it works pretty much well. But when I try an endpoint that involves a database call, no response is received. I tried to debug the application by adding some console logs and I think the problem is with the pg library. When the code reaches pool.query it just hangs there without giving any error. I tried to console.log , try catch errors but everytime the code stops at a pool.query() and it hangs. Any idea why this is happening ? Tried for hours to find a solution online but no luck.
I found another similar issue here https://github.com/brianc/node-postgres/issues/2069
My docker-compose.yml
version: '3'
services:
postgres:
container_name: postgres
image: mdillon/postgis
environment:
POSTGRES_MULTIPLE_DATABASES: user_data,gov_auth
POSTGRES_USER: commhawk
POSTGRES_PASSWORD: password
volumes:
- postgres:/data/postgres
- ./database:/docker-entrypoint-initdb.d
ports:
- "5433:5432"
restart: unless-stopped
rethinkdb:
container_name: rethinkdb
image: rethinkdb:latest
ports:
- "3004:8080"
- "29015:29015"
- "28015:28015"
volumes:
- rethinkdb:/data
user_data_service:
build: './user_data_service'
container_name: uds
ports:
- "3002:3000"
depends_on:
- postgres
environment:
DATABASE_URL: postgres://commhawk:password#postgres:5432/user_data
# volumes:
# - ./user_data_service:/src
# - container_node_modules:/src/node_modules
gov_authority_service:
build: './gov_authority_service'
container_name: gov_auth
ports:
- "3001:3000"
depends_on:
- postgres
environment:
DATABASE_URL: postgres://commhawk:password#postgres:5432/gov_auth
# volumes:
# - ./gov_authority_service:/src
socket_service:
build: './socket_service'
container_name: socket
ports:
- "3003:3000"
depends_on:
- rethinkdb
# volumes:
# - ./socket_service:/src
api_gateway:
image: express-gateway:latest
ports:
- "8080:8080"
depends_on:
- user_data_service
- socket_service
- gov_authority_service
volumes:
- ./api_gateway/gateway.config.yml:/var/lib/eg/gateway.config.yml
volumes:
postgres:
rethinkdb:
# container_node_modules:
Simply when I run like following, it stops without any result. When I include that inside try catch, no error is shown also. No way to deubug
let result = await pool.query("SELECT NOW()")
console.log(result)
Node-postgresql connection
const {Pool} = require("pg");
const pool = new Pool({
user: "commhawk",
host: "postgres",
database: "gov_auth",
password: "password",
port: "5432"
});
module.exports = pool;
Tried it with connectionString also. Still no luck !!
Can provide more info if needed. Thanks in advance
UPDATE (SOLUTION)
Okay so I figured out the error. When I built it on my local machine, it is using nodejs 13 and pg 8.0.2. But when I tried to built it on my server(like 1 month after building on the local), since I am using node:latest in my Dockerfile, it pulls the latest version which is nodejs 14. Nodejs 14 is incompatible with pg 8.0.2. So that is why the local versions runs well and the server version fails.
Solutions:
Downgrade nodejs to 13 to use with pg 8.0.2
Upgrade pg to 8.0.3 to use with nodejs 14

How to make a second container not run until the first one is already started in the docker-compose?

I am trying to run to container through a docker compose using "docker-compose up" but i get a error.
i have to container one for my app nodejs and the other for my database mongo db and they are connected to each other.
version: "2"
services:
app:
container_name: sam-node
restart: always
image: amazus/sam-apis
ports:
- "3000:3000"
links:
- mongo
mongo:
container_name: sam-mongo
image: amazus/sam-data
ports:
- "27017:27017"
the error that i got is : ailed to connect to server [mongo:27017] on first connect
Add depends_on parameter
app:
container_name: sam-node
restart: always
image: amazus/sam-apis
ports:
- "3000:3000"
depends_on:
- mongo
links:
- mongo
Also, if you'd like to start dependent container after full initialisation of the parent one, you should add command parameter to check if parent container was initialised and built.
Read more here
Why would you ever need this?
Because some services, such as database, could initialise for some time, and if you have some specific logic which requires immediate response, it's better to start dependent container after initialisation completion of the parent one.

Cannot connect to mongodb container

Hi I'm trying to develop simple api but I couldn't establish connection between api and mongodb. When I send a query to database it hangs and couldn't get response. I am trying to connect to mongo container with this code:
mongoose.connect("mongodb://mongo/micro-linkedin_api_1");
Here is my docker-compose file:
version: "3"
services:
api:
restart: always
build: ./api
ports:
- "3000:3000"
links:
- mongodb
env_file:
- ./api/.env
volumes:
- ./api:/usr/xd/job-bot/api
mongodb:
image: mongo
ports:
- "27017:27017"
scraper:
build: ./scraper
and my folder structure is like this one :
As I can see in docker-compose you defined 2 services:
api and mongodb
in Mongoose docs, this is how you connect:
const mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/test');
You did:
mongoose.connect("mongodb://mongo/micro-linkedin_api_1");
so the host is not correct, you used mongo instead of mongodb as defined in docker-compose services.

Resources