Simple Node app and mongo containers created using docker-compose below... What am I missing?
mongodb://user:password#mongo:27017/
version: '3.8'
services:
mongo:
image: mongo
restart: always
environment:
- MONGO_INITDB_ROOT_USERNAME=user
- MONGO_INITDB_ROOT_PASSWORD=password
app:
image: app
build:
context: ./app
dockerfile: Dockerfile
ports:
- "3000:3000"
depends_on:
- mongo
I've read several posts on the same issue and the official mongo docker page and seem to be doing everything correct. Keep getting the following msg.
app_1 | mongodb://user:password#mongo:27017/ {
app_1 | autoIndex: false,
app_1 | poolSize: 10,
app_1 | bufferMaxEntries: 0,
app_1 | useNewUrlParser: true,
app_1 | useUnifiedTopology: true
app_1 | }
app_1 | MongoDB connection with retry
app_1 | MongoDB connection unsuccessful, retry after 5 seconds. 2
I prepared interesting test for you
version: '3.8'
services:
mongo:
image: mongo
environment:
- MONGO_INITDB_ROOT_USERNAME=user
- MONGO_INITDB_ROOT_PASSWORD=password
healthcheck:
test: "echo 'db.runCommand(\"ping\").ok'"
interval: 5s
timeout: 5s
retries: 3
app:
image: mongo
command: "mongosh mongodb://user:password#mongo:27017/admin --eval \"printjson(db.test.insertOne({'a': 1}))\""
ports:
- "3000:3000"
depends_on:
- mongo
It will print
app_1 | {
app_1 | acknowledged: true,
app_1 | insertedId: ObjectId("63696c4e99703eb4ab9fba62")
app_1 | }
but if you will change
command: "mongosh mongodb://user:password#mongo:27017/admin --eval \"printjson(db.test.insertOne({'a': 1}))\""
to
command: "mongosh mongodb://user:password#mongo:27017/local --eval \"printjson(db.test.insertOne({'a': 1}))\""
you will see error. Even if you will add MONGO_INITDB_DATABASE to mongo with value local.
You can test it running in second console command:
docker-compose run app bash
and then try
mongosh mongodb://user:password#mongo:27017/admin --eval "printjson(db.test.insertOne({'a': 1}))"
with success
and
mongosh mongodb://user:password#mongo:27017/local --eval "printjson(db.test.insertOne({'a': 1}))"
that failing. You can see logs from mongo like
mongo_1 | {"t":{"$date":"2022-11-07T20:40:35.686+00:00"},"s":"I", "c":"ACCESS", "id":20249, "ctx":"conn6","msg":"Authentication failed","attr":{"mechanism":"SCRAM-SHA-256","speculative":true,"principalName":"user","authenticationDatabase":"local","remote":"172.20.0.3:50382","extraInfo":{},"error":"UserNotFound: Could not find user \"user\" for db \"local\""}}
mongo_1 | {"t":{"$date":"2022-11-07T20:40:35.687+00:00"},"s":"I", "c":"ACCESS", "id":20249, "ctx":"conn6","msg":"Authentication failed","attr":{"mechanism":"SCRAM-SHA-1","speculative":false,"principalName":"user","authenticationDatabase":"local","remote":"172.20.0.3:50382","extraInfo":{},"error":"UserNotFound: Could not find user \"user\" for db \"local\""}}
mongo_1 | {"t":{"$date":"2022-11-07T20:40:35.688+00:00"},"s":"I", "c":"NETWORK", "id":22944, "ctx":"conn5","msg":"Connection ended","attr":{"remote":"172.20.0.3:50374","uuid":"b77fff1f-b832-4900-9c2d-1e7fd1e79424","connectionId":5,"connectionCount":1}}
mongo_1 | {"t":{"$date":"2022-11-07T20:40:35.699+00:00"},"s":"I", "c":"NETWORK", "id":22944, "ctx":"conn6","msg":"Connection ended","attr":{"remote":"172.20.0.3:50382","uuid":"3995bcbf-706d-4bed-92a2-04736305b7c2","connectionId":6,"connectionCount":0}}
this problem is described in topic User not found on MongoDB Docker image with authentication
You can authenticate with user,password against admin db, not mydb
You can read more about creating database with user and password here:
How to create a DB for MongoDB container on start up?
I'm trying to dockerize my current pet project in which I use a NodeJS (ExpressJS) as a backend, React as a frontend and PostgreSQL as a database. On both backend and frontend I use TypeScript instead of JavaScript. I'm also using a Prisma as ORM for my database. I decided to have a standard three container's architecture, one for backend, one for database and one for frontend app. My Dockerfile's are as follows:
Frontend's Dockerfile
FROM node:alpine
WORKDIR /usr/src/frontend
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 3000
CMD ["npm", "run", "start"]
Backend's Dockerfile
FROM node:lts
WORKDIR /usr/src/backend
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 8000
RUN npx prisma generate
CMD ["npm", "run", "dev"]
there's also a .dockerignore file in the backend folder:
node_modules/
and my docker-compose.yml looks like this:
version: '3.9'
services:
db:
image: 'postgres'
ports:
- '5432:5432'
environment:
POSTGRES_USER: 'postgres'
POSTGRES_PASSWORD: 'postgres'
POSTGRES_DB: 'hucuplant'
server:
build:
context: ./backend_express
ports:
- "8000:8000"
environment:
DATABASE_URL: 'postgresql://postgres:postgres#localhost:5432/hucuplant?schema=public'
client:
build:
context: ./frontend
ports:
- "3000:3000"
After doing a docker-compose up --build everything starts well but when I try to register a new user on my site then I get the following error:
Error:
hucuplant-server-1 | Invalid `prisma.user.findUnique()` invocation in
hucuplant-server-1 | /usr/src/backend/src/routes/Auth.ts:44:57
hucuplant-server-1 |
hucuplant-server-1 | 41 auth.post("/register", async (req: Request, res: Response) => {
hucuplant-server-1 | 42 const { email, username, password } = req.body;
hucuplant-server-1 | 43
hucuplant-server-1 | → 44 const usernameResult: User | null = await prisma.user.findUnique({
hucuplant-server-1 | where: {
hucuplant-server-1 | ? username?: String,
hucuplant-server-1 | ? id?: Int,
hucuplant-server-1 | ? email?: String
hucuplant-server-1 | }
hucuplant-server-1 | })
However, the existing code in my Auth.ts file on the line 44 looks like this:
auth.post("/register", async (req: Request, res: Response) => {
const { email, username, password } = req.body;
const usernameResult: User | null = await prisma.user.findUnique({
where: {
username: username,
},
});
When I run my project locally everything works just fine but when I try to run the containerized app then those things break and differ quite much. What is causing that? How do I fix that?
This issue has to do with the fact that the file exists on the backend container but not the postgres container. How could I transfer the file between containers automatically?
I am currently trying to execute the following script:
COPY climates(
station_id,
date,
element,
data_value,
m_flag,
q_flag,
s_flag,
obs_time
)
FROM '/usr/api/2017.csv`
DELIMITER ','
CSV HEADER;
within a docker container running a sequelize backend connecting to a postgres:14.1-alpine container.
The following error is returned:
db_1 | 2022-08-30 04:23:58.358 UTC [29] ERROR: could not open file "/usr/api/2017.csv" for reading: No such file or directory
db_1 | 2022-08-30 04:23:58.358 UTC [29] HINT: COPY FROM instructs the PostgreSQL server process to read a file. You may want a client-side facility such as psql's \copy.
db_1 | 2022-08-30 04:23:58.358 UTC [29] STATEMENT: COPY climates(
db_1 | station_id,
db_1 | date,
db_1 | element,
db_1 | data_value,
db_1 | m_flag,
db_1 | q_flag,
db_1 | s_flag,
db_1 | obs_time
db_1 | )
db_1 | FROM '/usr/api/2017.csv'
db_1 | DELIMITER ','
db_1 | CSV HEADER;
ebapi | Unable to connect to the database: MigrationError: Migration 20220829_02_populate_table.js (up) failed: Original error: could not open file "/usr/api/2017.csv" for reading: No such file or directory
ebapi | at /usr/api/node_modules/umzug/lib/umzug.js:151:27
ebapi | at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
ebapi | at async Umzug.runCommand (/usr/api/node_modules/umzug/lib/umzug.js:107:20)
ebapi | ... 2 lines matching cause stack trace ...
ebapi | at async start (/usr/api/index.js:14:3) {
ebapi | cause: Error
ebapi | at Query.run (/usr/api/node_modules/sequelize/lib/dialects/postgres/query.js:50:25)
ebapi | at /usr/api/node_modules/sequelize/lib/sequelize.js:311:28
ebapi | at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
ebapi | at async Object.up (/usr/api/migrations/20220829_02_populate_table.js:10:5)
ebapi | at async /usr/api/node_modules/umzug/lib/umzug.js:148:21
ebapi | at async Umzug.runCommand (/usr/api/node_modules/umzug/lib/umzug.js:107:20)
ebapi | at async runMigrations (/usr/api/util/db.js:52:22)
ebapi | at async connectToDatabase (/usr/api/util/db.js:32:5)
ebapi | at async start (/usr/api/index.js:14:3) {
ebapi | name: 'SequelizeDatabaseError',
...
Here is my docker-compose.yml
# set up a postgres database version: "3.8" services: db:
image: postgres:14.1-alpine
restart: always
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
ports:
- "5432:5432"
volumes:
- db:/var/lib/postgresql/data
- ./db/init.sql:/docker-entrypoint-initdb.d/create_tables.sql api:
container_name: ebapi
build:
context: ./energybot
depends_on:
- db
ports:
- 3001:3001
environment:
DATABASE_URL: postgres://postgres:postgres#db:5432/postgres
DB_HOST: db
DB_PORT: 5432
DB_USER: postgres
DB_PASSWORD: postgres
DB_NAME: postgres
links:
- db
volumes:
- "./energybot:/usr/api"
volumes: db:
driver: local
I am trying to build a NestJS app with Prisma and PostgreSQL. I want to use docker; however, I got an error when I sent the request to the backend.
Here is my docker file
FROM node:14 AS builder
WORKDIR /app
COPY package*.json ./
COPY prisma ./prisma/
RUN npm install
RUN npx prisma generate
COPY . .
RUN npm run build
FROM node:14
COPY --from=builder /app/node_modules ./node_modules
COPY --from=builder /app/package*.json ./
COPY --from=builder /app/dist ./dist
EXPOSE 3000
CMD [ "npm", "run", "start:prod" ]
Here is my docker-compose.yml
version: '3.8'
services:
nest-api:
container_name: nest-api
build:
context: .
dockerfile: Dockerfile
ports:
- 3000:3000
depends_on:
- postgres
env_file:
- .env
postgres:
image: postgres:13
container_name: postgres
restart: always
ports:
- 5432:5432
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: task-management
env_file:
- .env
Here is my schema.prisma
// This is your Prisma schema file,
// learn more about it in the docs: https://pris.ly/d/prisma-schema
generator client {
provider = "prisma-client-js"
}
datasource db {
provider = "postgresql"
url = env("DATABASE_URL")
//url = "postgresql://postgres:postgres#localhost:5432/task-management?schema=public"
}
model Task {
id Int #id #default(autoincrement())
title String
description String
status TaskStatus #default(OPEN)
}
enum TaskStatus {
OPEN
IN_PRO
DOooNE
}
Here is the .env
# Environment variables declared in this file are automatically made available to Prisma.
# See the documentation for more detail: https://pris.ly/d/prisma-schema#using-environment-variables
# Prisma supports the native connection string format for PostgreSQL, MySQL, SQLite, SQL Server and MongoDB (Preview).
# See the documentation for all the connection string options: https://pris.ly/d/connection-strings
DATABASE_URL=postgresql://postgres:postgres#postgres:5432/task-management?schema=public
After I run the command:docker-compose up, everything is fine. However, if I send the request to the app, I get the following error:
nest-api | [Nest] 19 - 11/02/2021, 5:52:43 AM ERROR [ExceptionsHandler]
nest-api | Invalid `this.prisma.task.create()` invocation in
nest-api | /dist/tasks/tasks.service.js:29:33
nest-api |
nest-api | 26 return found;
nest-api | 27 }
nest-api | 28 async creatTask(data) {
nest-api | → 29 return this.prisma.task.create(
nest-api | The table `public.Task` does not exist in the current database.
nest-api | Error:
nest-api | Invalid `this.prisma.task.create()` invocation in
nest-api | /dist/tasks/tasks.service.js:29:33
nest-api |
nest-api | 26 return found;
nest-api | 27 }
nest-api | 28 async creatTask(data) {
nest-api | → 29 return this.prisma.task.create(
nest-api | The table `public.Task` does not exist in the current database.
nest-api | at cb (/node_modules/#prisma/client/runtime/index.js:38537:17)
nest-api | at async /node_modules/#nestjs/core/router/router-execution-context.js:46:28
nest-api | at async /node_modules/#nestjs/core/router/router-proxy.js:9:17
What changes should I make in the docker file to solve the problem?
What I'm trying to do
I'm trying to set up a docker-compose definition, where I have a mongoDB container, and a nodeJS container that connects to it.
version: "3.9"
services:
events-db:
image: mongo
volumes:
- db-volume:/data/db
environment:
MONGO_INITDB_ROOT_USERNAME: $SANDBOX_DB_USER
MONGO_INITDB_ROOT_PASSWORD: $SANDBOX_DB_PASS
MONGO_INITDB_DATABASE: sandboxdb
app:
image: node:15.12.0
user: node
working_dir: /home/node/app
volumes:
- ./:/home/node/app:ro
environment:
MDB_CONNECTION: mongodb://$SANDBOX_DB_USER:$SANDBOX_DB_PASS#events-db:27017/sandboxdb
command: node myapp
depends_on:
- events-db
volumes:
db-volume:
Along with a .env file that declares the credentials (planning to use proper env variables when I deploy this to a production environment):
SANDBOX_DB_USER=myuser
SANDBOX_DB_PASS=myp4ss
Finally, my nodejs script, myapp.js is simply trying to connect, grab a reference to a collection, and insert a document:
require('dotenv').config()
const { MongoClient } = require('mongodb')
async function main () {
console.log('Connecting')
const client = new MongoClient(process.env.MDB_CONNECTION, {
connectTimeoutMS: 10000,
useUnifiedTopology: true,
})
await client.connect()
const db = client.db()
const events = db.collection('events')
console.log('Inserting an event')
await events.insertOne({
type: 'foo',
timestamp: new Date(),
})
console.log('Done.')
process.exit(0)
}
if (require.main === module) {
main()
}
Result
When I run docker-compose config I see the following output, so I would expect it to work:
$ docker-compose config
services:
app:
command: node myapp
depends_on:
events-db:
condition: service_started
environment:
MDB_CONNECTION: mongodb://myuser:myp4ss#events-db:27017/sandboxdb
image: node:15.12.0
user: node
volumes:
- C:\workspace\dcsandbox:/home/node/app:ro
working_dir: /home/node/app
events-db:
environment:
MONGO_INITDB_DATABASE: sandboxdb
MONGO_INITDB_ROOT_PASSWORD: myp4ss
MONGO_INITDB_ROOT_USERNAME: myuser
image: mongo
volumes:
- db-volume:/data/db:rw
version: '3.9'
volumes:
db-volume: {}
However, when I run docker-compose up I see that my node container is unable to connect to the mongoDB to insert an event:
events-db_1 | {"t":{"$date":"2021-04-07T13:57:36.793+00:00"},"s":"I", "c":"NETWORK", "id":23016, "ctx":"listener","msg":"Waiting for connections","attr":{"port":27017,"ssl":"off"}}
app_1 | Connecting
events-db_1 | {"t":{"$date":"2021-04-07T13:57:38.811+00:00"},"s":"I", "c":"NETWORK", "id":22943, "ctx":"listener","msg":"Connection accepted","attr":{"remote":"172.27.0.3:34164","connectionId":1,"connectionCount":1}}
events-db_1 | {"t":{"$date":"2021-04-07T13:57:38.816+00:00"},"s":"I", "c":"NETWORK", "id":51800, "ctx":"conn1","msg":"client metadata","attr":{"remote":"172.27.0.3:34164","client":"conn1","doc":{"driver":{"name":"nodejs","version":"3.6.6"},"os":{"type":"Linux","name":"linux","architecture":"x64","version":"4.19.128-microsoft-standard"},"platform":"'Node.js v15.12.0, LE (unified)"}}}
events-db_1 | {"t":{"$date":"2021-04-07T13:57:38.820+00:00"},"s":"I", "c":"NETWORK", "id":22943, "ctx":"listener","msg":"Connection accepted","attr":{"remote":"172.27.0.3:34166","connectionId":2,"connectionCount":2}}
events-db_1 | {"t":{"$date":"2021-04-07T13:57:38.822+00:00"},"s":"I", "c":"NETWORK", "id":51800, "ctx":"conn2","msg":"client metadata","attr":{"remote":"172.27.0.3:34166","client":"conn2","doc":{"driver":{"name":"nodejs","version":"3.6.6"},"os":{"type":"Linux","name":"linux","architecture":"x64","version":"4.19.128-microsoft-standard"},"platform":"'Node.js v15.12.0, LE (unified)"}}}
events-db_1 | {"t":{"$date":"2021-04-07T13:57:38.822+00:00"},"s":"I", "c":"ACCESS", "id":20251, "ctx":"conn2","msg":"Supported SASL mechanisms requested for unknown user","attr":{"user":"myuser#sandboxdb"}}
events-db_1 | {"t":{"$date":"2021-04-07T13:57:38.823+00:00"},"s":"I", "c":"ACCESS", "id":20249, "ctx":"conn2","msg":"Authentication failed","attr":{"mechanism":"SCRAM-SHA-256","principalName":"myuser","authenticationDatabase":"sandboxdb","client":"172.27.0.3:34166","result":"UserNotFound: Could not find user \"myuser\" for db \"sandboxdb\""}}
events-db_1 | {"t":{"$date":"2021-04-07T13:57:38.824+00:00"},"s":"I", "c":"ACCESS", "id":20249, "ctx":"conn2","msg":"Authentication failed","attr":{"mechanism":"SCRAM-SHA-1","principalName":"myuser","authenticationDatabase":"sandboxdb","client":"172.27.0.3:34166","result":"UserNotFound: Could not find user \"myuser\" for db \"sandboxdb\""}}
events-db_1 | {"t":{"$date":"2021-04-07T13:57:38.826+00:00"},"s":"I", "c":"NETWORK", "id":22944, "ctx":"conn1","msg":"Connection ended","attr":{"remote":"172.27.0.3:34164","connectionId":1,"connectionCount":1}}
app_1 | /home/node/app/node_modules/mongodb/lib/cmap/connection.js:268
app_1 | callback(new MongoError(document));
app_1 | ^
app_1 |
app_1 | MongoError: Authentication failed.
app_1 | at MessageStream.messageHandler (/home/node/app/node_modules/mongodb/lib/cmap/connection.js:268:20)
app_1 | at MessageStream.emit (node:events:369:20)
app_1 | at processIncomingData (/home/node/app/node_modules/mongodb/lib/cmap/message_stream.js:144:12)
app_1 | at MessageStream._write (/home/node/app/node_modules/mongodb/lib/cmap/message_stream.js:42:5)
app_1 | at writeOrBuffer (node:internal/streams/writable:395:12)
app_1 | at MessageStream.Writable.write (node:internal/streams/writable:340:10)
app_1 | at Socket.ondata (node:internal/streams/readable:750:22)
app_1 | at Socket.emit (node:events:369:20)
app_1 | at addChunk (node:internal/streams/readable:313:12)
app_1 | at readableAddChunk (node:internal/streams/readable:288:9) {
app_1 | ok: 0,
app_1 | code: 18,
app_1 | codeName: 'AuthenticationFailed'
app_1 | }
events-db_1 | {"t":{"$date":"2021-04-07T13:57:38.832+00:00"},"s":"I", "c":"NETWORK", "id":22944, "ctx":"conn2","msg":"Connection ended","attr":{"remote":"172.27.0.3:34166","connectionId":2,"connectionCount":0}}
dcsandbox_app_1 exited with code 1
I've put the full output at https://pastebin.com/uNyJ6tiy
and the example code at this repo: https://github.com/akatechis/example-docker-compose-mongo-node-auth
After some more digging, I managed to figure it out. The issue is that the MONGO_INITDB_ROOT_USERNAME and MONGO_INITDB_ROOT_PASSWORD variables simply set the root user's credentials, and the MONGO_INITDB_DATABASE simply sets the initial database for scripts in /docker-entrypoint-initdb.d.
By default, the root user is added to the admin database, so by removing the /sandboxdb part of the connection string, I was able to have my node app authenticate against the admin DB as the root user.
While this doesn't quite accomplish what I wanted initially (to create a separate, non-root user for my database, and use that to authenticate), I think this puts me on the right path to using an init script to set up the user accounts I want to have.