I have 2 dockers composes. 1 implements an infrastructure with a reverse proxy, databases, database admins, etc.
version: '3.7'
volumes:
mysql80_volume:
name: MySql80-Volume
services:
reverse-proxy:
image: traefik:v2.2
container_name: "reverse-proxy"
restart: always
network_mode: bridge
ports:
- "80:80"
- "443:443"
# etc...
mysql80:
container_name: mysql80
image: mysql:8.0
restart: always
labels:
- traefik.enable=false
network_mode: bridge
ports:
- 3380:3306
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_PASSWORD=root
volumes:
- mysql80_volume:/var/lib/mysql
- ./my.cnf:/etc/my.cnf
And there is a docker-compose service, which describes the configs for the service.
version: '3.7'
services:
server:
container_name: project-api.com
build:
context: .
dockerfile: docker/Dockerfile
sysctls:
- net.ipv4.ip_unprivileged_port_start=0
restart: always
labels:
- traefik.enable=true
#labels for traefik
extra_hosts:
- "project-api.com www.project-api.com:127.0.0.1"
external_links:
- mysql80:mysql # <---- My link
network_mode: bridge
volumes:
- .:/app
I want to connect the database from docker-compose 1 to the docker-compose 2 file and pass it to the connection in the Typeform config.
Config for typeorm
type: "mysql",
host: process.env.TYPEORM_ENDPOINT,
database: process.env.TYPEORM_DB_NAME,
port: parseInt(process.env.TYPEORM_PORT) || 3306,
username: process.env.TYPEORM_USERNAME,
password: process.env.TYPEORM_PASSWORD,
.env
TYPEORM_ENDPOINT=mysql
TYPEORM_DB_NAME=red_driver
TYPEORM_USERNAME=root
TYPEORM_PASSWORD=root
TYPEORM_PORT=3306
But I am getting an error.
project-api.com | [nodemon] 2.0.7
project-api.com | [nodemon] to restart at any time, enter `rs`
project-api.com | [nodemon] watching path(s): *.*
project-api.com | [nodemon] watching extensions: ts,graphql
project-api.com | [nodemon] starting `ts-node index.ts`
project-api.com | Error: getaddrinfo ENOTFOUND mysql
project-api.com | at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:71:26)
project-api.com | --------------------
project-api.com | at Protocol._enqueue (/home/app/node_modules/mysql/lib/protocol/Protocol.js:144:48)
project-api.com | at Protocol.handshake (/home/app/node_modules/mysql/lib/protocol/Protocol.js:51:23)
project-api.com | at PoolConnection.connect (/home/app/node_modules/mysql/lib/Connection.js:116:18)
project-api.com | at Pool.getConnection (/home/app/node_modules/mysql/lib/Pool.js:48:16)
project-api.com | at /home/app/src/driver/mysql/MysqlDriver.ts:894:18
project-api.com | at new Promise (<anonymous>)
project-api.com | at MysqlDriver.createPool (/home/app/src/driver/mysql/MysqlDriver.ts:891:16)
project-api.com | at MysqlDriver.<anonymous> (/home/app/src/driver/mysql/MysqlDriver.ts:344:36)
project-api.com | at step (/home/app/node_modules/typeorm/node_modules/tslib/tslib.js:141:27)
project-api.com | at Object.next (/home/app/node_modules/typeorm/node_modules/tslib/tslib.js:122:57) {
project-api.com | errno: -3008,
project-api.com | code: 'ENOTFOUND',
project-api.com | syscall: 'getaddrinfo',
project-api.com | hostname: 'mysql',
project-api.com | fatal: true
project-api.com | }
The config property of typeorm must be the container name of MySQL. (host: 'mysql80')
i created a nodejs app which should use a URI to connect to rabbitmq. both are containerized with docker and are created by a docker-compose file. after running of "docker-compose up" the nodejs app returns an error:
Error: connect ECONNREFUSED X.X.X:X:5672
at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1133:16) {
errno: -111,
code: 'ECONNREFUSED',
syscall: 'connect',
address: '172.26.0.4',
port: 5672
}
when starting the api-server locally (not as a container-> as an node application), the connection to the containerized rabbitmq server estabilish without any problems.
my rabbitmq.conf file looks like:
default_vhost = /
default_user = guest
default_pass = guest
default_user_tags.administrator = true
default_permissions.configure = .*
default_permissions.read = .*
default_permissions.write = .*
loopback_users = none
listeners.tcp.default = 5672
management.listener.port = 15672
management.listener.ssl = false
management.load_definitions = /etc/rabbitmq/definitions.json
URI for connecting:
{
"mongoURI":"mongodb://mongo:27017",
"amqpURI": "amqp://guest:guest#rabbitmq:5672"
}
as you can see, the hostname is equal to the one, which is within the docker-compose file
finally the docker-compose file:
version: "3.8"
services:
react-app:
image: react-app
stdin_open: true
ports:
- "3000:3000"
networks:
- mern-app
api-server:
image: api-server
ports:
- "5000:5000"
networks:
- mern-app
depends_on:
- mongo
- rabbitmq
process-schedular:
image: process-schedular
ports:
- "5005:5005"
networks:
- mern-app
depends_on:
- mongo
- rabbitmq
mongo:
image: mongo:3.6.19-xenial
ports:
- "27017:27017"
networks:
- mern-app
volumes:
- mongo-data:/data/db
rabbitmq:
image: rabbitmq:3-management
hostname: rabbitmq
volumes:
- ./server/amqp/docker/enabled_plugins:/etc/rabbitmq/enabled_plugins
- ./server/amqp/docker/definitions.json:/etc/rabbitmq/definitions.json
- ./server/amqp/docker/rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf
ports:
- "5672:5672"
- "15672:15672"
networks:
- mern-app
networks:
mern-app:
driver: bridge
volumes:
mongo-data:
driver: local
Do you know why I get the following error when I try to run typeorm:run to execute migration?
node --require ts-node/register ./node_modules/typeorm/cli.js migration:run
Error during migration run:
Error: getaddrinfo ENOTFOUND users-service-db
at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:69:26) {
errno: -3008,
code: 'ENOTFOUND',
syscall: 'getaddrinfo',
hostname: 'users-service-db',
fatal: true
}
error Command failed with exit code 1.
my config is
users-service-db:
environment:
- MYSQL_ROOT_PASSWORD=password
- MYSQL_DATABASE=db
image: mysql:5.7.20
ports:
- "7201:3306"
the users-service-db is running does this Error: getaddrinfo ENOTFOUND users-service-db say that the host doesn't know what to do. Can you help?
After trying Answer 1 and 2 still getting the same error don't know what to do it worked before?
version: "3"
services:
api-gateway:
build:
context: "."
dockerfile: "./api-gateway/Dockerfile"
depends_on:
- chat-service
- users-service
ports:
- "7000:7000"
volumes:
- ./api-gateway:/opt/app
chat-service:
build:
context: "."
dockerfile: "./chat-service/Dockerfile"
depends_on:
- chat-service-db
ports:
- "7100:7100"
volumes:
- ./chat-service:/opt/app
chat-service-db:
environment:
- MYSQL_ROOT_PASSWORD=password
- MYSQL_DATABASE=db
image: mysql:5.7.20
ports:
- "7200:3306"
phpmyadmin:
image: phpmyadmin/phpmyadmin
ports:
- "7300:80"
volumes:
- ./phpmyadmin/config.user.inc.php:/etc/phpmyadmin/config.user.inc.php
users-service:
build:
context: "."
dockerfile: "./users-service/Dockerfile"
depends_on:
- users-service-db
ports:
- "7101:7101"
volumes:
- ./users-service:/opt/app
users-service-db:
environment:
- MYSQL_ROOT_PASSWORD=password
- MYSQL_DATABASE=db
image: mysql:5.7.20
ports:
- "7201:3306"
hostname: 'localhost'
finally I resolved the error thanks to #Eranga Heshan
I created an additional ormConfig.js file at pasted this:
export = {
"type": "mysql",
"host": "localhost",
"port": 7201,
"username": "root",
"password": "password",
"database": "db",
"synchronize": true,
"logging": false,
"entities": [
"src/entities/**/*.ts"
],
"migrations": [
"./src/db/migrations/**/*.ts"
],
"cli": {
"entitiesDir": "src/db/entities",
"migrationsDir": "src/db/migrations"
}
}
then
node --require ts-node/register ./node_modules/typeorm/cli.js migration:run --config src/db/migrations/ormConfig
Your VS Code terminal is running inside your machine. So it can't resolve users-service-db host.
You can do this in two ways.
1. Using a new config file and execute migrations from your localhost
Create a new typeorm connection config file migrationsOrmConfig.ts and put it inside your project (Let's say you put it in src/migrations directory)
export = {
host: 'localhost',
port: '7201',
type: 'mysql',
user : 'root',
password : 'password',
database : 'db' ,
};
Now you can modify the command you used earlier to run migrations
node --require ts-node/register ./node_modules/typeorm/cli.js migration:run --config src/migrations/migrationsOrmConfig
2. Execute migrations from a terminal within the container
In your VSCode terminal type
docker ps -a
Get the CONTAINER ID of user-service (Let's say it is CONTAINER_ID)
Open up a terminal inside the container
docker exec -it CONTAINER_ID /bin/bash
Execute the command you used earlier to run migrations (if the following command complained about typeorm node module not being found, you can install it inside the container)
node --require ts-node/register ./node_modules/typeorm/cli.js migration:run
What I'm trying to do
I'm trying to set up a docker-compose definition, where I have a mongoDB container, and a nodeJS container that connects to it.
version: "3.9"
services:
events-db:
image: mongo
volumes:
- db-volume:/data/db
environment:
MONGO_INITDB_ROOT_USERNAME: $SANDBOX_DB_USER
MONGO_INITDB_ROOT_PASSWORD: $SANDBOX_DB_PASS
MONGO_INITDB_DATABASE: sandboxdb
app:
image: node:15.12.0
user: node
working_dir: /home/node/app
volumes:
- ./:/home/node/app:ro
environment:
MDB_CONNECTION: mongodb://$SANDBOX_DB_USER:$SANDBOX_DB_PASS#events-db:27017/sandboxdb
command: node myapp
depends_on:
- events-db
volumes:
db-volume:
Along with a .env file that declares the credentials (planning to use proper env variables when I deploy this to a production environment):
SANDBOX_DB_USER=myuser
SANDBOX_DB_PASS=myp4ss
Finally, my nodejs script, myapp.js is simply trying to connect, grab a reference to a collection, and insert a document:
require('dotenv').config()
const { MongoClient } = require('mongodb')
async function main () {
console.log('Connecting')
const client = new MongoClient(process.env.MDB_CONNECTION, {
connectTimeoutMS: 10000,
useUnifiedTopology: true,
})
await client.connect()
const db = client.db()
const events = db.collection('events')
console.log('Inserting an event')
await events.insertOne({
type: 'foo',
timestamp: new Date(),
})
console.log('Done.')
process.exit(0)
}
if (require.main === module) {
main()
}
Result
When I run docker-compose config I see the following output, so I would expect it to work:
$ docker-compose config
services:
app:
command: node myapp
depends_on:
events-db:
condition: service_started
environment:
MDB_CONNECTION: mongodb://myuser:myp4ss#events-db:27017/sandboxdb
image: node:15.12.0
user: node
volumes:
- C:\workspace\dcsandbox:/home/node/app:ro
working_dir: /home/node/app
events-db:
environment:
MONGO_INITDB_DATABASE: sandboxdb
MONGO_INITDB_ROOT_PASSWORD: myp4ss
MONGO_INITDB_ROOT_USERNAME: myuser
image: mongo
volumes:
- db-volume:/data/db:rw
version: '3.9'
volumes:
db-volume: {}
However, when I run docker-compose up I see that my node container is unable to connect to the mongoDB to insert an event:
events-db_1 | {"t":{"$date":"2021-04-07T13:57:36.793+00:00"},"s":"I", "c":"NETWORK", "id":23016, "ctx":"listener","msg":"Waiting for connections","attr":{"port":27017,"ssl":"off"}}
app_1 | Connecting
events-db_1 | {"t":{"$date":"2021-04-07T13:57:38.811+00:00"},"s":"I", "c":"NETWORK", "id":22943, "ctx":"listener","msg":"Connection accepted","attr":{"remote":"172.27.0.3:34164","connectionId":1,"connectionCount":1}}
events-db_1 | {"t":{"$date":"2021-04-07T13:57:38.816+00:00"},"s":"I", "c":"NETWORK", "id":51800, "ctx":"conn1","msg":"client metadata","attr":{"remote":"172.27.0.3:34164","client":"conn1","doc":{"driver":{"name":"nodejs","version":"3.6.6"},"os":{"type":"Linux","name":"linux","architecture":"x64","version":"4.19.128-microsoft-standard"},"platform":"'Node.js v15.12.0, LE (unified)"}}}
events-db_1 | {"t":{"$date":"2021-04-07T13:57:38.820+00:00"},"s":"I", "c":"NETWORK", "id":22943, "ctx":"listener","msg":"Connection accepted","attr":{"remote":"172.27.0.3:34166","connectionId":2,"connectionCount":2}}
events-db_1 | {"t":{"$date":"2021-04-07T13:57:38.822+00:00"},"s":"I", "c":"NETWORK", "id":51800, "ctx":"conn2","msg":"client metadata","attr":{"remote":"172.27.0.3:34166","client":"conn2","doc":{"driver":{"name":"nodejs","version":"3.6.6"},"os":{"type":"Linux","name":"linux","architecture":"x64","version":"4.19.128-microsoft-standard"},"platform":"'Node.js v15.12.0, LE (unified)"}}}
events-db_1 | {"t":{"$date":"2021-04-07T13:57:38.822+00:00"},"s":"I", "c":"ACCESS", "id":20251, "ctx":"conn2","msg":"Supported SASL mechanisms requested for unknown user","attr":{"user":"myuser#sandboxdb"}}
events-db_1 | {"t":{"$date":"2021-04-07T13:57:38.823+00:00"},"s":"I", "c":"ACCESS", "id":20249, "ctx":"conn2","msg":"Authentication failed","attr":{"mechanism":"SCRAM-SHA-256","principalName":"myuser","authenticationDatabase":"sandboxdb","client":"172.27.0.3:34166","result":"UserNotFound: Could not find user \"myuser\" for db \"sandboxdb\""}}
events-db_1 | {"t":{"$date":"2021-04-07T13:57:38.824+00:00"},"s":"I", "c":"ACCESS", "id":20249, "ctx":"conn2","msg":"Authentication failed","attr":{"mechanism":"SCRAM-SHA-1","principalName":"myuser","authenticationDatabase":"sandboxdb","client":"172.27.0.3:34166","result":"UserNotFound: Could not find user \"myuser\" for db \"sandboxdb\""}}
events-db_1 | {"t":{"$date":"2021-04-07T13:57:38.826+00:00"},"s":"I", "c":"NETWORK", "id":22944, "ctx":"conn1","msg":"Connection ended","attr":{"remote":"172.27.0.3:34164","connectionId":1,"connectionCount":1}}
app_1 | /home/node/app/node_modules/mongodb/lib/cmap/connection.js:268
app_1 | callback(new MongoError(document));
app_1 | ^
app_1 |
app_1 | MongoError: Authentication failed.
app_1 | at MessageStream.messageHandler (/home/node/app/node_modules/mongodb/lib/cmap/connection.js:268:20)
app_1 | at MessageStream.emit (node:events:369:20)
app_1 | at processIncomingData (/home/node/app/node_modules/mongodb/lib/cmap/message_stream.js:144:12)
app_1 | at MessageStream._write (/home/node/app/node_modules/mongodb/lib/cmap/message_stream.js:42:5)
app_1 | at writeOrBuffer (node:internal/streams/writable:395:12)
app_1 | at MessageStream.Writable.write (node:internal/streams/writable:340:10)
app_1 | at Socket.ondata (node:internal/streams/readable:750:22)
app_1 | at Socket.emit (node:events:369:20)
app_1 | at addChunk (node:internal/streams/readable:313:12)
app_1 | at readableAddChunk (node:internal/streams/readable:288:9) {
app_1 | ok: 0,
app_1 | code: 18,
app_1 | codeName: 'AuthenticationFailed'
app_1 | }
events-db_1 | {"t":{"$date":"2021-04-07T13:57:38.832+00:00"},"s":"I", "c":"NETWORK", "id":22944, "ctx":"conn2","msg":"Connection ended","attr":{"remote":"172.27.0.3:34166","connectionId":2,"connectionCount":0}}
dcsandbox_app_1 exited with code 1
I've put the full output at https://pastebin.com/uNyJ6tiy
and the example code at this repo: https://github.com/akatechis/example-docker-compose-mongo-node-auth
After some more digging, I managed to figure it out. The issue is that the MONGO_INITDB_ROOT_USERNAME and MONGO_INITDB_ROOT_PASSWORD variables simply set the root user's credentials, and the MONGO_INITDB_DATABASE simply sets the initial database for scripts in /docker-entrypoint-initdb.d.
By default, the root user is added to the admin database, so by removing the /sandboxdb part of the connection string, I was able to have my node app authenticate against the admin DB as the root user.
While this doesn't quite accomplish what I wanted initially (to create a separate, non-root user for my database, and use that to authenticate), I think this puts me on the right path to using an init script to set up the user accounts I want to have.
my docker-compose.yml
version: '3.5' # specify docker-compose version
# Define the services/ containers to be run
services:
angular: # name of the first service
build: frontend # specify the directory of the Dockerfile
ports:
- "80:80" # specify port mapping
express: # name of the second service
build: backend # specify the directory of the Dockerfile
restart: always
ports:
- "100:3000" #specify ports mapping
links:
- mongo # link this service to the database service
mongo: # name of the third service
image: mongo # specify image to build container from
container_name: mongo
ports:
- "27017:27017" # specify port forwarding
when i issue docker-compose up express
in server its showing error in this line
mongoose.connect('mongodb://mongo:27017/mydb', {useNewUrlParser: true})
error is
express_1 | { MongoNetworkError: failed to connect to server [mongo:27017] on first connect [MongoNetworkError: connect ECONNREFUSED 172.18.0.2:27017]
express_1 | at Pool.<anonymous> (/usr/src/app/node_modules/mongodb/lib/core/topologies/server.js:438:11)
express_1 | at Pool.emit (events.js:198:13)
express_1 | at createConnection (/usr/src/app/node_modules/mongodb/lib/core/connection/pool.js:561:14)
express_1 | at connect (/usr/src/app/node_modules/mongodb/lib/core/connection/pool.js:994:11)
express_1 | at makeConnection (/usr/src/app/node_modules/mongodb/lib/core/connection/connect.js:31:7)
express_1 | at Socket.err (/usr/src/app/node_modules/mongodb/lib/core/connection/connect.js:294:7)
express_1 | at Object.onceWrapper (events.js:286:20)
express_1 | at Socket.emit (events.js:198:13)
express_1 | at emitErrorNT (internal/streams/destroy.js:91:8)
express_1 | at emitErrorAndCloseNT (internal/streams/destroy.js:59:3)
express_1 | at process._tickCallback (internal/process/next_tick.js:63:19)
express_1 | name: 'MongoNetworkError',
express_1 | [Symbol(mongoErrorContextSymbol)]: {} }
why its trying to connect 172.18.0.2:27017 instead of localhost:27017 or docker(192.168.99.100:27017)
thanks in advance scratching my head from last 2 days