I'm trying to set up my local development environment to the point that all my Node.js applications are developed inside a docker container. Our team works on Linux, macOS, and Windows so this should help us limit some of the issues that we see due to this.
We're using Sails.js for our Node framework, and I'm not sure if the issue is in my Docker setup, or an issue with Sails itself.
Here's my docker run command, which almost works:
docker run --rm -it -p 3000:3000 --name my-app-dev -v $PWD:/home/app -w /home/app -u node node:latest /bin/bash
This almost works, but the application we're developing needs access to the machine's localhost for some database applicationss (MongoDB and SQL Server) and to a RabbitMQ instance. SQL Server is on port 1433 (running in Docker), RabbitMQ is on port 5672 (also running in Docker) and MongoDB is on 27017, but not running in Docker.
When I run that Docker command and then start the application, I get an error saying that the application cannot connect to those localhost ports, which makes sense from what I've read because by default the docker container has its own localhost, which is where it would try to connect by default.
So, I added the following to the docker run command: --net=host, hoping to give the container access to my machine's localhost. This seems to get rid of the issue for RabbitMQ, but not MongoDB. There are two errors in the console for it:
2019-09-05 15:58:38.800 | error | error: Could not tear down the ORM hook. Error details: Error: Consistency violation: Attempting to tear down a datastore (`myMongoTable`) which is not currently registered with this adapter. This is usually due to a race condition in userland code (e.g. attempting to tear down the same ORM instance more than once), or it could be due to a bug in this adapter. (If you get stumped, reach out at http://sailsjs.com/support.)
at Object.teardown (/home/app/node_modules/sails-mongo/lib/index.js:390:19)
at /home/app/node_modules/waterline/lib/waterline.js:758:27
at /home/app/node_modules/waterline/node_modules/async/dist/async.js:3047:20
at eachOfArrayLike (/home/app/node_modules/waterline/node_modules/async/dist/async.js:1002:13)
at eachOf (/home/app/node_modules/waterline/node_modules/async/dist/async.js:1052:9)
at Object.eachLimit (/home/app/node_modules/waterline/node_modules/async/dist/async.js:3111:7)
at Object.teardown (/home/app/node_modules/waterline/lib/waterline.js:742:11)
at Hook.teardown (/home/app/node_modules/sails-hook-orm/index.js:246:30)
at Sails.wrapper (/home/app/node_modules/#sailshq/lodash/lib/index.js:3275:19)
at Object.onceWrapper (events.js:291:20)
at Sails.emit (events.js:203:13)
at Sails.emitter.emit (/home/app/node_modules/sails/lib/app/private/after.js:56:26)
at /home/app/node_modules/sails/lib/app/lower.js:67:11
at beforeShutdown (/home/app/node_modules/sails/lib/app/lower.js:45:12)
at Sails.lower (/home/app/node_modules/sails/lib/app/lower.js:49:3)
at Sails.wrapper [as lower] (/home/app/node_modules/#sailshq/lodash/lib/index.js:3275:19)
at whenSailsIsReady (/home/app/node_modules/sails/lib/app/lift.js:68:13)
at /home/app/node_modules/sails/node_modules/async/dist/async.js:3861:9
at /home/app/node_modules/sails/node_modules/async/dist/async.js:421:16
at iterateeCallback (/home/app/node_modules/sails/node_modules/async/dist/async.js:924:17)
at /home/app/node_modules/sails/node_modules/async/dist/async.js:906:16
at /home/app/node_modules/sails/node_modules/async/dist/async.js:3858:13
at /home/app/node_modules/sails/lib/app/load.js:261:22
at /home/app/node_modules/sails/node_modules/async/dist/async.js:421:16
at /home/app/node_modules/sails/node_modules/async/dist/async.js:1609:17
at /home/app/node_modules/sails/node_modules/async/dist/async.js:906:16
at /home/app/node_modules/sails/lib/app/load.js:186:25
at /home/app/node_modules/sails/node_modules/async/dist/async.js:3861:9
at /home/app/node_modules/sails/node_modules/async/dist/async.js:421:16
at iterateeCallback (/home/app/node_modules/sails/node_modules/async/dist/async.js:924:17)
at /home/app/node_modules/sails/node_modules/async/dist/async.js:906:16
at /home/app/node_modules/sails/node_modules/async/dist/async.js:3858:13
at afterwards (/home/app/node_modules/sails/lib/app/private/loadHooks.js:350:27)
at /home/app/node_modules/sails/node_modules/async/dist/async.js:3861:9
at /home/app/node_modules/sails/node_modules/async/dist/async.js:421:16
at iterateeCallback (/home/app/node_modules/sails/node_modules/async/dist/async.js:924:17)
at /home/app/node_modules/sails/node_modules/async/dist/async.js:906:16
at /home/app/node_modules/sails/node_modules/async/dist/async.js:3858:13
at /home/app/node_modules/sails/node_modules/async/dist/async.js:421:16
at iteratorCallback (/home/app/node_modules/sails/node_modules/async/dist/async.js:996:13)
at /home/app/node_modules/sails/node_modules/async/dist/async.js:906:16
at /home/app/node_modules/sails/lib/app/private/loadHooks.js:233:40
at processTicksAndRejections (internal/process/task_queues.js:75:11)
2019-09-05 15:58:38.802 | verbose | verbo: (The error above was logged like this because `sails.hooks.orm.teardown()` encountered an error in a code path where it was invoked without providing a callback.)
2019-09-05 15:58:38.808 | error | error: Failed to lift app: Error: Consistency violation: Unexpected error creating db connection manager:
MongoError: failed to connect to server [localhost:27017] on first connect [Error: connect ECONNREFUSED 127.0.0.1:27017
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1056:14) {
name: 'MongoError',
message: 'connect ECONNREFUSED 127.0.0.1:27017'
}]
at flaverr (/home/app/node_modules/flaverr/index.js:94:15)
at Function.module.exports.parseError (/home/app/node_modules/flaverr/index.js:371:12)
at Function.handlerCbs.error (/home/app/node_modules/machine/lib/private/help-build-machine.js:665:56)
at connectCb (/home/app/node_modules/sails-mongo/lib/private/machines/create-manager.js:130:22)
at connectCallback (/home/app/node_modules/sails-mongo/node_modules/mongodb/lib/mongo_client.js:428:5)
at /home/app/node_modules/sails-mongo/node_modules/mongodb/lib/mongo_client.js:335:11
at processTicksAndRejections (internal/process/task_queues.js:75:11)
at Object.error (/home/app/node_modules/sails-mongo/lib/index.js:268:21)
at /home/app/node_modules/machine/lib/private/help-build-machine.js:1514:39
at proceedToFinalAfterExecLC (/home/app/node_modules/parley/lib/private/Deferred.js:1153:14)
at proceedToInterceptsAndChecks (/home/app/node_modules/parley/lib/private/Deferred.js:913:12)
at proceedToAfterExecSpinlocks (/home/app/node_modules/parley/lib/private/Deferred.js:845:10)
at /home/app/node_modules/parley/lib/private/Deferred.js:303:7
at /home/app/node_modules/machine/lib/private/help-build-machine.js:952:35
at Function.handlerCbs.error (/home/app/node_modules/machine/lib/private/help-build-machine.js:742:26)
at connectCb (/home/app/node_modules/sails-mongo/lib/private/machines/create-manager.js:130:22)
at connectCallback (/home/app/node_modules/sails-mongo/node_modules/mongodb/lib/mongo_client.js:428:5)
at /home/app/node_modules/sails-mongo/node_modules/mongodb/lib/mongo_client.js:335:11
at processTicksAndRejections (internal/process/task_queues.js:75:11)
The first issue seems to be related to Sails.js and its sails-mongo ORM adapter. The second just seems to be an issue with connecting to the database. So I'm not sure if the first issue is a red herring and its underlying issue is the lack of database connection.
If anyone has any suggestions for how to run a Sails.js app inside a Docker container with access to the machine's localhost and MongoDB, I'd love some help with this!
Along with --network host in the docker run command, you need to define the host's IP in the connection properties and not localhost, since localhost in the container refers to the container itself. If you would like to make connection properties in the code consistent, you can have each developer set up a loopback alias in /etc/hosts, e.g. 127.0.0.1 my.host.com and set the connection properties to that host name ("my.host.com"), e.g. my.host.com:27017 for MongoDB.
By default, Docker creates a bridge network and assigns any container attached and the host OS an IP address. Running ifconfig and searching for docker0 interface will show the IP address range that Docker uses for the network.
This is normally quite useful because it isolates any running Docker container from the local network ensuring that only ports that are explicitly opened to the local network are exposed avoiding any potential conflicts.
Sometimes though, there are cases where a Docker container might require access to services for the host.
There are two options to achieve this:
Obtain the host IP address from within the container with the code below:
#get the IP of the host computer from within Docker container
/sbin/ip route|awk '/default/ { print $3 }'
You can Attach the Docker container to the local network by running this command:
docker run --network="host"
If you are using docker-compose you can add a network with host as driver
This will attach the container to the host network, which allows the Docker container to reach any services running on the host via localhost. It also works for any other Docker containers that are running on the local network or have their ports exposed to the localhost.
Related
I have created a PostgreSQL with a web server DOCKER CONTAINER and my app running with an API to manage all the endpoints.
The link of my git repo is: https://github.com/JulioValderrama/store-front-project.git
But I am facing problems with the connections with the Database, as depending on the Postgres HOST name value in my .env file one of the two servers (one local running in PORT 4000 and the web server running in the docker container in PORT 3000) will work or will fail.
I have tried the next:
Local server running in PORT 4000
Docker web server running in PORT 3000
HOST "127.0.0.1"
Going to¨:
http://localhost:3000/
Works fine and get response, now when trying to connect it to any of my database API:
http://localhost:3000/products
I get the error:
"Could not get PRODUCTS. Error: Error: connect ECONNREFUSED 127.0.0.1:5432"
Going to:
http://localhost:4000
Works fine and get response, now when trying to connect it to any of my database API:
http://localhost:4000/products
It works! I get the list of all my products!!
HOST "postgres"
I put "postgres" because I read online that you have to name HOST as the postgres docker image created, which is my case. And it works for the remote server.
Going to:
http://localhost:3000
Works fine and get response, then when trying to connect it to database API:
http://localhost:3000/products
It works!! It gives me the list of my products !!
Going to:
http://localhost:4000
Works fine and get response, then when trying to connect it to database API:
http://localhost:4000/products
It gives me the error:
"Could not get PRODUCTS. Error: Error: getaddrinfo ENOTFOUND postgres"
So it seems like there is an error when trying to connect to the database due to the server or the HOST name, I have no idea....
In docker-compose.yaml you have linked your machine 5432 to containers 5432 port.
If you are using docker container and want to reach postgres, use postgres as POSTGRES_HOST value.
If you are running application outside the docker environment use 0.0.0.0, 127.0.0.1.
Make sure you haven't installed postgres locally
brew uninstall postgres
Not sure why am I getting the SequelizeConnectionRefusedError. I verified that I am able to run all my docker images locally but when I try to run 'docker-compose up' command, I am running into Error: connect ECONNREFUSED 127.0.0.1:5432.
Based on my understanding of your question, here are my assumptions:
You are using MacOS
Your Postgres server is running in the host OS instead of in another docker container.
With that being said, this is a common problem with MacOS users who want to connect their docker containers to the Postgres server running in the host machine. As they are not in the same network, there is no way for your container to reach the Postgres server and hence, connecting to it via 127.0.0.1:5432 will definitely not reachable.
It will be trivial to solve in a Linux machine by adding network_mode: host so that the containers will be running in the same network as host machine hence is able to reach the Postgres server. However, due to the implementation of Docker on Mac where Docker host is actually being run in a hidden VM on top of your MacOS, this solution will not work here.
Some suggestions:
Migrate your Postgres server to run in a docker container (in the same docker-compose file if you will). You can always do a port mapping in order to access it from your Postbird.
Or if you still insist on running it locally in your MacOS, here is a workaround that involves creating another docker container in the same docker network and perform a revert SSH tunneling.
Here are the steps to migrate the Postgres server to using docker container
Update your docker-compose with a new db service:
db:
image: postgres:10.5-alpine
environment:
POSTGRES_USER: $UDAGRAM_USERNAME
POSTGRES_PASSWORD: $UDAGRAM_PASSWORD
POSTGRES_DB: $UDAGRAM_DATABASE
ports:
- 35432:5432
volumes:
- <path where you want to persist your database data>:/var/lib/postgresql/data
You can now connect to your new postgres using Postbird at localhost:35432
EDIT 1
If you run your Postgres instance in AWS RDS, you will not need to make the changes above but follow other steps:
Make sure that your network can reach the RDS endpoint at port 5432. A best practice here is to update the security group inbound rules to allow only port 5432 from only your IP address (how to do that is out of the scope of this answer but can easily be found from AWS documentation)
Update the value of UDAGRAM_HOST to be the RDS endpoint which can be found from the AWS RDS console.
I have a Node.js application where I use Redis, I am trying to connect the Docker container and the locally running Redis.
Tried solutions:
vim /usr/local/etc/redis.conf
Updated
bind 127.0.0.1
To
bind 0.0.0.0
Stopped the redis and start it again and tried running the docker
With the above thing tried running docker run -p 4000:8080 -p 6379:6379 -t node-app
Both above didn't worked getting the below error
Error: Redis connection to localhost:6379 failed - connect ECONNREFUSED 127.0.0.1:6379
Update: I am checking it on Mac.
In Dockerfile add this
Docker v19.03
ENV REDIS_HOST "redis://host.docker.internal"
when i using it on node.js
const REDIS_HOST = process.env.REDIS_HOST ? process.env.REDIS_HOST : ""
const client = redis.createClient(REDIS_HOST)
"docker.for.mac.localhost" instead of localhost or '127.0.0.1' will work :), it worked for me on mac machine.
If you use default networking (--network="bridge"), you could simply use the IP address of the gateway between the Docker host and the bridge network, i.e. 172.17.0.1. Here is the documentation. This would work on all platforms, not only on a Mac.
You just need to set the docker internal host in your node app's config.json file:
"redisHost": "host.docker.internal"
You don't need to change any Redis configuration on your local.
I am creating a PHP application(7.1) using docker container, after configuring the docker container I am unable to connect to the SSH getting the below error
SSH CONNECTION CLOSE - Error: getaddrinfo ENOTFOUND Couldnt connect to main site container Couldnt connect to main site container:2222Error: getaddrinfo ENOTFOUND Couldnt connect to main site container Couldnt connect to main site container:2222Error: connect ECONNREFUSED 172.18.0.4:2222Error: connect ECONNREFUSED 172.18.0.4:2222Error: connect ECONNREFUSED 172.18.0.4:2222 CREDENTIALS
please review the below screen cast for steps I followed
https://www.screencast.com/t/iB1lVK98n
Thanks
For a custom docker image, you need include port 2222 in the EXPOSE instruction for the Dockerfile. Although the root password is known, port 2222 cannot be accessed from the internet. It is an internal only port accessible only by containers within the bridge network of a private virtual network.
EXPOSE 2222 80
More information about this please refer to this link.
You also could check this dockerfile.
So, I'm trying to get Jenkins working inside of docker as an exercise to get experience using docker. I have a small linux server, running Ubuntu 14.04 in my house (computer I wasn't using for anything else), and have no issues getting the container to start up, and connect to Jenkins over my local network.
My issue comes in when I try to connect to it from outside of my local network. I have port 8080 forwarded to the serve with the container, and if I run a port checker it says the port is open. However, when I actually try and go to my-ip:8080, I will either get nothing if I started the container just with -p 8080:8080 or "Error: Invalid request or server failed. HTTP_Proxy" if I run it with -p 0.0.0.0:8080:8080.
I wanted to make sure it wasn't jenkins, so I tried getting just a simple hello world flask application to work, and had the exact same issue. Any recommendations? Do I need to add anything extra inside Ubuntu to get it to allow outside connections to go to my containers?
EDIT: I'm also just using the official Jenkins image from docker hub.
If you are running this:
docker run -p 8080:8080 jenkins
Then to connect to jenkins you will have to connect to (in essence you are doing port forwarding):
http://127.0.0.1:8080 or http://localhost:8080
If you are just running this:
docker run jenkins
You can connect to jenkins using the container's IP
http://<containers-ip>:8080
The Dockerfile when the Jenkins container is built already exposes port 8080
The Docker Site has a great amount of information on container networks.
https://docs.docker.com/articles/networking
"By default Docker containers can make connections to the outside world, but the outside world cannot connect to containers."
You will need to provide special options when invoking docker run in order for containers to accept incoming connections.
Use the -P or --publish-all=true|false for containers to accept incoming connections.
The below should allow you to access it from another network:
docker run -P -p 8080:8080 jenkins
if you can connect to Jenkins over local network from a machine different than the one docker is running on but not from outside your local network, then the problem is not docker. In this case the problem is what ever machine who is receiving outside connection (normally your router, modem or ...) does not know to which machine the outside request should be forwarded.
You have to make sure you are forwarding the proper port on your external IP to proper port on the machine which is running Docker. This can be normally done on your internet modem/router.