Unable to use MongoDB on GCP cloud run and connect it to a NodeJS - node.js

I am testing a nodejs app on GCP functions and a MongoDB instance in GCP cloud run(I know this is not a good idea) but again this is a test. I am able to get the mongo image running on cloud run and I am given a service endpoint as https://mongodb.foo.bar.run.app and I have the container port in this cloud run service as 27017.
When I try and edit the mongoose.connect() method to include this service uri I am not able to connect with the app. I have the mongodb cloud run instance open to receive traffic from all traffic(Again not a good idea), but I still can't get the node app to connect.
I have setup the connection in node as mongoose.connect('mongodb://mongodb.foo.bar.run.app:27017/test');
but I get this error
Error: connect ETIMEDOUT <IP ADDRESS GOES HERE>:27017
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1146:16)
at TCPConnectWrap.callbackTrampoline (internal/async_hooks.js:129:14)
Emitted 'error' event on NativeConnection instance at:
at NativeConnection.Connection.error (/Users/foo/nodejs/bar/node_modules/mongoose/lib/connection.js:443:8)
at /Users/foo/nodejs/bar/node_modules/mongoose/lib/connection.js:472:15
at /Users/foo/nodejs/bar/node_modules/mongoose/lib/drivers/node-mongodb-native/connection.js:59:21
at /Users/foo/nodejs/bar/node_modules/mongodb/lib/db.js:232:14
at Server.<anonymous> (/Users/foo/nodejs/bar/node_modules/mongodb/lib/server.js:240:9)
at Object.onceWrapper (events.js:421:26)
at Server.emit (events.js:314:20)
at Pool.<anonymous> (/Users/foo/nodejs/bar/node_modules/mongodb-core/lib/topologies/server.js:308:68)
at Pool.emit (events.js:314:20)
at Connection.<anonymous> (/Users/foo/nodejs/bar/node_modules/mongodb-core/lib/connection/pool.js:115:12)
at Object.onceWrapper (events.js:421:26)
at Connection.emit (events.js:314:20)
at Socket.<anonymous> (/Users/foo/nodejs/bar/node_modules/mongodb-core/lib/connection/connection.js:144:49)
at Object.onceWrapper (events.js:421:26)
at Socket.emit (events.js:314:20)
at emitErrorNT (internal/streams/destroy.js:100:8) {
name: 'MongoError'
}
I tried to curl the service uri as
curl https://mongodb.foo.bar.run.app
and I get a 200 response in the mongo logs GET 200 466B but if I
curl mongodb.foo.bar.run.app
I get a 302 response code GET 302 0B.
I looked at the mongoose documentation but I couldn't figure what I was doing wrong. I know the documentation states user and password on the uri string, but when I run it locally on my docker engine I did not use a user and password and it worked.

As mentioned in the comments, it's not possible to run MongoDB on Cloud Run since as mentioned here:
Cloud Run can only receive HTTP requests or Pub/Sub push events.
Connecting to MongoDB requires a TCP protocol rather than HTTP, that's why every call to it would fail even if the port is reachable.
Cloud Run is in this sense more similar to AWS Lambda than it's to AWS Fargate as it's even billed on a per request basis rather than on resources consumed.

Related

Node.js Error: Unexpected server response: 301 - Running a Node.js Websocket server on Namecheap Webhost

I'm trying to setup a Node.js Websocket server over at my webhost (Namecheap). However, I am getting this error:
events.js:292
throw er; // Unhandled 'error' event
^
Error: Unexpected server response: 301
at ClientRequest.<anonymous> (D:\****\node_modules\ws\lib\websocket.js:576:7)
at ClientRequest.emit (events.js:315:20)
at HTTPParser.parserOnIncomingClient [as onIncoming] (_http_client.js:596:27)
at HTTPParser.parserOnHeadersComplete (_http_common.js:119:17)
at TLSSocket.socketOnData (_http_client.js:469:22)
at TLSSocket.emit (events.js:315:20)
at addChunk (_stream_readable.js:295:12)
at readableAddChunk (_stream_readable.js:271:9)
at TLSSocket.Readable.push (_stream_readable.js:212:10)
at TLSWrap.onStreamRead (internal/stream_base_commons.js:186:23)
Emitted 'error' event on WebSocket instance at:
at abortHandshake (D:\****\node_modules\ws\lib\websocket.js:694:15)
at ClientRequest.<anonymous> (D:\****\node_modules\ws\lib\websocket.js:576:7)
[... lines matching original stack trace ...]
at TLSSocket.Readable.push (_stream_readable.js:212:10)
The code for the client and server came from here:
https://github.com/websockets/ws#Simple server
https://github.com/websockets/ws#sending-and-receiving-text-data
with ws://www.host.com/path changed to ws://<mydomainname>/<dir1>/<dir2>
Both client and server code runs properly when run locally. Http related code runs properly when used on my webhost. I am trying to get the client code to run locally and connect via websockets to my server hosted at Namecheap.
I suspect something related to the server is preventing connection. I'd ask support but I doubt if they can resolve this. Maybe there's a locked feature or something? Is there something else I could have missed?
Websockets cannot work on Namecheap's hosted servers. Their incoming ports are blocked and cannot be opened for security reasons per their customer's support.
If anyone is looking for a solution just in case, try out a Heroku free account.

MongoNetworkError on first connect using Docker and NodeJS

I'm still incredibly new to coding. I'm trying to launch an app using docker, everything goes well until I get this error:
{ MongoNetworkError: failed to connect to server [localhost:27017] on first connect [MongoNetworkError: connect ECONNREFUSED 127.0.0.1:27017]
at Pool.<anonymous> (/app/node_modules/mongodb/lib/core/topologies/server.js:433:11)
at Pool.emit (events.js:198:13)
at createConnection (/app/node_modules/mongodb/lib/core/connection/pool.js:577:14)
at connect (/app/node_modules/mongodb/lib/core/connection/pool.js:1007:11)
at makeConnection (/app/node_modules/mongodb/lib/core/connection/connect.js:31:7)
at callback (/app/node_modules/mongodb/lib/core/connection/connect.js:247:5)
at Socket.err (/app/node_modules/mongodb/lib/core/connection/connect.js:276:7)
at Object.onceWrapper (events.js:286:20)
at Socket.emit (events.js:198:13)
at emitErrorNT (internal/streams/destroy.js:91:8)
at emitErrorAndCloseNT (internal/streams/destroy.js:59:3)
at process._tickCallback (internal/process/next_tick.js:63:19)
name: 'MongoNetworkError',
[Symbol(mongoErrorContextSymbol)]: {} }
[nodemon] clean exit - waiting for changes before restart
I have mongodb running as a server on Windows 10, and I also have it running in another window, so I'm not sure what could be causing it. I've tried all solutions on this page, but to no avail. Thanks in advance!
Localhost is the container, not your machine. When working with containers, you need to think of container as a separate machine and use a full network adressess to communicate between your machine and container.
I was using a similar concept like yours. Try to run both node-app and Mongo service under the same network.
In Mongo connection string, the Hostname should be the same as your service name mentioned in docker-compose.yml file.

How to connect to a MongoDB running on an aws ec2 host (linux) out of docker

Situation:
I prototyped a small web (node.js) app and dockerized it for deployment and replicability purposes.
The app speaks with a MongoDB running directly on the host.
Problem:
On the server (AWS EC2 instance, only Port 80 and 443 open), I am not able to interact with MongoDB and I am wondering why.
docker run --net="host" -e 'NODE_ENV=production' -e 'MONGO_URI=mongodb://USER:PASSWORD!#172.31.32.1:27017/test_db' DOCKER_IMAGE
MongoDB connected ...
Warning: connect.session() MemoryStore is not
designed for a production environment, as it will leak
memory, and will not scale past a single process.
HTTP Server started on port 80
(node:1) DeprecationWarning: current Server Discovery and Monitoring engine is deprecated, and will be removed in a future version. To use the new Server Discover and Monitoring engine, pass option { useUnifiedTopology: true } to the MongoClient constructor.
MongoNetworkError: failed to connect to server [172.31.32.1:27017] on first connect [MongoNetworkError: connection timed out
at connectionFailureError (/app/server/node_modules/mongodb/lib/core/connection/connect.js:377:14)
at Socket.<anonymous> (/app/server/node_modules/mongodb/lib/core/connection/connect.js:287:16)
at Object.onceWrapper (events.js:284:20)
at Socket.emit (events.js:196:13)
at Socket._onTimeout (net.js:432:8)
at listOnTimeout (internal/timers.js:531:17)
at processTimers (internal/timers.js:475:7) {
name: 'MongoNetworkError',
[Symbol(mongoErrorContextSymbol)]: {}
}]
at Pool.<anonymous> (/app/server/node_modules/mongodb/lib/core/topologies/server.js:433:11)
at Pool.emit (events.js:196:13)
at /app/server/node_modules/mongodb/lib/core/connection/pool.js:571:14
at /app/server/node_modules/mongodb/lib/core/connection/pool.js:994:11
at /app/server/node_modules/mongodb/lib/core/connection/connect.js:40:11
at callback (/app/server/node_modules/mongodb/lib/core/connection/connect.js:262:5)
at Socket.<anonymous> (/app/server/node_modules/mongodb/lib/core/connection/connect.js:287:7)
at Object.onceWrapper (events.js:284:20)
at Socket.emit (events.js:196:13)
at Socket._onTimeout (net.js:432:8)
at listOnTimeout (internal/timers.js:531:17)
at processTimers (internal/timers.js:475:7) {
name: 'MongoNetworkError',
[Symbol(mongoErrorContextSymbol)]: {}
}
I initially tried localhost instead of the IP address, but that does not work. It throws an authentication error (which is somewhat strange). Since there is no host.docker.internal for Linux I had to (temporarily) resolve to the explicit IP address. The IP address that I am using I got via:
netstat -nr | grep '^0\.0\.0\.0' | awk '{print $2}'
What I find very strange is that I am not getting an Authentication error but a Timeout error, so to me it seems like the app is able to connect to Mongo. Also, the "MonogDB connected ..." would indicate that, as it is produced by the following line in my server script.
mongoose
.connect(DB, { useNewUrlParser: true })
.then(console.log("MongoDB connected ..."))
.catch(err => console.log(err));
For completeness sake, the same setup (i.e. dockerized app and running MongoDB directly on the host) worked without a problem locally.
Also, I am able to enter the Mongo Shell on the server via mongo.
Any explanation or tip is appreciated!
If mongo DB is running outside of the Docker, you can use the private IP of the EC2 instance to connect with Mongo DB (running on Host) from inside Docker.
EC2_PRIVATE_IP=$(curl http://169.254.169.254/latest/meta-data/local-ipv4)
docker run --net="host" -e 'NODE_ENV=production' -e 'MONGO_URI=mongodb://USER:PASSWORD!#$EC_PRIVATE_IP:27017/test_db' DOCKER_IMAGE

"Request error: Socket hang up" with nodeJS on Amazon EC2

I'm attempting to use the packages 'request promise' and 'cheerio' on NodeJS to scrape links from a website, via an Amazon EC2 instance.
I'm able to run my script on my own desktop (outside the Amazon EC2 instance) and scrape the links without any problems.
However, when I run the same script in my Amazon EC2 instance, I get the following error:
Unhandled rejection RequestError: Error: socket hang up
at new RequestError (/home/ec2-user/node_modules/request-promise-core/lib/errors.js:14:15)
at Request.plumbing.callback (/home/ec2-user/node_modules/request-promise-core/lib/plumbing.js:87:29)
at Request.RP$callback [as _callback] (/home/ec2-user/node_modules/request-promise-core/lib/plumbing.js:46:31)
at self.callback (/home/ec2-user/node_modules/request/request.js:186:22)
at emitOne (events.js:96:13)
at Request.emit (events.js:188:7)
at Request.onRequestError (/home/ec2-user/node_modules/request/request.js:845:8)
at emitOne (events.js:96:13)
at ClientRequest.emit (events.js:188:7)
at TLSSocket.socketOnEnd (_http_client.js:346:9)
Does this have something to do with my AWS security group settings? Currently, outbound traffic is configured to 'All Traffic' with destination 0.0.0.0/0, while inbound traffic is only set to the following configuration: SSH, TCP, Port 20, my own IP address. Should I be adding another rule for inbound traffic?
Alternatively, could this have something to do with my instance size? I'm currently using a micro instance that is free under AWS.
Or finally, could it be the case that the website has blocked access from AWS instances?

No living connections Error while Elasticsearch connections in nodejs

I am having this problem while connecting the elasticsearch connections.
var elasticsearch = require('elasticsearch');
var client = new elasticsearch.Client({ host: 'localhost:9200',
log: 'trace'});
Elasticsearch ERROR: 2016-07-19T19:09:26Z
Error: Request error, retrying -- connect ECONNREFUSED 127.0.0.1:9200
at Log.error (/root/git_build/FirstMoveChess/node_modules/elasticsearch/src/lib/log.js:225:56)
at checkRespForFailure (/root/git_build/FirstMoveChess/node_modules/elasticsearch/src/lib/transport.js:195:18)
at HttpConnector. (/root/git_build/FirstMoveChess/node_modules/elasticsearch/src/lib/connectors/http.js:154:7)
at ClientRequest.bound (/root/git_build/FirstMoveChess/node_modules/lodash-node/modern/internals/baseBind.js:56:17)
at emitOne (events.js:96:13)
at ClientRequest.emit (events.js:188:7)
at Socket.socketErrorListener (_http_client.js:308:9)
at emitOne (events.js:96:13)
at Socket.emit (events.js:188:7)
at emitErrorNT (net.js:1272:8)
at _combinedTickCallback (internal/process/next_tick.js:74:11)
at process._tickCallback (internal/process/next_tick.js:98:9)
Elasticsearch TRACE: 2016-07-19T19:09:27Z
-> HEAD http://localhost:9200/
I understand this question is quite old but I wanted to share how you can solve this problem.
If you are locally using elasticsearch
First thing you must do is running elasticsearch on your machine.
Error: Request error, retrying -- connect ECONNREFUSED 127.0.0.1:9200
Because above message indicates that you're not running elasticsearch locally.
So, visit the link and follow the insturction.
Docker environment
It gets much trickier here.
First, follow the instruction here.
And in case you're using node.js elasticsearch client, you have to specify elasticsearch host as 172.24.0.1.
If you use container_name or private IP of container in docker-compose.yml, it won't work.
In case of Docker Container Environment after changing from http://localhost:9200 to http://ipaddress:9200 in docker-compose.yml
please change the following live in docker-compose.yml that is related to CORS
Change this
** http.cors.allow-origin=/https?://localhost(:[0-9]+)?/ **
into this
*- http.cors.allow-origin= **

Resources