I haven't worked on my PC for few days.
Suddenly all the calls to mongo via mongoose hangs up, the callbacks are not called.
I checked that my call to .connect works, and that the connection state is 1 (connected).
I also made sure mongo service is running on localhost and the appropriate port 27017, and I can use the mongo console and query the db manually.
I also scanned the Internet for solutions but all I found was 'check that you're actually connected', and I verified that already.
Mongoose version 2.15.0, mongo version 2.4.9 and node js version is 4.4.2.
I fixed it.
Problem was duplicate references to the mongoose module.
I had a mongoose reference locally (which was connected), but my schema was present higher in the node_modules hierarchy, and it have used another mongoose instance which had no connection.
Once I removed the duplicate mongoose modules (npm uninstall mongoose one of them) it worked.
Above solutions didn't work for me so I fixed with following solution.
I had same issue where my db calls used to hang with no invocation to my callbacks or the promise resolution.
The problem was I used "createConnection()" to establish the connection with the db. But it didn't work perfectly.
Instead using "connect()" and "connection" imports worked.
Here is the sample code. Hope this helps.
import { connect, connection } from "mongoose";
const mongoUri = `mongodb://${my_mongo_host_&_port}`;
connect(mongoUri, {}); //to connect to my standalone db
//"connection" to listen to events
connection.on("connected", () => {
console.log("MongoDB connection established!", mongoUri);
});
I am working with "mongoose": "^6.6.1".
Related
Server crushed when i linked mongo atlas on the server.js file,error being generated is connection.once is not a functionserver.js file
I have tried to require once as an eventemitter but with no success
I think the way you have implemented this is wrong. When I am looking at another example i.e Mongoose.connection('once') what does it mean it has applied db.once on a DB where as you are doing it on a connection
Since today, I get the following error when I try to locally connect to a postgres database (v 12) using knex.js.
Unhandled rejection TimeoutError: Knex: Timeout acquiring a connection. The pool is probably full. Are you missing a .transacting(trx) call?
This happens on a project I've been working on for a year without any problems. Trying to isolate the issue, I created a new database with one table. When running the following lines of code, I get the same error:
const knex = require('knex');
const db = knex({
client: 'pg',
connection: 'postgresql://postgres:postgres#localhost/a_test',
pool: {
min: 0,
max: 10,
},
});
db.from('test_table')
.select(['id'])
.then(r => {
console.log(r);
});
I have no clue what might cause this. A couple of weeks ago everything worked fine and I didn't change anything in the meantime. I run postgres locally with postgresapp and when I connect to the database using psql, everything works fine. Any ideas where I could look to resolve this?
The problem
Nodejs V14 Made some breaking changes that affected the pg module! Which made it exit directly at connect() call.
One can know by downgrading to v13! (I call it the v14 HELL)! Which was a solution in the past!
A fix for pg was written in pg v8.0.3.
Fix for v14
If you are using postgres! With nodejs v14 and above ! Make sure to use the driver module pg at version >=8.0.3! And better upgrade to the latest
npm install pg#latest --save
If you are not using postgres! Try to update your DB driver! It may be the same! Also try with nodejs V13. To confirm it's the same problem! (V14 HELL)
What did happen in v14
If like me you like to know the details and what did happen !?
With node V14! Some breaking changes happened on the api! Also many things were changed! Including Openssl version!
For postgres! And pg module! The problem was as described in this comment per this thread:
The initial readyState (a private/undocumented API that
pg uses) of net.Socket seems to have changed from 'closed' to 'open'
in Node 14.
It’s hard to fix with perfect backwards compatibility, but I think I
have a patch that’s close enough.
And as per this PR!
You can see the changes in this diffing
In short as mentioned! The api for onReady changed for a net.Socket !
And the implemented solution was to not use onReady at all!
And as per this
Connection now always calls connect on its stream when connect is called on it.
In the older version the connect was called only if the socket is on closed state! readyState usage is eliminated!
Check this line
You can understand!
Depending on the implementation! Many things may or not be affected by those core changes!
Nodejs v14 relevant change
And because i wanted to see where the change happen! Here you go
https://github.com/nodejs/node/pull/32272
One can check the log of changes too:
https://github.com/nodejs/node/blob/master/doc/changelogs/CHANGELOG_V14.md
Detailed Why + exit and no logging error
Also to mention the breaking changes! Made pg make the process exit at connect() call. And that's what made it exit! And logging was to be seen!
In more detail for this! Here how it happened! Sequelize have the postgres dialect implementation! Which use pg! And pg client! create a connection! The connection have a connect event! When it connect it emit it! And because node v14 change the behavior of a stream to starting with open! The stream connection is skipped! Because of the readyState check (expected as close but it became open instead!)! And the stream is taken as connected (else block)! Where it is not! And the connect event is emitted directly! When that happen! The client either will call requestSsl() or startup() method of the connection object! And both will call this._stream.write. because the stream is not connected! An error happen! This error is not catch! Then the promise in sequelize driver! Will stay unresolved! And then the event loop get empty! Nodejs by default behavior just exit!
You can see the step through lines of code:
Sequelize pg adapter will call pg client to create a connection and the promise
pg client call connect on a connection object
pg connection connect() call and emit connect! Thinking the stream is connected because of V14 change
pg client connect event catched and callback run! requestSsl() or startup() will be run
One of the method get run and stream.write will be called (requestSsl(), startup())
Stream Error (not catched)
Promise in sequelize postgres adapter! Still unresolved!
event loop empty => Nodejs => Exit
Why nodejs exit (unresolved promises)
https://github.com/nodejs/node/issues/22088
Node exits without error and doesn't await promise (Event callback)
what happens when a Promise never resolves?
Looks like with Node 14 newer (>8.0.3) pg driver version should be used. https://github.com/knex/knex/issues/3912
It is a fact that this error can be caused by very many issues, today I found out a new one the hard way after scrolling up and down countless threads like this one to no avail.
when setting up the pool, there knex allows us to optionally register afterCreate callback, if this callback is added it is imperative that you make the call to the done callback that is passed as the last parameter to your registered callback or else no connection will be acquired leading to timeout.
.....
pool: {
afterCreate: (conn, done) => {
// .... add logic here ....
// you must call with new connection
done(null, conn);
},
}
.....
I just upgraded my psql using npm install pg#latest --save and my knex now is working.
Turns out it was the problem was node v14. When I use v13 or earlier it works.
"express": "^4.16.2",
"knex": "^0.14.2",
"objection": "^2.1.3",
"pg": "^8.0.3",
and npm install
i fixed my problem (end of the 4 day)
I am super new to node express and postgres and wondering the following:
const pg=require('pg').native
const client=new pg.Clirnt('postgres ...')
what is const?
pg is used to create a client to connect to the Postgres database-correct?
If so
var db = new Sequelize('postgres://localhost:5432/mydb')
would work too or would I just have created a database without connecting it?
Why exactly do I need to connect at all-to do what?
Thanks a lot!
const is constant in javascript which was introduced in ES6 specification.
node-postgres is a client for PostgreSQL.
Sequelize is using node-postgres for working with PostgreSQL database, so yes, in the nutshell, it will act like node-postgres.
Imagine the warehouse, that's your database, where you have different shelves, that's your tables, to take or put different items into warehouse you need workers that will do your instructions like - INSERT someitem INTO items_shelf;. So worker is the client like Sequelize or node-postgres. The important part, warehouse should be open, otherwise, workers couldn't access to it, so your database should be turned on.
Hope I'm explained understandable enough.
what is const?
TLDR; variables that can't be re-assigned. scoped the same way as var. Part of es6.
pg is used to create a client to connect to the postgres database?
yes, note you need to do npm install --save pg as well as npm install --save sequelize. the save flag adds the packages to the package.json file for your convenience.
would I just have created a database without connecting it?
That bit of code should instantiate a connector - you haven't modified the database, and you also don't really know if the connection works yet.
why exactly do I need to connect at all?
The pg library looks to use a connection pool; this means you set it up once, and then you use it repeatedly as desired and it handles the connections for you. You connect now so you can run queries against the database later.
This snippet of code connects to a postgres instance running locally on my machine, and tests that it can connect - per the docs
const Sequelize = require('sequelize');
var sequelize = new Sequelize('postgres://localhost:5432/postgres');
sequelize.authenticate().then(() => {
console.log('yay');
}).catch((e) => {
console.log('nooo', e);
});
I'm using the Mongoose next to the Node.js. The question is:
Does the following command for creation of the ObjectID value make a call to the server?
mongoose.Types.ObjectId()
I've checked my local MongoDB server log, and it doesn't show anything like a call to the single local MongoDB node/server for requesting a new ObjectID value. However, I'm not sure if the default server log is about all the operations(trivial and essential ops), or not!
NOTE: trivial ops here means the non-manipulative data operations!
A very simple script suggests that it does not make a call to the server:
Installing mongoose by running npm install mongoose
Then make a 2-liner index.js:
var mongoose = require('mongoose');
console.log(new mongoose.Types.ObjectId);
Because we haven't even connected yet.
I usually work with mongoskin because I like to be close to the database. Usually, I do a setup with a file like db.coffee, that contains just this:
mongo = require 'mongoskin'
# either local
module.exports = mongo.db 'mongodb://localhost/database'
# or remote
module.exports = mongo.db 'mongodb://<user>:<pass>#<host>:<port>/<db>?auto_reconnect=true'
Then I use it in my other sources:
db = require 'db'
users = db.collection 'users'
# Now use the collection in handlers and middleware
This seems to work perfectly fine when I am using a local mongo server, I've had an uptime for months and it never turned out to be a problem.
However, when I am using the remote second, I get problem if the server runs longer than just a few minutes - the connection to the mongodb seems lost, despite auto_reconnect. I guess this is because the localhost connection is never closed automatically.
However this led me to thinking if I am maybe using mongoskin in a wrong way, or if there's simply a bug with the auto_reconnect?
ensure mongoskin is using the 1.0.0 or higher driver