in my application I have a default database and other database I have to connect to in function of client's requests , since with mongoose in node as far as I understood: there is a pool of connections application wide, if I change database, it is changed for all the subsequent requests, I think it could cause some problems, what is the best way to switch Database with mongoose?
Mongoose 3.7.1 (unstable) supports switching databases.
Otherwise you'll need to create separate connection instances for each database.
Related
We have architecture problem on our project. This project requires sharding, as soon as we need almost unlimited scalability for the part of services.
Сurrently we use Node.js + MongoDb (Mongoose) and MySQL (TypeORM). Data is separated by databases through the simple 'DB Locator'. So node process needs connections to a lot of DBs (up to 1000).
Requests example:
HTTP request from client with Shop ID;
Get DB IP address/credentials in 'DB Locator' service by Shop ID;
Create connection to specific database with shop data;
Perform db queries.
We tried to implement it in two ways:
Create connection for each request, close it on response.
Problems:
we can't use connection after response (it's the main problem, because sometimes we need some asynchronous actions);
it works slower;
Keep all connections opened.
Problems:
reach simultaneous connections limit or some another limits;
memory leaks.
Which way is better? How to avoid described problems? Maybe there is a better solution?
Solution #1 perfectly worked for us on php as it runs single process on request and easily drops connections on process end. As we know, Express is pure JS code running in v8 and is not process based.
It would be great to close non-used connections automatically but can't find options to do that.
The short answer: stop using of MongoDB with Mongoose 😏
Longer answer:
MongoDB is document-oriented DBMS. The main usage case is when you have some not pretty structured data that you have to store, but you don't need to use too much. There is lazy indexing, dynamic typing and many more things that not allow you to use it as RDBMS, but it is great as a storage of logs or any serialized data.
The worth part here is Mongoose. This is the library that makes you feel like your trashbox is wonderful world with relations, virtual fields and many things that should not to be in DODBMS. Also, there is a lot of legacy code from previous versions that also make some troubles with connections management.
You already use TypeORM that may works instead Mongoose. With some restrictions, for sure.
It works exactly same way as MySQL connection management.
Here is some more data: https://github.com/typeorm/typeorm/blob/master/docs/mongodb.md#defining-entities-and-columns
In this case you may use you TypeORM Repository as transparent client that will init connections and close it or keep it alive on demand.
I have an app that is already working with the native Node Mongo driver (v3.0).
I'm now trying to slowly implement Mongoose in order to make the app easier to maintain. I would like to do this in a gradual way so I rewrote all the user related operations with Mongoose and the rest like it was before. I noticed that my app now creates two connections to my Mongo db. This is clearly because Mongoose knows nothing about my existing connection.
I would like to handle connecting and disconnecting to Mongo myself and give Mongoose a reference to the already existing connection but I can't find anything like this in the docs.
Is this even possible or will I need two different connections until my app is fully rewritten to use Mongoose exclusively?
EDIT: My app is being run as an AWS Lambda function which has to connect and disconnect to mongo on every request so having two concurrent connections per request is effectively halving my mongo db available connections. That’s why I’m concerned about having an extra connection.
Turns out the answer to this is to do it the other way around. Just connect to Mongoose and then grab the connection.
let mongoConnection = mongoose.connection.client
I would like sequelize to pool connections for multiple db hosts (postgres) so that if one crashes, my application stays up. In other words, I have multiple instances of the same db installed on different hosts (guaranteed to be in sync). I would like sequelize to pool connections to both dbs. Is this possible with sequelize directly? If not, what is the easiest way of doing this?
You need to set up a Postgres cluster and then connect to that from Sequelize. The configuration is done on the server side and will be transparent to the client.
https://wiki.postgresql.org/wiki/Replication,_Clustering,_and_Connection_Pooling
The connection pooling for Sequelize is defined in the connection, look for options.pool in the docs.
http://docs.sequelizejs.com/class/lib/sequelize.js~Sequelize.html
I have two Node.js applications each running Mongoose on a different machine. There is a single MongoDB database running on the first, and the second connects to it and adds documents periodically. I'm trying to add a hook to the creation of these documents so the server running the database is aware that other server has added data. I tried using the Schema.post() method, but it doesn't seem to work since there are two separate instances of Mongoose. Is this true or am I just implementing it incorrectly? I can get the hook to fire if the document is created on the same server, but not the other.
So my thought is to add the hook to MongoDB directly, instead of Mongoose, but I'm not sure how to go about doing that. Am I on the right track?
That is true, Schema.post() only work in the same process. You either need to use a library that tails MongoDB's oplog (like mongo-oplog), implement it yourself using a message queue (or pub/sub) that all instances are connected to (like Redis, RabbitMQ, etc) or use a database that supports this natively. PostgreSQL supports this with its NOTIFY feature for example.
I'm doing an API with Nodejs using Restify.
For the DB I'm using Mongodb (with mongoose).
I was wondering, what the best solution between sharing a db connection to my entire app or connecting to the db on each request ?
For now, I'm using the second option of this answer : sharing db connection
But I've seen a different pattern here : Node.js Web Application with Storage on MongoDB
I can't figure out, what is the best architecture ?
A list of pros and cons could be a great help.
Of course keeping one connection ( or pool of connections if mongoose supports it ) and reusing it is better, simply because creating connection on each request eats resources.