I am using latest Mongodb 3.5.7 with Node.js and Express,
What I do is I call an async function when an API is being called and the code is as following :
async function insert(data) {
try {
const connection = await client.connect();
db =client.db(dbName);
const insertion = await db.collection('inserts').insertOne(data);
//console.log(await client.close());
await console.log('finished waiting promises');
await client.close();
} catch (err) {
console.log(err.stack);
return Promise.reject('false');
}
return new Promise(resolve =>{
resolve();
})
}
`
In fact the first API Call works so well and successfully inserts the data into db , the issue starts with the second try which brings this error
I also tried promises in separated functions with every command ( connect - insert -close ) with call back promise and await for each one to be done.
As well tried calling connecting function which i created.then(setdbname).then(insert).then(closedb) ...
the options [servers] is not supported
the options [caseTranslate] is not supported
the options [dbName] is not supported
the options [credentials] is not supported
MongoError: topology was destroyed
at executeWriteOperation (C:\Users\raymo\Documents\Ticket\node_modules\mongodb\lib\core\topologies\replset.js:1188:21)
at ReplSet.insert (C:\Users\raymo\Documents\Ticket\node_modules\mongodb\lib\core\topologies\replset.js:1257:3)
at ReplSet.insert (C:\Users\raymo\Documents\Ticket\node_modules\mongodb\lib\topologies\topology_base.js:322:25)
at insertDocuments (C:\Users\raymo\Documents\Ticket\node_modules\mongodb\lib\operations\common_functions.js:259:19)
at InsertOneOperation.execute (C:\Users\raymo\Documents\Ticket\node_modules\mongodb\lib\operations\insert_one.js:26:5)
at C:\Users\raymo\Documents\Ticket\node_modules\mongodb\lib\operations\execute_operation.js:98:26
at new Promise (<anonymous>)
at executeOperation (C:\Users\raymo\Documents\Ticket\node_modules\mongodb\lib\operations\execute_operation.js:91:10)
at Collection.insertOne (C:\Users\raymo\Documents\Ticket\node_modules\mongodb\lib\collection.js:516:10)
at Object.insert (C:\Users\raymo\Documents\Ticket\routes\Handlers\DatabaseHandler.js:68:63)
as well I tried downgrading to mongo v. 3.4.0
I tried normal Callback functions and it works. but i need to upgrade and work with promises and await/async
Please suggest edits if my question is not clear.
client.close() closes the current db connection, including all the child db instances as stated in the documentation. This means that once you've called it, the current mongo client can no longer be used. You'd need to create an entirely new MongoClient and call connect() on it.
But - you should not do this, there's no need to manually create/close the connection for every database operation.
Just keep your connection instance after doing const connection = await client.connect(); and reuse/share it for your other operations. MongoClient already handles connection pooling for you.
Related
export const client = new MongoClient(
process.env.ATLAS_URI,
// TODO: Figure out what this is and why it's needed to turn off deprecation warning
{
useUnifiedTopology: true,
}
);
Following this guide and all make sense...but she is just doing one 'call' and then close().
I need to keep doing repeated calls:
export const getAllProducts = async () => {
try {
await client.connect();
const cursor = await client.db("products").collection("data").find();
return await cursor.toArray();
} catch (err) {
throw new Error(err);
} finally {
await client.close();
}
};
The first call is fine. After that: Error: MongoError: Topology is closed, please connect
I honestly don't quite understand what Topology means, but evidently it's the close() that's contributing to the issue.
It doesn't make sense that I set up new MongoClient and the ATLAS_URI does have the 'database name' in there...so why I have to connect specify that again?
Anyway, the main part of my ❓ stands: Do I just keep a separate process going and not close it? Do I start back with a whole new MongoClient each time? 😕
I'll just put a brief answer here incase anyone runs into this.
The Mongodb documentation for the Node.js driver will give you simple examples that include the client.connect()and client.close() methods just to give you a runnable example of making a simple call to the database but in a real server application you are just opening the connection to the client once during start up and typically only closing when the server application is being closed.
So in short: You don't need to open and close and connection everytime you want to perform some action on your database.
I'm trying to save some json files inside my database using a custom function I've wrote. To achieve that I must connect to the database which I'm trying to do using this piece of code at the start of the function:
let url = "mongodb://localhost:27017/database";
(async () => {
const directory = await fs.promises.readdir(__dirname + '/files')
let database = await mongoose.createConnection(url, {useNewUrlParser:true, useUnifiedTopology:true});
database.on('error', error => {
throw console.log("Couldn't Connect To The Database");
});
database.once('open', function() {
//Saving the data using Schema and save();
Weirdly enough, when executing database.once('open', function()) the callback function isn't being called at all and the program just skips the whole saving part and gets right to the end of the function.
I've searched the web for a solution, and one solution suggested to use mongoose.createConnection instant of mongoose.connect.
As you can see it didn't really fixed the issue and the callback function is still not being called.
How can I fix it, and why it happens?
Thanks!
mongoose.createConnection creates a connection instance and allows you to manage multiple db connections as the documentation states. In your case using connect() should be sufficient (connect will create one default connection which is accessible under mongoose.connection).
By awaiting the connect-promise you don't actually need to listen for the open event, you can simply do:
...
await mongoose.connect(url, {useNewUrlParser:true, useUnifiedTopology:true});
mongoose.model('YourModel', YourModelSchema);
...
The below code attempts to connect to a MongoDB instance.
When connected successfully it prints, closes the connection and thus terminates as expected.
However, when the connection is not successful (e.g. the MongoDB instance is not running) the below code fails to terminate even though the client variable is undefined.
What I can do to allow it to terminate in the case of a failed connection? Repro below.
Per this post (my nodejs script is not exiting on its own after successful execution) I tried running process._getActiveRequests() and process._getActiveHandles() to see what was active and thus preventing node.js from exiting. Those show that there are indeed active requests/handles but I am not sure how to close them.
const MongoClient = require('mongodb').MongoClient;
async function main(){
let client;
try {
client = await MongoClient.connect('mongodb://localhost:27017');
console.log('Connected successfully!');
} catch (error) {
console.log(`Failed to connect:${error}`);
} finally {
if (client)
client.close();
}
}
main();
It's a simple file from mongoose quick guide
mongoose.js
var mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/Chat');
var userSchema = mongoose.Schema({
name: String
});
var User = mongoose.model('User', userSchema);
var user = new User({name: 'Andy'});
user.save(); // if i comment it mongoose will keep one connection
User.find({}, function(err, data) { console.log(data); }); // the same if i comment it
I tried to use db.once method, but effect the same.
Why mongoose opens the second connection in this case?
Mongoose uses native mongo driver underneath, and it in turn uses connection pooling - I believe the default is 5 connections (Check here).
So your mongoose connection will use up to 5 simultaneous connections when it has simultaneous requests.
And since both user.save and User.find are asynchronous, those will be done simultaneously. So what your "program" tells node:
1. Ok, you need to shoot a `save` request for this user.
2. Also, you need to fire this `find` request.
The node runtime then reads these, runs through the whole of your function (until a return). Then it looks at it's notes:
I was supposed to call this save
I also need to call this find
Hey, mongo native driver (which is written in C++) - here are two tasks for you!
and then the mongo driver fires the first request. And it sees it is allowed to open more connections then one, so it does, and fires the second request too, without waiting for the first to finish.
If you called the find within a callback to save, it would be sequential, and the driver would probably reuse the connection it already had.
Example:
// open the first connection
user.save(function(err) {
if (err) {
console.log('I always do this super boring error check:', err);
return;
}
// Now that the first request is done, we fire the second one, and
// we probably end up reusing the connection.
User.find(/*...*/);
});
Or similar with promises:
user.save().exec().then(function(){
return User.find(query);
})
.then(function(users) {
console.log(users);
})
.catch(function(err) {
// if either fails, the error ends up here.
console.log(err);
});
By the way, you can tell mongoose to use only one connection if you need to, for some reason:
let connection = mongoose.createConnection(dbUrl, {server: {poolSize: 1}});
That would be the gist of it.
Read more on MongoLab blog and Mongoose website.
In so many introductory examples of using MongoDB, you see code like this:
var MongoClient = require('mongodb').MongoClient;
MongoClient.connect("mongodb://localhost:port/adatabase", function(err, db)
{
/* Some operation... CRUD, etc. */
db.close();
});
If MongoDB is like any other database system, open and close operations are typically expensive time-wise.
So, my question is this: Is it OK to simply do the MongoClient.connect("... once, assign the returned db value to some module global, have various functions in the module do various database-related work (insert documents into collections, update documents, etc. etc.) when they're called by other parts of the application (and thereby re-use that db value), and then, when the application is done, only then do the close.
In other words, open and close are done once - not every time you need to go and do some database-related operation. And you keep re-using that db object that was returned during the initial open\connect, only to dispose of it at the end, with the close, when you're actually done with all your database-related work.
Obviously, since all the I/O is asynch, before the close you'd make sure that the last database operation completed before issuing the close. Seems like this should be OK, but i wanted to double-check just in case I'm missing something as I'm new to MongoDB. Thanks!
Yes, that is fine and typical behavior. start your app, connect to db, do operations against the db for a long time, maybe re-connect if the connection ever dies unexpectedly, and then just never close the connection (just rely on the automatic close that happens when your process dies).
mongodb version ^3.1.8
Initialize the connection as a promise:
const MongoClient = require('mongodb').MongoClient
const uri = 'mongodb://...'
const client = new MongoClient(uri)
const connection = client.connect() // initialized connection
And then call the connection whenever you wish you perform an action on the database:
// if I want to insert into the database...
const connect = connection
connect.then(() => {
const doc = { id: 3 }
const db = client.db('database_name')
const coll = db.collection('collection_name')
coll.insertOne(doc, (err, result) => {
if(err) throw err
})
})
The current accepted answer is correct in that you may keep the same database connection open to perform operations, however, it is missing details on how you can retry to connect if it closes. Below are two ways to automatically reconnect. It's in TypeScript, but it can easily be translated into normal Node.js if you need to.
Method 1: MongoClient Options
The most simple way to allow MongoDB to reconnect is to define a reconnectTries in an options when passing it into MongoClient. Any time a CRUD operation times out, it will use the parameters passed into MongoClient to decide how to retry (reconnect). Setting the option to Number.MAX_VALUE essentially makes it so that it retries forever until it's able to complete the operation. You can check out the driver source code if you want to see what errors will be retried.
class MongoDB {
private db: Db;
constructor() {
this.connectToMongoDB();
}
async connectToMongoDB() {
const options: MongoClientOptions = {
reconnectInterval: 1000,
reconnectTries: Number.MAX_VALUE
};
try {
const client = new MongoClient('uri-goes-here', options);
await client.connect();
this.db = client.db('dbname');
} catch (err) {
console.error(err, 'MongoDB connection failed.');
}
}
async insert(doc: any) {
if (this.db) {
try {
await this.db.collection('collection').insertOne(doc);
} catch (err) {
console.error(err, 'Something went wrong.');
}
}
}
}
Method 2: Try-catch Retry
If you want more granular support on trying to reconnect, you can use a try-catch with a while loop. For example, you may want to log an error when it has to reconnect or you want to do different things based on the type of error. This will also allow you to retry depending on more conditions than just the standard ones included with the driver. The insert method can be changed to the following:
async insert(doc: any) {
if (this.db) {
let isInserted = false;
while (isInserted === false) {
try {
await this.db.collection('collection').insertOne(doc);
isInserted = true;
} catch (err) {
// Add custom error handling if desired
console.error(err, 'Attempting to retry insert.');
try {
await this.connectToMongoDB();
} catch {
// Do something if this fails as well
}
}
}
}
}