How do I execute MongoDB shell query using NodeJS - node.js

I know we have various packages avalable on GitHub to query MongoDB using NodeJS. They don't solve my use case. I am looking to execute MongoDB Shell commands from NodeJS which are different from MongoDB NodeJS commands
Example: NodeJS
db.collection('user').find({})
Example: MongoDB Shell
db.getCollection('user').find({})
Notice 'collection' and 'getCollection' are different.
I want to save MongoDB Shell queries as text. How do I execute those queries once I have read them from the database?

Try this adapted from the MongoDB documentation:
const { MongoClient } = require("mongodb");
// Replace the uri string with your MongoDB deployment's connection string.
const uri =
"mongodb+srv://<user>:<password>#<cluster-url>?w=majority";
const client = new MongoClient(uri);
async function run() {
try {
await client.connect();
const db = client.db("myDatabase");
const result = await db.getCollection('user').find({});
console.log(result);
} finally {
await client.close();
}
}
run().catch(console.dir);

You can use child_process library of NodeJs but it is litte bit tricky. You must wait for mongo shell to finish its words.
const { spawn } = require("child_process");
const child = spawn("/usr/bin/mongo");
setTimeout(() => {
child.stdin.write("show dbs\n");
}, 5000);
child.stdout.on("data", (data) => {
console.log(`command line says:\n${data}`);
});

Lets Try this code -
let { MongoClient } = require('mongodb');
const url =
`mongodb+srv://${process.env.DB_USER}:${process.env.DB_USER_PASSWORD}#
${process.env.DB_CLUSTER}.mongodb.net`;
async function connectDatabase() {
const client = await MongoClient.connect(url, {
useNewUrlParser: true,
useUnifiedTopology: true
});
const db = client.db("main");
return {
listings: db.collection("test_listings")
};
};
module.exports = { connectDatabase };

Related

How to create a database object using MongoDB with Next.js?

I followed this tutorial to set up MongoDB in my Next.js application: https://www.mongodb.com/developer/languages/javascript/nextjs-with-mongodb.
In a file called mongodb-config.js, I have
import { MongoClient } from 'mongodb'
const uri = process.env.MONGODB_URI
const options = {
useUnifiedTopology: true,
useNewUrlParser: true,
}
let client;
let dbPromise;
if (!process.env.MONGODB_URI) {
throw new Error('Please add your Mongo URI to .env.local')
}
// In production mode, it's best to not use a global variable.
client = new MongoClient(uri, options)
dbPromise = client.connect()
// Export a module-scoped MongoClient promise. By doing this in a
// separate module, the client can be shared across functions.
export default dbPromise
The above is how I configure my database.
When I need to use the database for my API, I do:
import dbPromise from "database/mongodb-config";
let db;
dbPromise.then((value) => {
const client = value;
db = client.db("database_name");
})
.catch((error)=>{
console.error(error);
});
db will be the variable that links to my database.
Now, I want to simplify this process and put everything in one file.
In the portion where I set up dbPromise in mongodb-config.js, I initialize the DB variable and export it:
client = new MongoClient(uri, options)
dbPromise = client.connect()
let db;
dbPromise.then((value)=>{
const client = value;
db = client.db("datatbase_name");
})
.catch((error)=>{
console.error(error);
});
export default db
I totally expect this to work. However, when I import db to another file and use it, the db is null. However, I waited until the dbPromise is resolved to pass db a value.
Use await like this.
dbPromise = client.connect()
let db;
await dbPromise.then((value)=>{
const client = value;
db = client.db("datatbase_name");
})
.catch(error){
console.error(error);
}
export default db
because mongoose is asynchronous so we have to use await before dbPromise otherwise export will be executed first before dbPromise.
Try this, instead of returning db, return an async function, which you can all with await, in other parts to obtain, the connection to the database, the function will not try to make repeated connections, if they are already present:
let db, connection;
async function dbSetup() {
try {
if(!connection) {
connection = await client.connect();
}
if(!db) {
db = await connection.db("datatbase_name");
}
return db;
} catch(error){
console.error(error);
}
}
export default dbSetup;
Another way is if you have node version 14.8 or greater, you can use top-level await functionality, which is basically using await, not inside an async function, like this:
client = new MongoClient(uri, options);
dbPromise = await client.connect();
let db = await dbPromise.db("datatbase_name");
export default db;

Using async/await with Mongoose recent version(6.2.7)

I have the following code to connect to Mongoose DB using Node.js. But I get some kind of warning or refactor notification from VS code to remove await from the the part when I try to connect.
Its say that: await has no effect in this kind of expression
From the documentation of Mongoose inside index.d.ts file;
export function connect(uri: string, options?: ConnectOptions): Promise<Mongoose>; returns promise
So I try to do like this:
// Provide connection to a new in-memory database server.
const connect = async () => {
// NOTE: before establishing a new connection close previous
await mongoose.disconnect()
mongoServer = await MongoMemoryServer.create()
try {
const mongoUri = await mongoServer.getUri()
await mongoose.connect(mongoUri, opts) // No need to use await??
} catch (error) {
console.log(error)
}

Not sure why node is continuously running

I can't figure out why this app keeps running. I've tried using the why-is-node-running package but I'm not perfectly sure how to read the output properly. Here's the first output of it:
There are 30 handle(s) keeping the process running
# TCPWRAP
/node_modules/mongodb/lib/core/connection/connect.js:269 - socket = tls.connect(parseSslOptions(family, options));
/node_modules/mongodb/lib/core/connection/connect.js:29 - makeConnection(family, options, cancellationToken, (err, socket) => {
/node_modules/mongodb/lib/core/sdam/monitor.js:182 - connect(monitor.connectOptions, monitor[kCancellationToken], (err, conn) => {
/node_modules/mongodb/lib/core/sdam/monitor.js:206 - checkServer(monitor, e0 => {
/node_modules/mongodb/lib/core/sdam/monitor.js:92 - monitorServer(this);
My guess is it has something to do with MongoDB not closing properly. Although, when I removed all of the other functions between opening the client and closing it, it opened and closed perfectly.
Adding process.exit() at the end closes program properly, but I'd like to figure out why it isn't closing.
A summary of the app is that it is getting data from MongoDB, cleaning it, and then writing it into Firestore - so a lot of async actions going on, but I didn't see Firestore-related stuff pop up in the why-is-node-running logs.
const GrabStuffFromDBToCalculate = require("./helpers/GrabStuffFromDBToCalculate");
const SendToFirestore = require("./helpers/SendToFirestore");
const log = require("why-is-node-running");
const { MongoClient } = require("mongodb");
require("dotenv").config();
const main = async () => {
try {
const client = await MongoClient.connect(process.env.MONGODB_URI, {
useNewUrlParser: true,
useUnifiedTopology: true
});
const collection = await client.db("test").collection("testcollection");
const trip_object = await GrabStuffFromDBToCalculate(collection);
SendToFirestore(trip_object);
client.close();
log(); // "There are 30 handle(s) keeping the process running including node_modules/mongodb/lib/core/connection/connect.js:269 - socket = tls.connect(parseSslOptions(family, options));"
// process.exit() // this closes everything but I'd rather not have to use this
} catch (err) {
console.log(err);
client.close();
}
};
const runAsync = async () => {
await main(); // this exists because I'm usually running multiple main() functions
};
runAsync();
SendToFirestore code:
const firebase = require("firebase");
const firebaseConfig = require("../config");
module.exports = SendToFirestore = trip_object => {
if (!firebase.apps.length) {
firebase.initializeApp(firebaseConfig);
}
const db = firebase.firestore();
db.doc(`hello/${object._id}`).set({
objectid:object._id
});
};
GrabStuffFromDBToCalculate code (way simplified):
module.exports = GrabStuffFromDBToCalculate = async collection => {
const cursor = await collection
.aggregate([
// does a bunch of stuff here
])
.toArray();
const newObj = cursor[0];
return newObj;
};
Making my comment into an answer since it led to the missing piece.
Node does not shut down because you have an open Firestore connection. You will have to call terminate to allow the SDK to shut down and release resources:
db.terminate();
Which is relevant for allowing nodejs to shut itself down automatically.
Also, I'm not sure you understood that I was suggesting that you use await as in
await client.close()
before calling log() so you are sure that the client connection has been closed before you do the logging. client.close() is an asynchronous method so your original code would log() before that close was complete.

MongoClient throw MongoError: server instance pool was destroyed

I saw these posts on SO describing this error. Most of them was by the reason that JavaScript is async and mongoClient.close() called outside of callback. That's not my case, but I don't know what else can be the reason.
const MongoClient = require('mongodb').MongoClient;
const url = "mongodb://localhost:27017/";
const mongoClient = new MongoClient(url, {
useNewUrlParser: true
});
module.exports = class Mongo {
insertOne(article) {
mongoClient.connect((err, client) => {
const db = client.db('grabber');
db.collection("zr").insertOne(article, (err, res) => {
if (err) throw err;
mongoClient.close();
});
});
};
}
I observed that you open mongoClient.connect() in the insertOne() method, and also call mongoClient.close() within that method, with mongoClient as a global variable.
My hunch is that either:
There's another method that calls mongoClient that was closed by this method, or
You called the insertOne(article) twice
I can confirm that the second reason is the most likely one. Here's the code I tried:
const MongoClient = require('mongodb').MongoClient;
const url = "mongodb://localhost:27017/";
const mongoClient = new MongoClient(url, {
useNewUrlParser: true
});
class Mongo {
insertOne(article) {
mongoClient.connect((err, client) => {
const db = client.db('grabber');
db.collection("zr").insertOne(article, (err, res) => {
if (err) throw err;
mongoClient.close();
});
});
};
};
x = new Mongo()
setTimeout(function() { x.insertOne({'a': 1}); }, 1000);
setTimeout(function() { x.insertOne({'a': 2}); }, 2000);
The two setTimeout was there to ensure that the two insertOne() are called one after another. Result:
MongoError: server instance pool was destroyed
The way your code is currently structured, the node driver creates a new connection pool every time insertOne() is called. This is not optimal, and prevents the node driver to utilize connection pooling.
Instead of calling mongoClient.connect() inside insertOne(), call it globally outside of class Mongo. Pass the global connection object (the returned object from mongoClient.connect()) instead of the mongoClient object itself to your insertOne() method.

Sequelize won't close after bulkcreate

I'm just getting started on Sequelize to write to a MSSQL db - all working well, but my node program never finishes and returns to the terminal. Code looks like this:
const Sequelize = require('sequelize');
var sequelize = new Sequelize(...);
const JournalLine = sequelize.define('journalLine', {...});
JournalLine.sync().then(() => {
JournalLine.bulkCreate([...])
});
Is there something I need to close off in order to end the program?
You did not provided callback function or utilise Promise, following will work
const Sequelize = require('sequelize');
var sequelize = new Sequelize(...);
const JournalLine = sequelize.define('journalLine', {...});
JournalLine.sync().then(() => {
return JournalLine.bulkCreate([...]).then( result => {
console.log('done');
return result;
})
});

Resources