My question is strictly related to MEAN stack. I want to fetch the total count of documents stored in my MongoDB databse. My express server is working fine but on postman I can see still it is still making request even though i got correct count on console:
Here is my code:
api.js
const uri = 'mongodb://tanzee.......ue&w=majority'
const { MongoClient } = require("mongodb");
const client = new MongoClient(uri);
router.get('/totalmovies', function(req, res) {
run().catch(console.dir);
});
async function run() {
try {
await client.connect();
const database = client.db("mycollection");
const movies = database.collection("movies");
const estimate = await movies.estimatedDocumentCount();
console.log(`Estimated number of documents in the movies collection: ${estimate}`);
} finally {
await client.close();
}
}
I took help from the official doc of mongodb: https://docs.mongodb.com/drivers/node/usage-examples/count/
Output:
(node:15511) DeprecationWarning: current URL string parser is deprecated, and will be removed in a future version. To use the new parser, pass option { useNewUrlParser: true } to MongoClient.connect.
Estimated number of documents in the movies collection: 80
I'm new to this stack. Once the output is fetched, my code should stop the GET call immediately. Please point out my mistakes.
Your code is correct, the request is pending as you are not sending any response thats why the requester is still waiting to receive one. A simple fix is to send data back to the requester.
router.get('/totalmovies', async function(req, res) {
const result = await run();
// check how are you getting result when it is a success/failure
// apply condition same in the below if/else
// it just for reference, it might be correct though
if (result > 0) {
res.status(200).send(result);
} else {
res.status(404).send({'No Data in DB'});
}
});
(node:15511) DeprecationWarning: current URL string parser is
deprecated, and will be removed in a future version. To use the new
parser, pass option { useNewUrlParser: true } to MongoClient.connect.
for this to work:
const client = new MongoClient(uri, { useNewUrlParser: true })
Don't close and rebuild the connection for every request:
async function run() {
try {
await client.connect();
const database = client.db("mycollection");
const movies = database.collection("movies");
const estimate = await movies.estimatedDocumentCount();
console.log(`Estimated number of documents in the movies collection: ${estimate}`);
return estimate;
} catch (e){
console.error('Error in fetching the data from DB')
}
}
Related
I am Trying to implement transaction in my project, since I would like to update more than one collection on the same api
I am sharing the sample code, I tried to understand the concept
sample code
const mongoose = require("mongoose");
const session = await mongoose.startSession();
session.startTransaction();
const opts = { session };
try {
const sample = await sample_table.findOneAndUpdate({
user_name : "nishanth"
},{status: "Inactive"},
{opts});
const transaction = await transaction_table.findOneAndUpdate({
bank_name : SBI
},{status: "Inactive"}, { opts });
await session.commitTransaction();
session.endSession();
return res.send({message: "success", sample_data: sample, transaction_data:
transaction});
}catch(err)
{
await session.abortTransaction();
session.endSession();
return res.send({err: err.message});
}
In the above code, I am trying to update two collections status to Inactive with a filter.
I had give an undefined variable for the transaction_table so the second update query will fail before commit and go to catch block and perform abortTransaction
When I use the above code, It updates the sample_table collection in db. could someone explain me where I am going wrong and share me a sample project for the above example
I am using "mongoose": "^5.13.14", version
We processing the order requests, on the time of multiple HTTP requests hitting the same route function means the function not working properly
Actually, we write the async await operation for all DB related queries
our coding example is
buyCtrl.orderPlace = async function (req, res, next) {
var response = await table1.find({exchange:ref_m}).sort({_id:-1}).limit(1);
if(response.length > 0)
{
var incRes = await table2.findOneAndUpdate({_id:1},{a:b}, {new: true});
var OrdersIns = new Orders();
OrdersIns.userid = userid;
OrdersIns.from = 'a';
OrdersIns.to = 'b';
await OrdersIns.save();
await compare(positiondetails[0]);
res.status(200).json({status:true,message:'Order placed successfully'});
}
}
const compare = async (curOrder) => {
var result = await Orders.find({userid:mongoose.Types.ObjectId(userid),type:opp}).sort({posTimestamp:1});
if(result.length>0)
{
for(var pi=0;pi<result.length;pi++)
{
if(parseFloat(curOrder.qty)==parseFloat(result[i].qty))
{
var updatedata = {a:b}
await Orders.findOneAndUpdate({_id:mongoose.Types.ObjectId(curOrder._id)}, updatedata, {new: true});
await Orders.remove({_id:mongoose.Types.ObjectId(result[i]._id)});
}
else
{
var updatedata = {a:b}
await Orders.findOneAndUpdate({_id:mongoose.Types.ObjectId(result[i]._id))}, updatedata, {new: true});
await Orders.remove({_id:mongoose.Types.ObjectId(curOrder._id)});
}
}
}
}
This is the flow we have I cannot paste my full code so make some example code
Please check and share your ideas
when we make an automation test with more concurrent requests the compare and update is not working properly
Thanks in advance
For atomic operations such as increment and decrement, this can be done using MongoDB's provided API. Another solution is to work with versioning, which MongoDB provides out of the box. The concurrent connection might update the object which changed before the update could take place.
I'm building a Slackbot that makes a call to an Express app, which then needs to 1) fetch some other data from the Slack API, and 2) insert resulting data in my database. I think I have the flow right finally using async await, but the operation is timing out because the original call from the Slackbot needs to receive a response within some fixed time I can't control. It would be fine for my purposes to ping the bot with a response immediately, and then execute the rest of the logic asynchronously. But I'm wondering the best way to set this up.
My Express route looks like:
const express = require('express');
const router = express.Router();
const knex = require('../../db/knex.js');
const slack = require('../../services/slack_helpers');
// POST api/slack/foo
router.post('/foo', async (req, res) => {
let [body, images] = await slack.grab_context(req);
knex('texts')
.insert({ body: body,
image_ids: images })
.then(text => { res.send('worked!'); }) // This sends a response back to the original Slackbot call
.catch(err => { res.send(err); })
});
module.exports = router;
And then the slack_helpers module looks like:
const { WebClient } = require('#slack/web-api');
const Slack = new WebClient(process.env.SLACKBOT_TOKEN);
async function grab_context(req) {
try {
const context = await Slack.conversations.history({ // This is the part that takes too long
channel: req.body.channel_id,
latest: req.headers['X-Slack-Request-Timestamp'],
inclusive: true,
limit: 5
});
} catch (error) {
return [error.toString(), 'error'];
}
return await parse_context(context);
};
function parse_context(context) {
var body = [];
context.messages.forEach(message => {
body.push(message.text);
});
body = body.join(' \n');
return [body, ''];
}
module.exports = {
grab_context
};
I'm still getting my head around asynchronous programming, so I may be missing something obvious. I think basically something like res.send perhaps needs to come before the grab_context call? But again, not sure the best flow here.
Update
I've also tried this pattern in the API route, but still getting a timeout:
slack.grab_context(req).then((body, images) => {
knex ...
})
Your timeout may not be coming from where you think. From what I see, it is coming from grab_context. Consider the following simplified version of grab_context
async function grab_context_simple() {
try {
const context = { hello: 'world' }
} catch (error) {
return [error.toString(), 'error']
}
return context
}
grab_context_simple() /* => Promise {
<rejected> ReferenceError: context is not defined
...
} */
You are trying to return context outside of the try block where it was defined, so grab_context will reject with a ReferenceError. It's very likely that this error is being swallowed at the moment, so it would seem like it is timing out.
The fix is to move a single line in grab_context
async function grab_context(req) {
try {
const context = await Slack.conversations.history({
channel: req.body.channel_id,
latest: req.headers['X-Slack-Request-Timestamp'],
inclusive: true,
limit: 5
});
return await parse_context(context); // <- moved this
} catch (error) {
return [error.toString(), 'error'];
}
};
I'm wondering the best way to set this up.
You could add a higher level try/catch block to handle errors that arise from the /foo route. You could also improve readability by staying consistent between async/await and promise chains. Below is how you could use async/await with knex, as well as the aforementioned try/catch block
const express = require('express');
const router = express.Router();
const knex = require('../../db/knex.js');
const slack = require('../../services/slack_helpers');
const insertInto = table => payload => knex(table).insert(payload)
const onFooRequest = async (req, res) => {
try {
let [body, images] = await slack.grab_context(req);
const text = await insertInto('texts')({
body: body,
image_ids: images,
});
res.send('worked!');
} catch (err) {
res.send(err);
}
}
router.post('/foo', onFooRequest);
module.exports = router;
I can't figure out why this app keeps running. I've tried using the why-is-node-running package but I'm not perfectly sure how to read the output properly. Here's the first output of it:
There are 30 handle(s) keeping the process running
# TCPWRAP
/node_modules/mongodb/lib/core/connection/connect.js:269 - socket = tls.connect(parseSslOptions(family, options));
/node_modules/mongodb/lib/core/connection/connect.js:29 - makeConnection(family, options, cancellationToken, (err, socket) => {
/node_modules/mongodb/lib/core/sdam/monitor.js:182 - connect(monitor.connectOptions, monitor[kCancellationToken], (err, conn) => {
/node_modules/mongodb/lib/core/sdam/monitor.js:206 - checkServer(monitor, e0 => {
/node_modules/mongodb/lib/core/sdam/monitor.js:92 - monitorServer(this);
My guess is it has something to do with MongoDB not closing properly. Although, when I removed all of the other functions between opening the client and closing it, it opened and closed perfectly.
Adding process.exit() at the end closes program properly, but I'd like to figure out why it isn't closing.
A summary of the app is that it is getting data from MongoDB, cleaning it, and then writing it into Firestore - so a lot of async actions going on, but I didn't see Firestore-related stuff pop up in the why-is-node-running logs.
const GrabStuffFromDBToCalculate = require("./helpers/GrabStuffFromDBToCalculate");
const SendToFirestore = require("./helpers/SendToFirestore");
const log = require("why-is-node-running");
const { MongoClient } = require("mongodb");
require("dotenv").config();
const main = async () => {
try {
const client = await MongoClient.connect(process.env.MONGODB_URI, {
useNewUrlParser: true,
useUnifiedTopology: true
});
const collection = await client.db("test").collection("testcollection");
const trip_object = await GrabStuffFromDBToCalculate(collection);
SendToFirestore(trip_object);
client.close();
log(); // "There are 30 handle(s) keeping the process running including node_modules/mongodb/lib/core/connection/connect.js:269 - socket = tls.connect(parseSslOptions(family, options));"
// process.exit() // this closes everything but I'd rather not have to use this
} catch (err) {
console.log(err);
client.close();
}
};
const runAsync = async () => {
await main(); // this exists because I'm usually running multiple main() functions
};
runAsync();
SendToFirestore code:
const firebase = require("firebase");
const firebaseConfig = require("../config");
module.exports = SendToFirestore = trip_object => {
if (!firebase.apps.length) {
firebase.initializeApp(firebaseConfig);
}
const db = firebase.firestore();
db.doc(`hello/${object._id}`).set({
objectid:object._id
});
};
GrabStuffFromDBToCalculate code (way simplified):
module.exports = GrabStuffFromDBToCalculate = async collection => {
const cursor = await collection
.aggregate([
// does a bunch of stuff here
])
.toArray();
const newObj = cursor[0];
return newObj;
};
Making my comment into an answer since it led to the missing piece.
Node does not shut down because you have an open Firestore connection. You will have to call terminate to allow the SDK to shut down and release resources:
db.terminate();
Which is relevant for allowing nodejs to shut itself down automatically.
Also, I'm not sure you understood that I was suggesting that you use await as in
await client.close()
before calling log() so you are sure that the client connection has been closed before you do the logging. client.close() is an asynchronous method so your original code would log() before that close was complete.
I am familiar with how to get documents from the Mongo shell, but am having difficulty getting documents using find() when connecting via Node.
What I'm getting right now looks like a lot of cursor info, but not the actual documents.
What do I need to change with the following code so that I get the actual documents logged to the console for 'results'?
const config = require('./../../../configuration');
const url = config.get('MONGO_URL');
const dbName = config.get('MONGO_DATABASE');
const MongoClient = require('mongodb').MongoClient;
const client = new MongoClient(url);
module.exports = async function getSchedules() {
let results;
return new Promise((resolve, reject) => {
client.connect(async function (err) {
if (err) return reject(err);
try {
const db = await client.db(dbName);
results = await db.collection('schedules').find();
} catch (error) {
return reject(error);
}
return resolve(results);
});
});
};
... And here's where I actually try and get the documents:
async function getSchedulesFromDB() {
await getSchedules().then((schedules => {
console.log('schedules: ', schedules); // expect result here
return schedules;
}));
}
When I used this same kind of code structure on a findOne(), it worked. But here when using a find() it is not. What am I missing? Does find() work fundamentally differently than findOne()?
Yes. find() returns a cursor over which you must iterate. findOne() returns a single doc, not a cursor. If you want an array of results, you must "build it yourself" by iterating the cursor, something like:
results = [];
db.collection('schedules').find().forEach(function(d) { results.push(d); });