Implementing bulk insert in mongodb using mongoose.
The data is successfully getting saved in DB but the express is not sending the response back.
Tried it using insertMany also tried bulkWrite
try {
await Collection.insertMany(docsToBeInserted);
console.log("Insert Successful");
res.status(200).send('ok');
} catch(err) {
res.status(500).json({err})
}
Insert Successful is getting print and all the documents are successfully inserted in database but in client the call never completes after some time the call fails throwing ERR_EMPTY_RESPONSE
I usually handle this specific case in this way:
return Collection.insertMany(docsToBeInserted)
.then(res => {
res.status(200).send('ok');
})
.catch(err => {
res.status(500).json({ err })
});
I have removed async-await and leveraged Promise returned by insertMany into sending the apt response according to your sample code.
You can also replace res.status(200).send('ok') with res.send('ok') as Express will set Http status to 200.
I would recommend adding console.error(err) statement to your error handling on the code for the stack trace.
You can store the result of insertMany into a variable, and send it as response like this:
try {
const response = await Collection.insertMany(docsToBeInserted);
console.log("Insert Successful");
return res.status(200).send(response.data);
} catch (err) {
return res.status(500).json({ err })
}
Related
I am currently writing a route which allows me to recieve information from a stored procudre I have in a database. I have written a request in AngularJS and a route in NodeJS but I am just recieving a pending request in the chrome Network developer window. I can see that the console.log in the NodeJs app has the data I require so it has retrieved it but there is nothing coming back in any of the console logs in the the AngularJS app.
Here is the code for the both the angularJS app and the Node App:
AnglaurJS:
checkForReplenishmentEmptyTrolley = async () => {
LIBRIS.extensions.openLoadingModal();
console.log('in checkForReplenishmentEmptyTrolley');
try {
const varPromise = await $http.get(`${LIBRIS.config.stockService}stockMovement/checkForUnattachedTrolley`)
.then((response) => {
console.log(response);
// Request completed successfully
}, (error) => {
// Request error
console.log(error);
});
console.log(varPromise.data);
// 1. check that there are no ghost replenish - lines 1-15
console.log('in try/catch');
console.log('promise', varPromise);
} catch (error) {
console.log(error);
}
},
NodeJS code:
app.get(`${ROUTE}/attachTrolley`, async function(req, res){
const newRequest = await DB.newRequest();
console.log('we have made it to the route');
try {
console.log('we have made it to the Try/Catch route');
newRequest.input();
const record = await newRequest.execute('dbo.usp_STK_CheckForUnattachedTrolley');
res.json(record)
console.log(record, 'record');
} catch (err){
handleError(res, err);
console.log(err);
}
});
The problem is that you are doing a .then on a awaited promises and not returning anything from that. You have two choice here
Either return response from then so when you try to access the value here console.log(varPromise.data); it works.
Or remove the .then alltogather as it is not required because you are awaiting it any ways.
Basically just do this
checkForReplenishmentEmptyTrolley = async () => {
LIBRIS.extensions.openLoadingModal();
console.log("in checkForReplenishmentEmptyTrolley");
try {
const varPromise = await $http.get(`${LIBRIS.config.stockService}stockMovement/checkForUnattachedTrolley`);
console.log(varPromise.data);
// 1. check that there are no ghost replenish - lines 1-15
console.log("in try/catch");
console.log("promise", varPromise);
} catch (error) {
console.log(error);
}
};
Hope this fixes your issue.
Solved it! I had no return statement in my route!
I need to store my values from the request body to the cloud firestore and sent back the foruminsertdata.Name back in the response. But I am not able to do this.
const functions = require('firebase-functions');
const admin =require("firebase-admin");
admin.initializeApp(functions.config().firebase);
const db = admin.firestore();
exports.helloWorld = functions.https.onRequest((req, res) => {
if(req.method === 'POST'){
foruminsertdata = req.body;
db.collection('forum').add({
Name: foruminsertdata.Name,
Description: foruminsertdata.Description,
Heading: foruminsertdata.Heading,
PostedOn: foruminsertdata.PostedOn,
Status: foruminsertdata.Status,
})
.then(ref => {
console.log('Added document with ID: ', ref.id);
return res.status(200).json(
{
message: foruminsertdata.Name
});
})
.catch(err => {
console.log('Error getting documents', err);
});
res.json({
message: foruminsertdata.Status,
});
}
})
I don't know what is happening...Whatever I do I always get the output as
{
message: foruminsertdata.Status,
}
in which "foruminsertdata.Status" has some value that I give
but what I expect the output as
{
message: foruminsertdata.Name
}
Your function is immediately returning foruminsertdata.Status to the client without waiting for the promise from the database operations to resolve. Any function that returns a promise is asynchronous and returns immediately. Execution will continue in the callbacks you attach to it.
I'm not sure why you have two calls to res.json() in your code, but if you want to send a response only after your query completes, you'll remove the second one and just send a response after the query is done. You will probably also want to send a response in the catch callback as well to indicate an error.
I want to test the saveRecords function for the failure and before that, I have to authenticate and connect the MongoDB. This is the code.
before(() => {
sinon.stub(Authentication, 'authenticate').returns(true);
sinon.stub(mongodb, 'connect').resolves("connected");
sinon.stub(models, 'saveRecords').throws(new Error("Error while saving record"));
});
it('Should error out if record is not inserted into the mongodb
collection', () => {
orderWebhook(req, res)
expect(res.result).to.contain("Error while saving record");
});
Here is the code I am testing.
exports.orderWebhook = async (req, res) => {
try {
const isAuthenticated = Authentication.authenticate(req);
if (isAuthenticated) {
await mongodb.connect();
await models.saveRecords(req.body");
res.status(200).send('Saved Successfully!');
} else {
res.status(403).send('Error! Auth failed!');
}
} catch (error) {
res.status(400).send(error.message);
}
}
I am assuming that this code will stub the authenticate then connect MongoDB and then try to insert the record and throw the error. But it is running two times when I debug with the VSCode debugger.
The first time it is returning true for the authenticate function and not resolving the MongoDB connect and return to expect immediately.
The second time it is running all three properly and throwing the expected error.
It is failing when I run the test in the terminal, What could be the issue?
Update: I noticed that the problem is related to the promise. Sinon is resolving the request and I am using await mongodb.connect(); but it is not working as expected, and if I remove await and return value instead of promise then it works.
the answer to this question: How to get node to exit when mongo connect fails contains async/wait code for a connection
however, my code (running on node v11.5.0 and mongodb v3.1.13) is failing to catch:
(async function() {
let db;
try {
db = await MongoClient.connect(uri, { useNewUrlParser: true });
console.log("RETURN", db);
} catch (err) {
console.log('EXITING');
process.exit(1);
}
}());
to prove the point I intentionally give a uri without credentials:
mongodb://undefined#cluster0-shard-00-00-z4j9e.azure.mongodb.net:27017,cluster0-shard-00-01-z4j9e.azure.mongodb.net:27017,cluster0-shard-00-02-z4j9e.azure.mongodb.net:27017/test?ssl=true&replicaSet=Cluster0-shard-0&authSource=admin&retryWrites=true
and what I get is output like this:
/Users/ekkis/dev/mongo/node_modules/mongodb/lib/topologies/replset.js:346
throw err;
^
MongoError: password must be a string
at passwordDigest (/Users/ekkis/dev/mongo/node_modules/mongodb-core/lib/auth/scram.js:63:43)
at ScramSHA1.ScramSHA.auth (/Users/ekkis/dev/mongo/node_modules/mongodb-core/lib/auth/scram.js:175:25)
at authenticate (/Users/ekkis/dev/mongo/node_modules/mongodb-core/lib/connection/pool.js:232:17)
at authenticateLiveConnections (/Users/ekkis/dev/mongo/node_modules/mongodb-core/lib/connection/pool.js:819:7)
at /Users/ekkis/dev/mongo/node_modules/mongodb-core/lib/connection/pool.js:864:5
at waitForLogout (/Users/ekkis/dev/mongo/node_modules/mongodb-core/lib/connection/pool.js:855:34)
at Pool.auth (/Users/ekkis/dev/mongo/node_modules/mongodb-core/lib/connection/pool.js:862:3)
at Server.auth (/Users/ekkis/dev/mongo/node_modules/mongodb-core/lib/topologies/server.js:931:20)
at auth (/Users/ekkis/dev/mongo/node_modules/mongodb-core/lib/topologies/replset.js:1474:19)
at ReplSet.auth (/Users/ekkis/dev/mongo/node_modules/mongodb-core/lib/topologies/replset.js:1492:5)
so if the error had been caught, the console should have displayed the word 'EXITING', but does not. additionally, I contend an exception was thrown because otherwise the returned value would have been printed, which it was not
how can this be? what do I need to do to get it to work?
* Appendix I *
In fact, the promises version of this exhibits the same odd behaviour, it doesn't catch:
MongoClient
.connect(uri, { useNewUrlParser: true })
.then(dbc => {
console.log('SUCCESS');
})
.catch(err => {
console.log('EXITING');
process.exit(1);
});
and yes, I tested the callback version, which also suffers the same malady. Incidentally, passing an empty string for the uri works well. I don't get it
* Appendix II *
In fact, the problem seems to be particular to the credentials passed i.e. if I pass:
mongodb://x:y#cluster0-shard-[...]
I catch a "MongoError: authentication fail" as expected. passing:
mongodb://#cluster0-shard-[...]
interestingly returns a connection but credentials missing a ":" fail in this odd way, so:
mongodb://ekkis#cluster0-shard-[...]
fails to catch
Looks to me like it's a bug with however MongoClient is setting up its connections. You won't be able to use try & catch to handle asynchronously thrown errors within MongoClient code.
const {MongoClient} = require("mongodb");
process.on("uncaughtException", (err) => {
console.log("process err", err);
process.exit(1)
})
async function run () {
let db;
try {
// connection url will throw because password isn't provided
db = await MongoClient.connect("mongodb://myUsername:#localhost", { useNewUrlParser: true });
} catch (err) {
console.log('Exiting from thrown error', err);
process.exit(1);
}
}
run();
Here's a simplified example of what's happening -- the error will end up "uncaught" and caught by the uncaughtException handler
process.on("uncaughtException", (err) => console.log("uncaught", err));
try {
setTimeout(() => {
throw new Error("asynchronously thrown error");
})
} catch (err) {
console.log("Error will not be caught here")
}
When I was using mongo version 3.6.1, it was not an issue and i was able to handle the thrown exception using catch. But after a few days on another project this type of error occurred and was showing as the error thrown from
%project_folder%/node_modules/mongodb/lib/utils.js:668
(Don't mind about the slash in the path string.)
The mongodb version this time is 3.6.3. Upon checking the code in that file at the mentioned line I found the below piece of code. where the caught error is again being thrown.
fn(function(err, res) {
if (err != null) {
try {
callback(err);
} catch (error) {
return process.nextTick(() => {
throw error;
});
}
return;
}
callback(err, res);
});
I changed the throw error to console.error(error) and the problem got resolved. But still you need to be caught somewhere in our code where connect function is called.
I think this is because the above piece of code is checking for the presence of error and passing it to the callback function and then again throwing the same error again. I suppose it is the MongoDB driver developer community's responsibility to resolve this issue.
I am new to Node.js, and try to use Node.js Mongodb drive API changeStream (https://mongodb.github.io/node-mongodb-native/3.0/api/ChangeStream.html#hasNext). I do not know how to deal with Nodejs return Promise. My code is as below.
change_stream.hasNext().then(function(){
const change = change_stream.next();
console.log(change)
})
.catch(function (error) {
console.error('An error occurred', error);
});
The output is:
Promise { }
Could anyone help? Thanks!