I am using promise in bluebird to send mysql queries and manage the control flow & errors. Here is what it looks like:
sendQuery1(..)
.then(sendQuery2(..))
.then(function(results from last query){
if(rain){
res.render(...)
}else{
/*
I need to send additional 2 queries here
*/
}
}).catch(errors);
promise chain is very convenient, but I found out the error handling will be a messy if there are multiple subchains inside.
For here I probably need to write the following inside the /* */
return sendQuery3(..)
.then(sendQuery4(..))
.then(function(..){
res.render(".....")
}).catch(error2);
Are there any better ways to handle this type of problems?
I don't exactly see your problem here. When you have a chain inside a chain (which you normally don't, you should check your architecture right there), you can just catch the errors like shown.
I'd advise you use a global (or local) error handler function, and pass that to the catch-function. Therefore, even when you have multiple catches, you can use the same error-handler.
The in my opinion best solution would be to create a "promise chain bypass", therefore using catch to skip certain parts based on your condition. If this is not what you are looking for, please specify your problem.
You will catch the error, there's no problem in your code. Yup it gets messy but you can go with the following approach to keep the code clean and still implement your logic:
sendQuery1(..)
.then(sendQuery2(..))
.then(function(results from last query){
if(rain){
res.render(...)
throw new Error('breakChain'); //intentionally throwing error to skip the remaining chain
}
return; //will act like 'else'
})
.then(sendQuery3(..))
.then(sendQuery4(..))
.catch(function (e) {
if(e.message != 'breakChain') //act on error if it was other than 'breakChain'
throw e;
});
What you are discussing here is branching of the chain based on some logic condition. It is generally better to actually branch the chain rather than throw an error just to abort the rest of the chain. This keeps errors as errors rather than the scheme that Shaharyar proposed of synthesizing an error that isn't really an error.
You can branch the chain by returning a new promise chain from within a .then() handler like this:
sendQuery1(..).then(function(r1) {
return sendQuery2(...);
}).then(function(r2){
if (rain){
// processing is done, so just render
res.render(...)
} else {
// return promise here to attach this new branch to the original chain
return sendQuery3(..).then(sendQuery4).then(function() {
// process last query
});
}
}).catch(errors);
FYI, since you've only posted pseudo-code, not real code and we can't see which functions need access to which prior results in order to do their work, we can't fully optimize the code. This is to show an example of how branching works inside a promise chain. That's the principle you probably want to use. If you show your real code, we can offer a much more specific and optimized answer.
Related
I have an updateDocument method in a class for a service layer in a node.js, express, mongoose application. I'm wondering what is the best practice for handing cases where the update method didn't update a document, for example if the wrong id is passed in.
Version 1: If the update wasn't successful, return an object with success: false:
async updateDocument(id, updates) {
const output = await this.DocumentModel.updateOne({ _id: id }, updates);
let message = 'Something went wrong';
let success = false;
let updatedItem = null;
if (output.nModified) {
message = 'Successfully updated document.';
success = true;
updatedItem = await this.getDocument(id);
}
return { message, success, updatedItem};
}
Version 2: If the update wasn't successful, throw an error:
async updateDocument(id, updates) {
const output = await this.DocumentModel.updateOne({ _id: id }, updates);
let updatedItem;
if (output.nModified) {
updatedItem = await this.getDocument(id);
} else{
throw new Error("The document wasn't updated")
}
return updatedItem;
}
Do you always throw an exception when the input, such as a bad id, isn't correct? Or could you return information about the update being a success or not? As newbie node.js developer, I'm not sure I am grasping the full picture enough to recognize problems with either method.
There is no golden way, only principles that lead to robust and well-maintainable software.
Generally, you should use a try-catch-statement for all kinds of errors that are not in your control (connections, disk space, credentials, ...) . The errors should then be handled as soon as possible, but not before. The reason for this is that you often don't know, yet, how to handle an error in an appropriate manner at a lower layer.
For logical "errors" that you can expect (wrong input format, missing username, unknown options, ...), you should use an if-statement or a validation function and then throw an error, if anything is not as expected.
In your case, you should check, if the methods updateOne or getDocument can throw errors. If yes, you should wrap these functions with a try-catch-statement.
A few more tips:
Both versions of your code are good. But I would prefer version 2 because it is more concise.
If you are sure that there is always an output object, you can destruct the nModified property like this:
const { nModified } = await this.DocumentModel.updateOne({ _id: id }, updates);
If you use a negative if-statement, you can reduce the depth of indentation and you can use const variables:
if (!nModified) {
throw new Error("The document wasn't updated")
}
const updatedItem = await this.getDocument(id);
Now, you could also directly return this.getDocument(id) and don't need the variable updatedItem anymore.
You can finally handle your errors in your controller classes.
You can use custom error classes to be consistent in your error handling and error messages all over your app.
I hope this is at least a bit helpful.
References
These are some similar questions with good answers. But you need to take care because many code examples are not in modern JavaScript.
A general discussion about the pros and cons of Error-Handling vs.
if-else-statements is done here:
What is the advantage of using try {} catch {} versus if {} else {}
Error-Handling in Node.js is discussed here in this thread:
Node.js Best Practice Exception Handling
It seemed like there were a lot of different opinions on this and not one go-to method. Here's some information I found and what I ended up doing.
When to throw an exception?
Every function asks a question. If the input it is given makes that question a fallacy, then throw an exception. This line is harder to draw with functions that return void, but the bottom line is: if the function's assumptions about its inputs are violated, it should throw an exception instead of returning normally.
Should a retrieval method return 'null' or throw an exception when it can't produce the return value?
Answer 1:
Whatever you do, make sure you document it. I think this point is more important than exactly which approach is "best".
Answer 2:
If you are always expecting to find a value then throw the exception if it is missing. The exception would mean that there was a problem.
If the value can be missing or present and both are valid for the application logic then return a null.
More important: What do you do other places in the code? Consistency is important.
Where should exceptions be handled?
Answer 1: in the layer of code that can actually do something about the error
Exceptions should be handled in the layer of code that can actually do something about the error.
The "log and rethrow" pattern is often considered an antipattern (for exactly the reason you mentioned, leads to a lot of duplicate code and doesn't really help you do anything practical.)
The point of an exception is that it is "not expected". If the layer of code you are working in can't do something reasonable to continue successful processing when the error happens, just let it bubble up.
If the layer of code you are working in can do something to continue when the error happens, that is the spot to handle the error. (And returning a "failed" http response code counts as a way to "continue processing". You are saving the program from crashing.)
-source: softwareengineering.stackexchange
Answer 2: Handle errors centrally, not within a middleware
Without one dedicated object for error handling, greater are the chances of important errors hiding under the radar due to improper handling. The error handler object is responsible for making the error visible, for example by writing to a well-formatted logger, sending events to some monitoring product like Sentry, Rollbar, or Raygun. Most web frameworks, like Express, provide an error handling middleware mechanism. A typical error handling flow might be: Some module throws an error -> API router catches the error -> it propagates the error to the middleware (e.g. Express, KOA) who is responsible for catching errors -> a centralized error handler is called -> the middleware is being told whether this error is an untrusted error (not operational) so it can restart the app gracefully. Note that it’s a common, yet wrong, practice to handle errors within Express middleware – doing so will not cover errors that are thrown in non-web interfaces.
-source; Handle errors centrally, not within a middleware
More: Best Practice Node.js: Error Handling
So it seems like these two principles disagree. #1 says to handle it right away if you can. So for me it would be in the service layer. But the #2 says handle it centrally, like in the server file. I went with #2.
My decision: throw the error in a custom error class
It combined a few methods people suggested. I am throwing the error, but I'm not "log and rethrow"-ing, as the answer above warned against. Instead, I put the error in a custom error with more information and throw that. It is logged and handled centrally.
So first in my service layer this is how an error is thrown:
async addUser(user) {
let newUser;
try {
newUser = await this.UserModel.create(user);
} catch (err) {
throw new ApplicationError( // custom error
{
user, // params that are useful
err, //original error
},
`Unable to create user: ${err.name}: ${err.message}` // error message
);
}
return newUser;
}
ApplicationError is a custom error class that takes an info object and a message. I got this idea from here:
In this pattern, we would start our application with an ApplicationError class this way we know all errors in our applications that we explicitly throw are going to inherit from it. So we would start off with the following error classes:
-source: smashingmagazine
You could put other helpful information in your custom error class, even maybe what EJS template to use! So you could really handle the error creatively depending on how you structure your custom error class. I don't know if that's "normal", maybe it's not SOLID to include the EJS template, but I think it's an interesting concept to explore. You could think about other ways that might be more SOLID to dynamically react to errors.
This is the handleError file for now, but I will probably change it up to work with the custom error to create a more informative page. :
const logger = require("./logger");
module.exports = (err, req, res, next) => {
if (res.headersSent) {
return next(err);
}
logger.log("Error:", err);
return res.status(500).render("500", {
title: "500",
});
};
Then I add that function to my server file as the last middleware:
app.use(handleError);
In conclusion, it seems like there's a bit of disagreement on how to handle errors though it seems more people think you should throw the error and probably handle it centrally. Find a way that works for you, be consistent, and document it.
I am still quite new to Node.js and can't seem to find anything to help me around this.
I am having an issue of getting the query from my last record and adding it to my variable.
If I do it like below: -
let lastRecord = Application.find().sort({$natural:-1}).limit(1).then((result) => { result });
Then I get the value of the variable showing in console.log as : -
Promise { <pending> }
What would I need to do to output this correctly to my full data?
Here is it fixed:
Application.findOne().sort({$natural:-1}).exec().then((lastRecord) => {
console.log(lastRecord); // "lastRecord" is the result. You must use it here.
}, (err) => {
console.log(err); // This only runs if there was an error. "err" contains the data about the error.
});
Several things:
You are only getting one record, not many records, so you just use findOne instead of find. As a result you also don't need limit(1) anymore.
You need to call .exec() to actually run the query.
The result is returned to you inside the callback function, it must be used here.
exec() returns a Promise. A promise in JavaScript is basically just a container that holds a task that will be completed at some point in the future. It has the method then, which allows you to bind functions for it to call when it is complete.
Any time you go out to another server to get some data using JavaScript, the code does not stop and wait for the data. It actually continues executing onward without waiting. This is called "asynchronisity". Then it comes back to run the functions given by then when the data comes back.
Asynchronous is simply a word used to describe a function that will BEGIN executing when you call it, but the code will continue running onward without waiting for it to complete. This is why we need to provide some kind of function for it to come back and execute later when the data is back. This is called a "callback function".
This is a lot to explain from here, but please go do some research on JavaScript Promises and asynchronisity and this will make a lot more sense.
Edit:
If this is inside a function you can do this:
async function someFunc() {
let lastRecord = await Application.findOne().sort({$natural:-1}).exec();
}
Note the word async before the function. This must me there in order for await to work. However this method is a bit tricky to understand if you don't understand promises already. I'd recommend you start with my first suggestion and work your way up to the async/await syntax once you fully understand promises.
Instead of using .then(), you'll want to await the record. For example:
let lastRecord = await Application.find().sort({$natural:-1}).limit(1);
You can learn more about awaiting promises in the MDN entry for await, but the basics are that to use a response from a promise, you either use await or you put your logic into the .then statement.
This is a common process for me in my previous works, so i usually have a very complex use case take for example
async function doThis(){
for (100x) {
try {
insertToDatabase()
await selectAndManipulateData()
createEmailWorker()
/** and many more **/
} catch {
logToAFile()
}
}
}
The code works, but its complicated 1 function doing all the things, the only reason i do this is because i can verify in real time if one function fails i can make sure the other function wont run so there wont be any incorrect data.
What i want to know is, what is the best architecture in defining a project structure that is not sacrificing the data integrity? (or is it already good enough?)
const doThis = async() => {
try {
for (100x) {
await insertToDatabase();
await selectAndManipulateData();
await createEmailWorker();
/** and many more **/
}
}
catch {
await logToAFile();
}
}
The best way of doing this is, you should always use await to call any function and make sure to with es6 syntax's as it gives a lot more feature. Your function should always be an async.
Always put your loop in try catch as it will give you any error in catch and it will calling function specific.
Actually, I would separate the persistence, manipulation and email jobs. Consider storing your data is a single responsibility. In addition to this, your modification and email workers should work as scheduled jobs. Once the jobs triggered, they should check if there is data related to its responsibility.
Another way is changing these scheduled jobs with triggered jobs. You can build a chain of responsibility that triggers next jobs and they would decide to work or not.
I am making two axios calls like this
axios
.all([call1(), call2()])
.then(axios.spread(function(Resp1, Resp2) {}).catch(onError)
If one of the calls fails with a 500 server error, flow goes to catch block. I want it to go to then block even if one call fails and only go to catch if both fail. How can I do this?
It is not possible with .all of axios. But you could do something else. Create a method which gets a promise as param, and executes it, and in case of failure, just return null or whatever. This way you will be able to control the error case.
For that I recommend you read this article:
https://pouchdb.com/2015/05/18/we-have-a-problem-with-promises.html
More precicly subtitle Rookie mistake #5: using side effects instead of returning
I'm still learning the node.js ropes and am just trying to get my head around what I should be deferring, and what I should just be executing.
I know there are other questions relating to this subject generally, but I'm afraid without a more relatable example I'm struggling to 'get it'.
My general understanding is that if the code being executed is non-trivial, then it's probably a good idea to async it, as to avoid it holding up someone else's session. There's clearly more to it than that, and callbacks get mentioned a lot, and I'm not 100% on why you wouldn't just synch everything. I've got some ways to go.
So here's some basic code I've put together in an express.js app:
app.get('/directory', function(req, res) {
process.nextTick(function() {
Item.
find().
sort( 'date-modified' ).
exec( function ( err, items ){
if ( err ) {
return next( err );
}
res.render('directory.ejs', {
items : items
});
});
});
});
Am I right to be using process.nextTick() here? My reasoning is that as it's a database call then some actual work is having to be done, and it's the kind of thing that could slow down active sessions. Or is that wrong?
Secondly, I have a feeling that if I'm deferring the database query then it should be in a callback, and I should have the actual page rendering happening synchronously, on condition of receiving the callback response. I'm only assuming this because it seems like a more common format from some of the examples I've seen - if it's a correct assumption can anyone explain why that's the case?
Thanks!
You are using it wrong in this case, because .exec() is already asynchronous (You can tell by the fact that is accepts a callback as a parameter).
To be fair, most of what needs to be asynchronous in nodejs already is.
As for page rendering, if you require the results from the database to render the page, and those arrive asynchronously, you can't really render the page synchronously.
Generally speaking it's best practice to make everything you can asynchronous rather than relying on synchronous functions ... in most cases that would be something like readFile vs. readFileSync. In your example, you're not doing anything synchronously with i/o. The only synchronous code you have is the logic of your program (which requires CPU and thus has to be synchronous in node) but these are tiny little things by comparison.
I'm not sure what Item is, but if I had to guess what .find().sort() does is build a query string internally to the system. It does not actually run the query (talk to the DB) until .exec is called. .exec takes a callback, so it will communicate with the DB asynchronously. When that communication is done, the callback is called.
Using process.nextTick does nothing in this case. That would just delay the calling of its code until the next event loop which there is no need to do. It has no effect on synchronicity or not.
I don't really understand your second question, but if the rendering of the page depends on the result of the query, you have to defer rendering of the page until the query completes -- you are doing this by rendering in the callback. The rendering itself res.render may not be entirely synchronous either. It depends on the internal mechanism of the library that defines the render function.
In your example, next is not defined. Instead your code should probably look like:
app.get('/directory', function(req, res) {
Item.
find().
sort( 'date-modified' ).
exec(function (err, items) {
if (err) {
console.error(err);
res.status(500).end("Database error");
}
else {
res.render('directory.ejs', {
items : items
});
}
});
});
});