I know how to handle specific errors in promises but I sometimes have pieces of code that looks like this:
somePromise.then(function(response){
otherAPI(JSON.parse(response));
});
Sometimes, I get invalid JSON which causes a silent failure here when JSON.parse throws. In general I have to remember to add a .catch handler to every single promise in my code and when I don't I have no way to find out where I forgot one.
How do I find these suppressed errors in my code?
Edit
We've finally fixed this in Node.js 15, it took 5 years but native promise rejections now behave like uncaught exceptions - so it's fine to just add a process.on('uncaughtException' handler.
In Modern Node.js
Starting with io.js 1.4 and Node 4.0.0 you can use the process "unhandledRejection" event:
process.on("unhandledRejection", function(reason, p){
console.log("Unhandled", reason, p); // log all your errors, "unsuppressing" them.
throw reason; // optional, in case you want to treat these as errors
});
This puts an end to unhandled rejections problems and the difficulty of tracking them down in your code.
In Older NodeJS
These events were not yet back-ported to older versions of NodeJS and are unlikely to be. You can use a promise library that extends the native promise API such as bluebird which will fire the same events as there are in modern versions.
It's also worth mentioning that there are several userland promise libraries that offer the unhandled rejection detection facilities and much more such as bluebird (which also has warnings) and when.
Related
We have a fairly complex code base in NodeJS that runs a lot of Promises synchronously. Some of them come from Firebase (firebase-admin), some from other Google Cloud libraries, some are local MongoDB requests. This code works mostly fine, millions of promises being fulfilled over the course of 5-8 hours.
But sometimes we get promises rejected due to external reasons like network timeouts. For this reason, we have try-catch blocks around all of the Firebase or Google Cloud or MongoDB calls (the calls are awaited, so a rejected promise should be caught be the catch blocks). If a network timeout occurs, we just try it again after a while. This works great most of the time. Sometimes, the whole thing runs through without any real problems.
However, sometimes we still get unhandled promises being rejected, which then appear in the process.on('unhandledRejection', ...). The stack traces of these rejections look like this, for example:
Warn: Unhandled Rejection at: Promise [object Promise] reason: Error stack: Error:
at new ApiError ([repo-path]\node_modules\#google-cloud\common\build\src\util.js:59:15)
at Util.parseHttpRespBody ([repo-path]\node_modules\#google-cloud\common\build\src\util.js:194:38)
at Util.handleResp ([repo-path]\node_modules\#google-cloud\common\build\src\util.js:135:117)
at [repo-path]\node_modules\#google-cloud\common\build\src\util.js:434:22
at onResponse ([repo-path]\node_modules\retry-request\index.js:214:7)
at [repo-path]\node_modules\teeny-request\src\index.ts:325:11
at runMicrotasks (<anonymous>)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
This is a stacktrace which is completely detached from my own code, so I have absolutely no idea where I could improve my code to make it more robust against errors (error message seems to be very helpful too).
Another example:
Warn: Unhandled Rejection at: Promise [object Promise] reason: MongoError: server instance pool was destroyed stack: MongoError: server instance pool was destroyed
at basicWriteValidations ([repo-path]\node_modules\mongodb\lib\core\topologies\server.js:574:41)
at Server.insert ([repo-path]\node_modules\mongodb\lib\core\topologies\server.js:688:16)
at Server.insert ([repo-path]\node_modules\mongodb\lib\topologies\topology_base.js:301:25)
at OrderedBulkOperation.finalOptionsHandler ([repo-path]\node_modules\mongodb\lib\bulk\common.js:1210:25)
at executeCommands ([repo-path]\node_modules\mongodb\lib\bulk\common.js:527:17)
at executeLegacyOperation ([repo-path]\node_modules\mongodb\lib\utils.js:390:24)
at OrderedBulkOperation.execute ([repo-path]\node_modules\mongodb\lib\bulk\common.js:1146:12)
at BulkWriteOperation.execute ([repo-path]\node_modules\mongodb\lib\operations\bulk_write.js:67:10)
at InsertManyOperation.execute ([repo-path]\node_modules\mongodb\lib\operations\insert_many.js:41:24)
at executeOperation ([repo-path]\node_modules\mongodb\lib\operations\execute_operation.js:77:17)
At least this error message says something.
All my Google Cloud or MongoDB calls have await and try-catch blocks around them (and the MongoDB reference is recreated in the catch block), so if the promise were rejected inside those calls, the error would be caught in the catch block.
A similar problem sometimes happens in the Firebase library. Some of the rejected promises (e.g. because of network errors) get caught by our try-catch blocks, but some don't, and I have no possibility to improve my code, because there is no stack trace in that case.
Now, regardless of the specific causes of these problems: I find it very frustrating that the errors just happen on a global scale (process.on('unhandledRejection', ...), instead of at a location in my code where I can handle them with a try-catch. This makes us lose so much time, because we have to restart the whole process when we get into such a state.
How can I improve my code such that these global exceptions do not happen again? Why are these errors global unhandled rejections when I have try-catch blocks around all the promises?
It might be the case that these are the problems of the MongoDB / Firebase clients: however, more than one library is affected by this behavior, so I'm not sure.
a stacktrace which is completely detached from my own code
Yes, but does the function you call have proper error handling for what IT does?
Below I show a simple example of why your outside code with try/catch can simply not prevent promise rejections
//if a function you don't control causes an error with the language itself, yikes
//and for rejections, the same(amount of YIKES) can happen if an asynchronous function you call doesn't send up its rejection properly
//the example below is if the function is returning a custom promise that faces a problem, then does `throw err` instead of `reject(err)`)
//however, there usually is some thiAPI.on('error',callback) but try/catch doesn't solve everything
async function someFireBaseThing(){
//a promise is always returned from an async function(on error it does the equivalent of `Promise.reject(error)`)
//yet if you return a promise, THAT would be the promise returned and catch will only catch a `Promise.reject(theError)`
return await new Promise((r,j)=>{
fetch('x').then(r).catch(e=>{throw e})
//unhandled rejection occurs even though e gets thrown
//ironically, this could be simply solved with `.catch(j)`
//check inspect element console since stackoverflow console doesn't show the error
})
}
async function yourCode(){
try{console.log(await someFireBaseThing())}
catch(e){console.warn("successful handle:",e)}
}
yourCode()
Upon reading your question once more, it looks like you can just set a time limit for a task and then manually throw to your waiting catch if it takes too long(because if the error stack doesn't include your code, the promise that gets shown to unhandledRejection would probably be unseen by your code in the first place)
function handler(promise,time){ //automatically rejects if it takes too long
return new Promise(async(r,j)=>{
setTimeout(()=>j('promise did not resolve in given time'),time)
try{r(await promise)} catch(err){j(err)}
})
}
async function yourCode(){
while(true){ //will break when promise is successful(and returns)
try{return await handler(someFireBaseThing(...someArguments),1e4)}
catch(err){yourHandlingOn(err)}
}
}
Elaborating on my comment, here's what I would bet is going on: You set up some sort base instance to interact with the API, then use that instance moving forward in your calls. That base instance is likely an event emitter that itself can emit an 'error' event, which is a fatal unhandled error with no 'error' listener setup.
I'll use postgres for an example since I'm unfamiliar with firebase or mongo.
// Pool is a pool of connections to the DB
const pool = new (require('pg')).Pool(...);
// Using pool we call an async function in a try catch
try {
await pool.query('select foo from bar where id = $1', [92]);
}
catch(err) {
// A SQL error like no table named bar would be caught here.
// However a connection error would be emitted as an 'error'
// event from pool itself, which would be unhandled
}
The solution in the example would be to start with
const pool = new (require('pg')).Pool(...);
pool.on('error', (err) => { /* do whatever with error */ })
Top-level await support was added to Node.js in 14.3.0 via --experimental-top-level-await and later to --harmony-top-level-await.
The Problem
I need to use a top level await in my ESM script file, if it is supported by the current Node.js runtime. And further, I need to set a boolean flag to indicate that the promise was successfully awaited at the top level.
An example of what I mean:
let topLevelAwaitEnabled;
try {
await Promise.resolve(); // replaced with an actual promise
topLevelAwaitEnabled = true;
} catch (ignored) {
topLevelAwaitEnabled = false;
}
console.log(topLevelAwaitEnabled);
// carry on with the rest of the application, regardless of success or failure
// at some point, topLevelAwaitEnabled is checked to conditionally execute some code
If top level await support is enabled, this succeeds fine. However, if it is not supported, this will result in the following error during parsing and cannot be caught at runtime with a try/catch:
$ node test.js...\test.js:3
await Promise.resolve(); // replaced with an actual promise
^^^^^
SyntaxError: await is only valid in async function
So the question is: How can I use a top level await if it is supported, without creating incompatibility issues with Node.js runtimes that do not support top level await (either no CLI flag was specified or simply no runtime support)?
If the answer is "it is not possible", I would like an explanation as to why this is impossible.
In the case I am actually committing an XY problem, the underlying issue is I need a top-level dynamic import.
Note: I am well aware that top level await is not recommended for a variety of reasons, however it is crucial for a specific functionality of my application and does not impose any issue with my use case. Alternatives will likely not suffice.
Attempts
I have tried the following methods, to no avail:
eval: I have tried replacing the await line with an eval("await Promise.resolve()"), in the hope the code was evaluated in the current context. Unfortunately, even if top level await is supported, this will result in the same error, as it does not seem to inherit the current context.
vm.compileFunction: Same issue was eval(), top level await is not supported.
vm.SourceTextModule: Evaluation is asynchronous and would need to be awaited at the top level to check if it is supported... which is a catch 22.
conditional execution of the await based on process.version and process.execArgv: The error during parsing - it never actually executes the code, so conditional execution is ruled out.
As far as I know this is not possible because the parser will simply error out. The compiler will not understand the await directive and will not complete its cycle. This is probably similar to using a word that's simply not a recognized keyword.
The closest you can get is using an anonymous function.
Seems like you might be able to check the version of node being used at runtime like so process.version and then you can use also use process.argv to check for any flags passed in when starting the process.
Using the above methods, you can first check the version of node being used, and then if relevant, you can check for the needed flags.
Node docs on checking for CLI args.
(async()=>{await promise})()
I am using a 3rd party package from npm, which in turn connects to some external API on IP address X.X.X.X and it crashes with the following error. The reason is clear, the network was down for a moment, and BOOM my entire program halts:
events.js:177
throw er; // Unhandled 'error' event
^
Error: connect ENETUNREACH X.X.X.X:80
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1054:14)
Emitted 'error' event at:
at Socket.socketErrorListener (_http_client.js:410:9)
at Socket.emit (events.js:200:13)
at emitErrorNT (internal/streams/destroy.js:91:8)
at emitErrorAndCloseNT (internal/streams/destroy.js:59:3)
at processTicksAndRejections (internal/process/task_queues.js:84:9) {
errno: 'ENETUNREACH',
code: 'ENETUNREACH',
syscall: 'connect',
address: 'X.X.X.X',
port: 80
}
I am deliberately not saying which module on npm it is that causes the problem because I can't control what 3rd party module authors are doing. If I complained to them they may or may not fix it but my coding life needs to go on. Even if any code I call has a coding error, it shouldn't completely crash my calling script. There should be a way for me to catch the condition before it cramps my style completely.
What I have tried: I use error handling meticulously, try/catch around any 3rd party async library calls etc. Example:
var ThirdPartyModule = require("thirdPartyModule");
try {
await ThirdPartyModule.doIt("some", "params");
} catch (err) {
console.log("Ok so the module call failed. Let's try something else here, but don't abort my script please!");
}
The catch does does nothing. Every time the module is called and the connection error happens, it completely tanks my entire program with the above "throw er;".
My question is, what additional "wrapper" code can I write around my calls to the library to "catch" any crashes in code that isn't mine, so my code keeps executing?
I am still kind of a noob in nodejs, so I think I am missing some greater nodejs concept that's at work here. I come from C++ or Java so whenever something crashes in these languages, I can always catch() it so I am puzzled how the module can "escape" try/catch in my main script.
Note that I am not looking for answers like "You should make sure to always be connected to the Internet, that's what ENETUNREACH is about." Instead, I am trying to understand how to be the master of the "sub modules" that I am calling, i.e. catch their errors, or if I can't be their master, I want to understand why what I want is impossible in nodejs.
Thanks!
EDIT: in a similar question linked-to below, a commenter suggests to add a process.on("uncaughtException") handler. I will try this out now.
If this is the correct solution, please post it as an answer and explain the underlying concept. Why can't I just rely on try / catch which is what a reformed Java programmer would do? What did the author of the module on npm do to not pass the Error up the chain in his async function?
EDIT 2: Perhaps my question ends up being a duplicate of Catch all uncaughtException for Node js app - I would love an explanation though why some crash deep down must be caught globally like that, and doesn't percolate up the caller chain. Is it because the 3rd party coder made an error, and failed to transform all possible problems that could occur in his code, into proper throws in his async code?
EDIT 3: Ok I managed to track down the problem. The library that I used was not the culprit. It innocently called and promisified a 4th-party library (also on npm) which connects to a server via http.request(). Just that the author of that 4th-party library forgot to install a request.on('error') callback, so when the Internet connection has problems (never happens, right!), that becomes an unhandled situation. The 3rd party library never gets the expected callback "on error" from the 4th party library, and thus never manages to promisify that error situation. Instead, the error lingers unhandled in nodejs ether, crashing my entire script. So there...
Making absolutely no assumptions about the structure of the code inside the library, you're stuck with the below:
process.on('uncaughtException', function (err) {
//do something
});
Reason being: If the library was using promises throughout, then the try/catch block would have caught it. It's very likely that inside the library the part throwing the error is inside a callback.
Best practice for async code should be to return an error to a callback, return a rejected Promise or throw inside a async block, and never throw inside functions using callbacks or raw Promises. However it seems extremely likely that is exactly what is happening. There is no way to handle the error gracefully at that point, only the catch-all uncaughtException.
Essentially in the case of a thrown error inside a Promise or a traditional async callback, the code is throwing a synchronous error and there is no synchronous try/catch block to handle it. And you can't inject one from outside the module.
So in short the error has no 'proper' mechanism to be handled. The third party module writers should be ashamed.
Try to chain it alike this:
await ThirdPartyModule.doIt("some", "params").catch(err => {
console.log(err);
});
(that's untested code, since the ThirdPartyModule is unknown).
I use Node with Monk, a simple library for MongoDB. It doesn't have insertOne and so on. And it's totally annoying to get a deprecation warning on each insert. Does MongoDB have an option to disable deprecation warnings?
(node:17737) DeprecationWarning: collection.insert is deprecated. Use insertOne, insertMany or bulkWrite instead.
You didn't add your node code (and lots of time has passed) but for future answer searchers - some mongoose methods can receive a SaveOptions object.
ie. when using collection.save you can pass this in like so:
doc.save({ suppressWarning: true })
If possible - it's better to "fix" the warnings by moving on from the deprecated apis to the recommended ones.
Basically I am looking for a substitute to var x = prompt("Question") inside of an asynchronous function specifically for node.js. Does such a thing exist? I have tried dependencies such as inquirer but I am getting unhandled promise rejection warnings.