I'm trying to find the best place to handle connectivity errors, or any other global errors that crash the server.
What is the right place to catch adapter/global errors and not have them crash the server?
Specifically, I want to handle these types of errors in a graceful way:
Error spawning mySQL connection:
error: Hook failed to load: orm (Error: connect ECONNREFUSED)
error: Error encountered while loading Sails core!
error: Error: connect ECONNREFUSED
from the sails docs: http://sailsjs.org/#!documentation/config.500
thats the error handling sails exposes from within the config
if your error passes that, you can hook in there, otherwise you can hook in node's process
process.on('uncaughtException', function (err) {
if (err.toString() === 'Error spawning mySQL connection') {
//rende some error page
}
})
if the exception thrown is async the only way to catch it is trough process
do note however, that these kinds of errors are almost always unrecoverable, so crashing (and restarting) is the best approach
most modules loaded use local variables and expose only a subset of their internals trough module.exports, unloading a module and restarting its local code can be done, but you would need to unload all dependant modules and all modules holding references to it also. Thats why the normal approach is to let it crash
Related
I'm using a few external API's (some in timers, every minute or so) and sometimes I get some connection errors because of network problems or because the external systems are down for some reason. When I get this errors, the app restarts, and if the error persists the app continues restarting.
Any ideas on how can I ignore the connection error and keep the app running?
Code Example:
try {
var req = https.request(options, callback);
req.write(JSON.stringify(params));
req.end();
} catch (e) {
throw e;
}
Based on your code example. You're doing throw e inside your try catch. Essentially, you're catching an error and then throwing the error. Just do console.error(err) or however you want to handle that error, without throwing. This is what will cause your instance to stop.
The only way I have found to "catch" EPIPE errors thrown asynchronously by a socket timing out or closing prematurely is to directly attach an event handler to the socket object itself, as demonstrated in the documentation here:
https://nodejs.org/api/errors.html
const net = require('net');
const connection = net.connect('localhost');
// Adding an 'error' event handler to a stream:
connection.on('error', (err) => {
// If the connection is reset by the server, or if it can't
// connect at all, or on any sort of error encountered by
// the connection, the error will be sent here.
console.error(err);
});
This works, but is in many cases unhelpful -- if you're accessing a database or another service that has a node driver, the request and socket objects are likely inaccessible from your app code.
The most obvious solution is "don't do things that generate these errors" but since any non-trivial application is dependent on other services, no amount of input-checking in advance can guarantee that the service on the other end won't hang up unexpectedly, throwing an EPIPE in your code and in all likelihood crashing Node.
So, the options for handling this situation seem to be:
Let the error crash your app and use nodemon or supervisor to automatically restart. This isn't clean, but it seems like the only way to really guarantee you'll get back up and running safely.
Write custom connection clients for dependent services. This let's you attach error handlers where known problems could occur. But it violates DRY and means that you're now on the hook for maintaining your own custom client code when otherwise reasonable open source solutions already exist. Basically, it adds a huge maintenance burden for a slightly cleaner solution to a fairly rare problem.
Am I missing something, or are those the best options available?
Using node.js, when I run the program
setTimeout(() => console.log("Timed out"), 0);
console.log("finishing");
I see
finishing
Timed out
But when I add a throw before "finishing"
setTimeout(() => console.log("Timed out"), 0);
throw new Error();
console.log("finishing");
I see
throw new Error();
^
Error
at Object.<anonymous> ...(stack trace here)...
And I don't see any mention of "Timed out".
Why is that? Even though the initial context would throw, once the stack was freed up, I expected the callback I passed to setTimeout would still run.
Does having an uncaught exception cause all timeouts to get canceled? Is this feature documented somewhere?
If I have multiple timeouts, is there a way for me to make sure that all the other timeouts continue to run when they can even if one of them happens to throw?
Unlike a web application running on browser, a Node application runs as a process on top of Google V8 JavaScript Engine. If you look into https://nodejs.org/api/timers.html is states that
The timer functions within Node.js implement a similar API as the timers API provided by Web Browsers but use a different internal implementation that is built around the Node.js Event Loop.
As the above statement explains, even though the same global functions are available in both cases, their implementations are different. Therefore when an uncaught exception occurs in a Node application, all code related to timeouts will stop as the process is terminated. The best way to handle this is to properly handle all exceptions. You can use the below code to capture all uncaught exceptions from the process level itself.
process.on('uncaughtException', function(error) {
console.log(error);
});
I've a node.js express app running in IIS. I found that the app is crashing frequently due to some uncaught exception. Hence I used process.on('uncaughtException') to restart the service in case of uncaught exception. I'm able to get the error as "ECONNRESET" but I'm unable to get where this actually happened. Is there any way to capture the error source or line number which caused the exception?
Use the process.on('uncaughtException'… event which provides you with the Error object that contains the call stack
process.on('uncaughtException', function(err){
console.error(err.stack);
process.exit();
});
In my node.js app which uses the cluster module, I'm intermittently seeing errors like this:
events.js:71
throw er; // Unhandled 'error' event
^
Error: read ECONNRESET
at errnoException (net.js:863:11)
at TCP.onread (net.js:524:19)
This brings down my whole app, and so far the only way I've been able to deal with these is by binding a process.on('uncaughtException'). I'd like to figure out the underlying cause, but the above stack trace is pretty useless.
Is there some way to figure out what's causing these exceptions?
I should note that I'm seeing these only in the cluster master, not the workers, which leads me to suspect that they have something to do with the way the cluster modules does its magic in distributing connections to workers.
This answer was helpful: https://stackoverflow.com/a/11542134/233370
Basically, I installed longjohn and was then able to get the full async stack trace to figure out the underlying cause (rabbit.js in my case).
It seems that express enabled keep-alive by default.
In order to close connection after response you can add
res.set("Connection", "close");
Alternatively you can add a middleware in your app to close connection after each response:
app.use(function(req, res, next) {
res.set("Connection", "close");
next();
});