I am new to socket.io. So my socket.io server sometimws crashes giving this below error
timers.js:103
if (!process.listeners('uncaughtException').length) throw e;
^
Error: socket hang up
at createHangUpError (http.js:1360:15)
at ServerResponse.OutgoingMessage._writeRaw (http.js:507:26)
at ServerResponse.OutgoingMessage._send (http.js:476:15)
at ServerResponse.OutgoingMessage.write (http.js:749:16)
at XHRPolling.doWrite (E:\sitesroot\0\node_modules\socket.io\lib\transports\
xhr-polling.js:67:17)
at XHRPolling.HTTPPolling.write (E:\sitesroot\0\node_modules\socket.io\lib\t
ransports\http-polling.js:132:8)
at XHRPolling.Transport.packet (E:\sitesroot\0\node_modules\socket.io\lib\tr
ansport.js:515:15)
at Object.<anonymous> (E:\sitesroot\0\node_modules\socket.io\lib\transports\
http-polling.js:79:12)
at Timer.list.ontimeout (timers.js:101:19)
It doesnt show where or why the error is happening so pretty sure its nothing to do with the code i have written. Could be something with the transports? I dont have much knowledge on it. Any suggestions on how to stop it from crashing would be highly appreciated. Thanks
The problem is as #miktam stated.
To fix this you need to add an error listener to your code.
Add this code to your application:
//Error handler
process.on('uncaughtException', function (exception) {
// handle or ignore error
console.log(exception);
});
When ever there is an error it will console.log it instead of crashing it. I had the exact same problem and this fixed it.
Check this issue
Quoting Isaac Schlueter:
The good news is that you're no longer leaking memory. The bad news is
that you do indeed need to add error listeners to your objects.
In most apps, you can usually treat ECONNRESET as roughly the same as
a graceful close. Just make sure that you actually stop using that
socket, since it's now closed. However, it's not a graceful close,
since it's not in any sense "graceful". Prior to v0.8.20, node would
happily buffer all writes to reset sockets, causing memory explosion
death.
For us new to websocket, you might be missing some parameters that are required by socket.io library.
What you might be using is ws://localhost which works for Websocket library, but might need to use ws://localhost:3000/socket.io/?EIO=4&transport=websocket when using socket.io
Related
I am using a 3rd party package from npm, which in turn connects to some external API on IP address X.X.X.X and it crashes with the following error. The reason is clear, the network was down for a moment, and BOOM my entire program halts:
events.js:177
throw er; // Unhandled 'error' event
^
Error: connect ENETUNREACH X.X.X.X:80
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1054:14)
Emitted 'error' event at:
at Socket.socketErrorListener (_http_client.js:410:9)
at Socket.emit (events.js:200:13)
at emitErrorNT (internal/streams/destroy.js:91:8)
at emitErrorAndCloseNT (internal/streams/destroy.js:59:3)
at processTicksAndRejections (internal/process/task_queues.js:84:9) {
errno: 'ENETUNREACH',
code: 'ENETUNREACH',
syscall: 'connect',
address: 'X.X.X.X',
port: 80
}
I am deliberately not saying which module on npm it is that causes the problem because I can't control what 3rd party module authors are doing. If I complained to them they may or may not fix it but my coding life needs to go on. Even if any code I call has a coding error, it shouldn't completely crash my calling script. There should be a way for me to catch the condition before it cramps my style completely.
What I have tried: I use error handling meticulously, try/catch around any 3rd party async library calls etc. Example:
var ThirdPartyModule = require("thirdPartyModule");
try {
await ThirdPartyModule.doIt("some", "params");
} catch (err) {
console.log("Ok so the module call failed. Let's try something else here, but don't abort my script please!");
}
The catch does does nothing. Every time the module is called and the connection error happens, it completely tanks my entire program with the above "throw er;".
My question is, what additional "wrapper" code can I write around my calls to the library to "catch" any crashes in code that isn't mine, so my code keeps executing?
I am still kind of a noob in nodejs, so I think I am missing some greater nodejs concept that's at work here. I come from C++ or Java so whenever something crashes in these languages, I can always catch() it so I am puzzled how the module can "escape" try/catch in my main script.
Note that I am not looking for answers like "You should make sure to always be connected to the Internet, that's what ENETUNREACH is about." Instead, I am trying to understand how to be the master of the "sub modules" that I am calling, i.e. catch their errors, or if I can't be their master, I want to understand why what I want is impossible in nodejs.
Thanks!
EDIT: in a similar question linked-to below, a commenter suggests to add a process.on("uncaughtException") handler. I will try this out now.
If this is the correct solution, please post it as an answer and explain the underlying concept. Why can't I just rely on try / catch which is what a reformed Java programmer would do? What did the author of the module on npm do to not pass the Error up the chain in his async function?
EDIT 2: Perhaps my question ends up being a duplicate of Catch all uncaughtException for Node js app - I would love an explanation though why some crash deep down must be caught globally like that, and doesn't percolate up the caller chain. Is it because the 3rd party coder made an error, and failed to transform all possible problems that could occur in his code, into proper throws in his async code?
EDIT 3: Ok I managed to track down the problem. The library that I used was not the culprit. It innocently called and promisified a 4th-party library (also on npm) which connects to a server via http.request(). Just that the author of that 4th-party library forgot to install a request.on('error') callback, so when the Internet connection has problems (never happens, right!), that becomes an unhandled situation. The 3rd party library never gets the expected callback "on error" from the 4th party library, and thus never manages to promisify that error situation. Instead, the error lingers unhandled in nodejs ether, crashing my entire script. So there...
Making absolutely no assumptions about the structure of the code inside the library, you're stuck with the below:
process.on('uncaughtException', function (err) {
//do something
});
Reason being: If the library was using promises throughout, then the try/catch block would have caught it. It's very likely that inside the library the part throwing the error is inside a callback.
Best practice for async code should be to return an error to a callback, return a rejected Promise or throw inside a async block, and never throw inside functions using callbacks or raw Promises. However it seems extremely likely that is exactly what is happening. There is no way to handle the error gracefully at that point, only the catch-all uncaughtException.
Essentially in the case of a thrown error inside a Promise or a traditional async callback, the code is throwing a synchronous error and there is no synchronous try/catch block to handle it. And you can't inject one from outside the module.
So in short the error has no 'proper' mechanism to be handled. The third party module writers should be ashamed.
Try to chain it alike this:
await ThirdPartyModule.doIt("some", "params").catch(err => {
console.log(err);
});
(that's untested code, since the ThirdPartyModule is unknown).
And actually, I don't fully understand why my code is not in the stack trace, if node is single threaded. Maybe I'm fundamentally misunderstanding, something, but why does my application sometimes die with a stack trace that doesn't have anything I've written in it?
I'm writing a pretty simple proxy server using node/express. As an example, I was periodically getting this "socket hangup error":
Error: socket hang up
at createHangUpError (_http_client.js:250:15)
at Socket.socketOnEnd (_http_client.js:342:23)
at emitNone (events.js:91:20)
at Socket.emit (events.js:185:7)
at endReadableNT (_stream_readable.js:926:12)
at _combinedTickCallback (internal/process/next_tick.js:74:11)
at process._tickCallback (internal/process/next_tick.js:98:9) code: 'ECONNRESET' }
And since none of the javascript files in the stack trace are mine, I had no idea where this was coming from. It was basically trial and error, trying to catch errors and adding .on style error-handlers until I found the right place.
I feel like I'm fundamentally missing something - what I should I be doing differently in order to debug errors like this? How do I know where to handle it if I can't see what (in my code) is causing it? How do I know whether I should be using a try/catch block, or something like request.on('error') {...}?
Some errors, like the one mentioned by you, are not caused by your code. In fact it is caused by the absence of code in your application. (For example, your application code is probably missing the code for gracefully handling ECONNRESET i.e. remote socket disconnection.
Now, to your question about how to debug such errors (including third-party code). Of course, you can use stack-trace and longjohn etc.
But, for me, the easier & quicker solution is to run the application in debug mode with --inspect option, with Chrome debugger to inspect it (no breakpoints), with Pause on Exceptions option enabled. That's all you need to do. Now whenever there is an exception, chrome debugger will pause the application exactly at the line where the exception is thrown. Makes it a lot easier to find such bugs.
Hope this helps you!
You could do something like for debugging such errors.
process.on('uncaughtException', function(err) {
console.log(err.stack);
throw err;
});
You could also increase your stack trace size limit and/or stack size.
node --stack_trace_limit=200 app.js //defaults to 10
node --stack-size=1024 app.js // defaults to 492kB
Contrary to your assumption, the single thread paradigm of node.js cause these types of error. In a multi-thread environment like java, all the called functions are executed inside the caller function:
java -> A -> B -> C -> D
So if the A() is wrapped in a try and catch all the internal exceptions are caught.
But in Async environment, the callback function is executed outside the caller function:
node -> A -> B(C)
node -> I -> C -> D
Here the function A call async function B (Usually a library) with callback function C as an argument, function B start an async task after it's finished node.js call function I which is an internal function of that library and it calls function C. Here you see I, C and D are called outside of your code.
So there are two things to consider here:
1- you must wrap your callback function code in try and catch.
2- there may be an exception in function I which you can not catch in your code. These are those exceptions that you are referring which none of the javascript files in the stack trace are yours because it's not initiated in your code.
Now if the function B (its library) is good written it must provide some means to catch those exceptions. One of them would be on('error', cb). So you should always check the library documentation to see how you can catch and handle such exceptions.
If the library is poorly written and nothing is provided to catch its exceptions in your code, you can use a debugger like chrome inspector or WebStorm debugger to catch them but there is no other way than manipulating its source code so at least they are bypassed to your code, or you can file a bug report.
Node.js also provide an uncaught exception handler which catches all uncaught exceptions:
process.on('uncaughtException', (err) => {
process.exit(1);
});
although it's risky to continue execution after here because things could be in an uncertain state, but it's a good place to log errors.
Stack Trace does not contain your file because this type of error is created server connection issue like:
Host Timeout
[ 'code' ]
e.code => ECONNRESET
Console output
{ Error: socket hang up
at createHangUpError (_http_client.js:250:15)
at TLSSocket.socketCloseListener (_http_client.js:282:23)
at emitOne (events.js:101:20)
at TLSSocket.emit (events.js:188:7)
at TCP._handle.close [as _onclose] (net.js:492:12) code: 'ECONNRESET' }
OR when the server terminates the connection unexpectedly or does not send a response.
For more info check node app error codes.
or refer to this issue.
You might try the built in debugger for node.js. This debugger allows you to step through code that is not yours.
node debug script.js
Another good tool is node-inspector.
This article has some other debug options that could be useful in determining the cause of your exception: How do I debug Node.js applications?
I am trying to understand how to build my error handling system for my api.
Let's say I have a the following line in a controller method :
var age = json.info.age;
with
json = {"id":1, "name":"John", info": {"age":27, "sex":"m"}}
Let's say that the object doesn't contain an info field, I'll get the following error TypeError: Cannot read property 'info' of undefined and my server will crash.
Is there a way to make a higher level abstraction and catch all the potential errors that I could have? Or should I have a try/catch system for each of the methods of my controllers?
BEWARE OF THE CODE BELOW, IT WILL BITE YOU WHENEVER IT CAN!
Don't use the code snippet below if you do not understand its
implications, please read the whole answer.
You can use the node way for uncaught errors. Add this in your config/bootstrap.js
Updated the snippet below to add what was said in the comments, also added a warning about using a global to respond to the user.
process.on('uncaughtException', function (err) {
// Handle your errors here
// global.__current__ is added via middleware
// Be aware that this is a bad practice,
// global.__current__ being a global, can change
// without advice, so you might end responding with
// serverError() to a different request than the one
// that originated the error if this one happened async
global.__current__.res.serverError();
})
Now, can doesn't mean should. It really depends on your needs, but do not try to catch BUGS in your code, try to catch at a controller level the issues that might not happen every time but are somehow expected, like a third-party service that responded with empty data, you should handle that in your controller. The uncaughtException is mainly for logging purposes, its better to let your app crash if there is a bug. Or you can do something more complicated (that might be better IMHO), which is to stop receiving requests, respond to the error 500 (or a custom one) to user that requested the faulty endpoint, and try to complete the other requests that do not relate to that controller, then log and shutdown the server. You will need several instances of sails running to avoid zero downtime, but that is material for another question. What you asked is how to get uncaught exceptions at a higher lvl than the controllers.
I suggest you read the node guide for error handling
Also read about domains, even thought they are deprecated you can use them, but you would have to deal with them per controller action, since sails does not provide any help with that.
I hope it helps.
You can check this way if you want to:
if (object != null && object.response != null && object.response.docs != null){
//Do your stuff here with your document
}
I don't really get what is your "object" variable in the first place, so i don't know if you can check it at a different level, is it a sails parameter to your controller ?
So that's how I did it, thanks to Zagen's answer.
module.exports.bootstrap = function(cb) {
process.on('uncaughtException', function (err) {
//Handle your errors here
logger.fatal(err);
global.__current__.res.serverError();
})
cb();
};
I send a generic error 500 to the user if any uncaught exception is thrown, and I log the error to the fatal level. On that way, my server is still accessible 24/7 and I can monitor the logs at another level and trigger an alarm on a fatal error. I can then fix the exception that was thrown.
I'm experimenting with Dart and using the new streamSpawnFunction to create a new isolate.
I'm running my code in Dartium but i've noticed that if some kind of unrecoverable error occurs in the isolate i get no error message on the console. Because breakpoints in Isolate code are not working debugging is really painful.
The old Port based Isolate spawn function (spawnFunction) has a callback function for handling errors. I wonder why this is not available with streamSpawnFunction. Is there a new way to subscribe to the error events of an Isolate?
The missing functionality of streamSpawnFunction is just an oversight. I filed http://dartbug.com/9208 and I will try to fix it next week.
I'm not sure if it is a known problem that breakpoints don't work in isolates. I will let you file a bug-report (http://dartbug.com) so the devs can ask you questions and you are kept informed on the process.
I'm trying to find the most elegant way for my node.js app to die when something happens. In my particular case, I have a config file with certain require parameters that have to be met before the server can start and be properly configured.
One way I have found to do this is:
var die = function(msg){
console.log(msg)
process.exit(1);
}
die('Test end');
Is there a better way to handle this kind of situation?
better use console.error if you are doing process.exit immediately after.
console.log is non-blocking and puts your message into write queue where it is not processed because of exit()
update: console.log also blocks in latest versions (at least since 0.8.x).
If you want to abruptly exit then this will do just fine. If you want do any clean up you should do that first after which node.js will probably stop anyway, because nothing keeps event loop running.