I am using node-bunyan and bunyan-logstash-tcp in my nodejs application to send the logs to logstash (1.4.2) and elasticsearch (1.4.2).
Whenever the logstash server disconnects or is not reachable, my nodejs application crashes giving the following error
ERROR
-------
events.js:72
throw er; // Unhandled 'error' event
^
Error: connect ECONNREFUSED
at errnoException (net.js:904:11)
at Object.afterConnect [as oncomplete] (net.js:895:19)
bunyan-logstash-tcp should actually handle this error.
Can anyone please help me to solve this nodejs crashing issue.
I was able to figure out the issue.
Error event needs to be handled while creating tcp bunyan stream
stream: bunyantcp.createStream({
host: '127.0.0.1',
port: 9998
}).on('error', console.log)
This is not mentioned in the bunyan-logstash-tcp documentation, but was there in an example code.
UPDATE: Sample configuration
this.log = bunyan.createLogger({
name: name,
streams: [
{
level: 'error',
type: 'raw',
serializers: bunyan.stdSerializers,
stream: bunyantcp.createStream({
application: process.title,
max_connect_retries: 10, // Don't give up on reconnecting
retry_interval: 1000 * 60 // Wait 1s between reconnect attempts
}).on('error', console.log)
}
],
level: 'debug'
});
For logstash you could probably switch to UDP from TCP.
There are few important advantages:
no connection issues (no reconnect problems, etc)
less performance overhead
less network overhead
Logentries' got a nice blogpost about it: https://blog.logentries.com/2014/07/tcp-or-udp-for-logging/
Take a look at some out-of-the box libs for logstash (elk stack) that support UDP: https://github.com/devmetrics/devmetrics-nodejs-core
Related
I'm encountering ECONNRESET errors that are crashing my node server when I run a Nessus Essentials basic network scan:
node:events:505
throw er; // Unhandled 'error' event
^
Error: read ECONNRESET
at TCP.onStreamRead (node:internal/stream_base_commons:217:20)
Emitted 'error' event on Socket instance at:
at emitErrorNT (node:internal/streams/destroy:157:8)
at emitErrorCloseNT (node:internal/streams/destroy:122:3)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
errno: -54,
code: 'ECONNRESET',
syscall: 'read'
}
I don't know exactly what Nessus is doing and I haven't managed to reproduce the error any other way, but I've identified two interesting facts:
The problem only occurs when running on Node versions < 16.16.0
It also goes away if I remove socket.io, or if I instantiate socket.io using a different port or HTTP server than my express server
For context, I've simplified my application down to a minimal project still exhibits the error:
import express from 'express';
import { Server } from 'socket.io';
import morgan from 'morgan';
import winston from 'winston';
const logger = winston.createLogger({
level: 'debug',
format: winston.format.combine(
winston.format.timestamp({ format: 'YYYY-MM-DD HH:mm:ss:ms' }),
winston.format.splat(),
winston.format.prettyPrint(),
winston.format(info => {
info.level = `[${info.level.toUpperCase()}]`;
return info;
})(),
winston.format.printf(info => {
if (typeof info.message === 'object') {
info.message = JSON.stringify(info.message, null, 3);
}
return `${info.timestamp} ${info.level} ${info.message}`;
}),
),
transports: [new winston.transports.Console()]
});
const morganMiddleware = morgan(
':remote-addr :method :url :status :response-time ms',
{
stream: {
write: message => logger.http(message.trim()),
},
});
const app = express();
app.use(morganMiddleware);
const port = 3001;
const httpServer = app.listen(port, () => {
logger.info(
`Server is now listening on port ${port} ✓`,
);
new Server(httpServer);
});
export default app;
If I remove the new Server(httpServer); line, the app doesn't crash. So the error seems to be linked to socket.io and Express sharing a connection, and something going wrong during the vulnerability scan (something causing the socket to be dropped?), but I haven't managed to debug it further.
I have tried all sorts of ways of catching the error, but the only way that works is using process.on('uncaughtException'). That's not helpful however, because at that point, there is no safe way to recover from the error. The error bypasses all the error handling of both Express and socket.io.
I could of course "solve" the problem by upgrading to a more recent version of node, but I need to understand the problem in order to be sure that it's actually fixed and won't surface again in some other form. Also, I would like to be able to make my app more resilient by catching this sort of error if they were to occur in the future.
Or perhaps there's some way to separate socket.io from express that without using separate ports (which wouldn't be practical in my use-case). Can I use proxy websocket related requests through Express to socket.io without sharing an HTTP server?
Any suggestions, either to help understand/debug the problem, or to work around it, would be welcome.
After much investigation, I figured out what was going on.
The ECONNRESET error I was getting occurred when Socket.io received an HTTP2 upgrade request: upgrade events are intercepted by Socket.io so that websocket upgrades can be handled, but for HTTP2 upgrades, the socket was incorrectly being disconnected without any error handler to catch the error, which in turn led to the the application crashing.
I was able to fix the bug, which was released as engine.io 6.2.1.
The reason I was not seeing the issue with versions of Node older than 16.16.0 was actually unrelated: Nessus was, for some reason, sending malformed HTTP requests for the HTTP2 upgrade, using LF as line separators instead of CRLF. Starting with v16.16.0, Node now rejects such requests with a 400 Bad Request, which avoids the Socket.io issue.
How and where do I handle this error and stop my app from crashing ?
$ node app.js
Server Started at port 3000
events.js:200
throw er; // Unhandled 'error' event
^
Error: 14 UNAVAILABLE: Name resolution failed for target firestore.googleapis.com:443
at Object.callErrorFromStatus (D:\food_list\node_modules\#grpc\grpc-js\build\src\call.js:30:26)
at Object.onReceiveStatus (D:\food_list\node_modules\#grpc\grpc-js\build\src\client.js:327:49)
at Object.onReceiveStatus (D:\food_list\node_modules\#grpc\grpc-js\build\src\client-interceptors.js:303:181)
at Http2CallStream.outputStatus (D:\food_list\node_modules\#grpc\grpc-js\build\src\call-stream.js:114:27)
at Http2CallStream.maybeOutputStatus (D:\food_list\node_modules\#grpc\grpc-js\build\src\call-stream.js:153:22)
at Http2CallStream.endCall (D:\food_list\node_modules\#grpc\grpc-js\build\src\call-stream.js:140:18)
at Http2CallStream.cancelWithStatus (D:\food_list\node_modules\#grpc\grpc-js\build\src\call-stream.js:441:14)
at ChannelImplementation.tryPick (D:\food_list\node_modules\#grpc\grpc-js\build\src\channel.js:214:32)
at ChannelImplementation._startCallStream (D:\food_list\node_modules\#grpc\grpc-js\build\src\channel.js:244:14)
at Http2CallStream.start (D:\food_list\node_modules\#grpc\grpc-js\build\src\call-stream.js:418:22)
Emitted 'error' event on ClientReadableStreamImpl instance at:
at Object.onReceiveStatus (D:\food_list\node_modules\#grpc\grpc-js\build\src\client.js:327:28)
at Object.onReceiveStatus (D:\food_list\node_modules\#grpc\grpc-js\build\src\client-interceptors.js:303:181)
[... lines matching original stack trace ...]
at Http2CallStream.start (D:\food_list\node_modules\#grpc\grpc-js\build\src\call-stream.js:418:22)
at BaseStreamingInterceptingCall.start (D:\food_list\node_modules\#grpc\grpc-js\build\src\client-interceptors.js:275:19) { code: 14,
details: 'Name resolution failed for target firestore.googleapis.com:443',
metadata: Metadata { internalRepr: Map {}, options: {} }
}
ps: The app crashes when I run the app without internet.
This is a DNS error. DNS is a service that's used in internet connections. Your app is trying to access the internet, Firestore specifically, but cannot. That's why there's an error. There's no simple fix to this, as you would have to also program behavior for the app to function while offline.
The error stack trace says this:
Emitted 'error' event on ClientReadableStreamImpl instance at:
This means that the error is being thrown because the stream is emitting an error event and you are not handling that event. You can fix that by calling stream.on('error', callback) on the stream that you are reading from.
My server stops with this error how can i solve it? Or what it is?
events.js:182
throw er; // Unhandled 'error' event
^
Error: read ECONNRESET
at exports._errnoException (util.js:1024:11)
at TCP.onread (net.js:610:25)
Thanks for answers
"ECONNRESET" means the other side of the TCP conversation abruptly closed its end of the connection. This is most probably due to one or more application protocol errors. You could look at the API server logs to see if it complains about something.
To know more about ECONNRESET, see this answer.
The message Unhandled 'error' event is suggesting that you are not listening for the error event in your code.
You can catch errors like these by catching the error event like the following
connection.on("error", function(err){ // handle "error" event so nodejs will not crash
console.log(err);
});
hello every one i am new in nodejs and want to use the light streamer for my site there is a example on git when i tried to deployed it on my local instance throwing the below error any idea will be appreciated thanks in advance...
here is the example i want to deploy
Light streamer nodjs example
E:\wamp\www\nodeJs\lightstream>node helloworld.js
events.js:85
throw er; // Unhandled 'error' event
^
Error: connect ECONNREFUSED
at exports._errnoException (util.js:746:11)
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1000:19)
ECONNREFUSED suggests that the application is not able to connect to the desired TCP port (most likely it is being blocked by a firewall or there is no application listening on that port).
Assuming you have not changed the configuration in helloworld.js:
Have you installed the lightstream server locally and checked it is running? If so, check Windows Firewall and add rules to allow inbound ports 6663 and 6664.
I'm trying to implement a BOOTP server in Node, for which broadcasting is a necessity. Sadly the docs are a little bit confusing and I'm getting weird errors all the way. Funny enough, the errors are different on Windows 7 and Ubuntu.
Did someone actually manage to send a UDP broadcast to 255.255.255.255 or receive one under this address?
Could someone provide me a simple Node UDP broadcasting demo?
Using punt I tried to bind a connection to 255.255.255.255 on port 5000 and I get this error EADDRNOTAVAIL
I think the address it too general. See this link
Here is the code, which is just a slightly modified version of a punt example.
var punt = require('punt');
var server = punt.bind('255.255.255.255:5000');
var a = punt.connect('255.255.255.255:5000');
server.on('message', function(msg){
console.log(msg);
});
setInterval(function(){
a.send({ hello: 'world' });
}, 150);
which yields this error:
events.js:72
throw er; // Unhandled 'error' event
^
Error: bind EADDRNOTAVAIL
at errnoException (dgram.js:439:11)
at dgram.js:206:28
at dns.js:72:18
at process._tickCallback (node.js:415:13)
at Function.Module.runMain (module.js:499:11)
at startup (node.js:119:16)
at node.js:901:3