I am working with multiple AWS intances connected to the same mongo database (inside Compose.io Elastic deployment) but I keep getting the error server <url>:<port> sockets closed after a few minutes. Can anyone give me any hint about what may be wrong with the connection code?
CONNECTION CODE
var url = "mongodb://<user>:<password>#<url1>:<port1>,<url2>:<port2>/<dbName>?replicaSet=<replicaSetName>";
var options = {
server : {"socketOptions.keepAlive": 1},
replSet : { "replicaSet": <replicaSetName>, "socketOptions.keepAlive": 1 }
};
MongoClient.connect(url, options, function(err, db) { ... });
ERROR MESSAGE
Potentially unhandled rejection [2] MongoError: server <url>:<port> sockets closed
at null. (/var/app/current/node_modules/mongodb/node_modules/mongodb-core/lib/topologies/server.js:328:47)
at g (events.js:199:16)
at emit (events.js:110:17)
at null. (/var/app/current/node_modules/mongodb/node_modules/mongodb-core/lib/connection/pool.js:101:12)
at g (events.js:199:16)
at emit (events.js:110:17)
at Socket. (/var/app/current/node_modules/mongodb/node_modules/mongodb-core/lib/connection/connection.js:142:12)
at Socket.g (events.js:199:16)
at Socket.emit (events.js:107:17)
at TCP.close (net.js:485:12)
Related
I'm trying to onboard a new developer, that is using Windows 11 as the only one on our small team. I've guided him through installing WSL2 and Ubuntu 20.04.3 LTS (linux kernel: 5.10.93.2-microsoft-standard-WSL2).
We are 3 other developers who are using native Ubuntu, WSL2 Ubuntu 21.04 and macOS, respectively.
We are all on nodejs 16.14 with the exact same package-lock.json file.
He is the only one getting an Error [ERR_STREAM_WRITE_AFTER_END]: write after end.
tldr;
Both errors are about writing to a Buffer.
Is anyone aware of any related issues to nodejs's Buffer implementation on Windows 11?
We are using thinky as outlined below:
'use strict';
const createThinky = require('thinky');
const { rethinkdbConfig } = require ('./utils/config.js');
const thinky = createThinky (rethinkdbConfig);
// Thinky is our ORM
var type = thinky.type;
// Creates the thinky DB model for payments - we save the amount and the stripe customerID for each transaction
// You can execute any rethinkdb query language on the model, eg Payment.count().execute()
var Payment = thinky.createModel("payments", {
id: type.string(),
amount: type.number(),
customerID: type.string(),
project: type.string(),
projectName: type.string(),
projectPercentage: type.number(),
firefundPercentage: type.number(),
type: type.string(),
processor: type.string(),
email: type.string(),
charged: type.boolean(),
recharged: type.boolean()
});
module.exports = {
model: Payment,
};
Now, it doesn't matter if he connects to a local instance of rethinkdb (2.4.1~0focal), our staging or production rethinkdb (2.3.5~0trusty) on AWS.
I got him to try Netcat with nc -zv [url] 28015 to see if he could connect at all and he got connectıon successful. So I do not think it is a firewall issue.
Error stack trace
node ./bin/www
firefund:www Listening on port 3000 +0ms
node:events:498
throw er; // Unhandled 'error' event
^
Error [ERR_STREAM_WRITE_AFTER_END]: write after end
at new NodeError (node:internal/errors:371:5)
at _write (node:internal/streams/writable:319:11)
at Socket.Writable.write (node:internal/streams/writable:334:10)
at Connection._sendProof (/home/edel_weiss/firefund-production/node_modules/rethinkdbdash/lib/connection.js:294:19)
at /home/edel_weiss/firefund-production/node_modules/rethinkdbdash/lib/connection.js:248:12
at Object.tryCatch (/home/edel_weiss/firefund-production/node_modules/rethinkdbdash/lib/helper.js:170:3)
at Connection._computeSaltedPassword (/home/edel_weiss/firefund-production/node_modules/rethinkdbdash/lib/connection.js:247:12)
at Socket.<anonymous> (/home/edel_weiss/firefund-production/node_modules/rethinkdbdash/lib/connection.js:184:18)
at Socket.emit (node:events:520:28)
at Socket.emit (node:domain:475:12)
at addChunk (node:internal/streams/readable:315:12)
at readableAddChunk (node:internal/streams/readable:289:9)
at Socket.Readable.push (node:internal/streams/readable:228:10)
at TCP.onStreamRead (node:internal/stream_base_commons:190:23)
Emitted 'error' event on Connection instance at:
at Socket.<anonymous> (/home/edel_weiss/firefund-production/node_modules/rethinkdbdash/lib/connection.js:129:12)
at Socket.emit (node:events:520:28)
at Socket.emit (node:domain:475:12)
at emitErrorNT (node:internal/streams/destroy:157:8)
at emitErrorCloseNT (node:internal/streams/destroy:122:3)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
code: 'ERR_STREAM_WRITE_AFTER_END'
}
The stack trace indicates that it is a connection error in node_modules/rethinkdbdash/lib/connection.js.
connection.js
connection.js: line 294 is this.connection.write(Buffer.concat([new Buffer(message.toString()), NULL_BUFFER])) at the bottom.
Connection.prototype._sendProof = function(authentication, randomNonce, saltedPassword) {
var clientFinalMessageWithoutProof = "c=biws,r=" + randomNonce;
var clientKey = crypto.createHmac("sha256", saltedPassword).update("Client Key").digest()
var storedKey = crypto.createHash("sha256").update(clientKey).digest()
var authMessage =
"n=" + this.user + ",r=" + this.randomString + "," +
authentication + "," +
clientFinalMessageWithoutProof
var clientSignature = crypto.createHmac("sha256", storedKey).update(authMessage).digest()
var clientProof = helper.xorBuffer(clientKey, clientSignature)
var serverKey = crypto.createHmac("sha256", saltedPassword).update("Server Key").digest()
this.serverSignature = crypto.createHmac("sha256", serverKey).update(authMessage).digest()
this.state = 2
var message = JSON.stringify({
authentication: clientFinalMessageWithoutProof + ",p=" + clientProof.toString("base64")
})
this.connection.write(Buffer.concat([new Buffer(message.toString()), NULL_BUFFER]))
}
Variant Error
He has also reported a variant of the same error as posted below:
node ./bin/www
firefund:www Listening on port 3000 +0ms
node:events:498
throw er; // Unhandled 'error' event
^
Error [ERR_STREAM_WRITE_AFTER_END]: write after end
at new NodeError (node:internal/errors:371:5)
at _write (node:internal/streams/writable:319:11)
at Socket.Writable.write (node:internal/streams/writable:334:10)
at /home/edel_weiss/firefund-production/node_modules/rethinkdbdash/lib/connection.js:143:23
at Object.tryCatch (/home/edel_weiss/firefund-production/node_modules/rethinkdbdash/lib/helper.js:170:3)
at Socket.<anonymous> (/home/edel_weiss/firefund-production/node_modules/rethinkdbdash/lib/connection.js:142:12)
at Socket.emit (node:events:523:35)
at Socket.emit (node:domain:475:12)
at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1143:10)
Emitted 'error' event on Connection instance at:
at Socket.<anonymous> (/home/edel_weiss/firefund-production/node_modules/rethinkdbdash/lib/connection.js:129:12)
at Socket.emit (node:events:520:28)
at Socket.emit (node:domain:475:12)
at emitErrorNT (node:internal/streams/destroy:157:8)
at emitErrorCloseNT (node:internal/streams/destroy:122:3)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
code: 'ERR_STREAM_WRITE_AFTER_END'
}
This variant suggest that the error is at line 143 in connection.js:
self.connection.write(Buffer.concat([versionBuffer, authBuffer, NULL_BUFFER]));
self.connection.on('connect', function() {
self.connection.removeAllListeners('error');
self.connection.on('error', function(error) {
self.emit('error', error);
});
var versionBuffer = new Buffer(4)
versionBuffer.writeUInt32LE(protodef.VersionDummy.Version.V1_0, 0)
self.randomString = new Buffer(crypto.randomBytes(18)).toString('base64')
var authBuffer = new Buffer(JSON.stringify({
protocol_version: PROTOCOL_VERSION,
authentication_method: AUTHENTIFICATION_METHOD,
authentication: "n,,n=" + self.user + ",r=" + self.randomString
}));
helper.tryCatch(function() {
self.connection.write(Buffer.concat([versionBuffer, authBuffer, NULL_BUFFER]));
}, function(err) {
// The TCP connection is open, but the ReQL connection wasn't established.
// We can just abort the whole thing
self.open = false;
reject(new Err.ReqlDriverError('Failed to perform handshake with '+self.host+':'+self.port).setOperational());
});
});
If you read this far - THANK YOU!
Both errors are about writing to a Buffer.
Is anyone aware of any related issues to nodejs's Buffer implementation on Windows 11?
Definitely not sure if this is THE answer, just spit-balling here that maybe you could make a custom Buffer.concat function to work in case it really is the nodejs Buffer.concat being the cause of the problem here ;-; hope it works I'm on windows 10 however
function BufferConcat(buffers){ //buffers is the array of buffers
var bytes=Buffer.byteLength, i=0
var length=buffers.reduce((b1,b2)=>bytes(b1)+bytes(b2))
const BufferToReturn=Buffer.alloc(length)
for(let buffer in buffers){
let values=Object.values(buffer)
for(let item of buffer){BufferToReturn[i++]=item}
}
return BufferToReturn
}
I'm getting the error described below when running my node.js app after perfoming a few api calls.
The error does not always show in the exactly same place/line of code. But most of the times it is at the end of the api call.
events.js:167
throw er; // Unhandled 'error' event
^
Error: connect ECONNREFUSED 127.0.0.1:443
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1113:14)
Emitted 'error' event at:
at TLSSocket.socketErrorListener (_http_client.js:391:9)
at TLSSocket.emit (events.js:182:13)
at emitErrorNT (internal/streams/destroy.js:82:8)
at emitErrorAndCloseNT (internal/streams/destroy.js:50:3)
at process._tickCallback (internal/process/next_tick.js:63:19)
Based on similar questions here at SO my hypothesis is that a) there is something using 127.0.0.1:443 and therefore conflicting with my app or b) node is trying to use 127.0.0.1:443 but there is nothing there for it to use (my app is listening to localhost :3000).
Hyphothesis a) doesn't seem likely since after running netstat -ano | findstr 127.0.0.1:443 nothing shows up (when app is running and right after it terminates).
Also killed every node.exe and mongod.exeb using any port in my computer, closed the terminal and restarted the node app without success.
In case error is related with hypothesis b) I'm not sure how to address it.
api.post('/parsePOpdf', wagner.invoke(function(Pdfeq, Pdfdocspec, Product, User, Order){
return async function(req,res){
//... some code
pdfParser.on("pdfParser_dataError", errData => console.error(errData.parserError) );
pdfParser.on("pdfParser_dataReady", async function(pdfData) {
fs.writeFile("./test.json", JSON.stringify(pdfData), function(err){
console.log(err);
});
let pages = pdfData.formImage.Pages;
//console.log('pages 557', pages);
let order = {
orderDetails : {
supplier : [{
item : []
}]
}
};
for (const page of pages){
let value = await getItemsInPDF(page, productKeys, pdfParsingDetails, order, Product, customer, supplierLink, User);
//... more code
order = value;
}
return res.json(order);
});
pdfParser.loadPDF(pdfFile);
}
}));
I would expect the code to finish without throwing this error.
It turns out that the problem was in the api code: an http.get line to fetch a remote file was generating the conflict. This makes sense since the error was not present for other endpoints of the api.
So learning is that if the terminal reports no app using the suspected conflicting port (see question) answser should be within the same code and you need to go line by line to identify which one is causing the problem (instead of focusing on other apps trying to use the same port, like I was focusing on).
Using Polling like below to check if the content of the file is changed then, other two functions are called
var poll_max_date=AsyncPolling(function (end,err) { if(err) {
console.error(err); } var stmp_node_id=fs.readFileSync(path.join(__dirname,'node_id'),"utf8");
console.log("--------loaded node : "+stmp_node_id);
if(druid_stmp_node_id!=stmp_node_id) {
// MAX DATA CUT-OFF DRUID QUERY
druid_exe.max_date_query_fire();
// // DRUID QUERY FOR GLOBAL DATA
druid_exe.global_druid_query_fire();
druid_stmp_node_id=stmp_node_id; }
end(); }, 1800000).run();//30 mins
Its working fine for sometime, but then getting below error like after 4 - 5hours :
events.js:167
throw er; // Unhandled 'error' event
^
Error: read ECONNRESET
at TCP.onStreamRead (internal/stream_base_commons.js:111:27) Emitted 'error' event at:
at emitErrorNT (internal/streams/destroy.js:82:8)
at emitErrorAndCloseNT (internal/streams/destroy.js:50:3)
tried using fs.watch to monitor the changes in the file instead of polling like below :
let md5Previous = null; let fsWait = false;
fs.watch(dataSourceLogFile, (event, filename) => { if (filename) {
if (fsWait) return;
fsWait = setTimeout(() => {
fsWait = false;
}, 1000);
const md5Current = md5(fs.readFileSync(dataSourceLogFile));
if (md5Current === md5Previous) {
return;
}
md5Previous = md5Current;
console.log(`${filename} file Changed`);
// MAX DATA CUT-OFF DRUID QUERY
druid_exe.max_date_query_fire();
// DRUID QUERY FOR GLOBAL DATA
druid_exe.global_druid_query_fire(); } });
Its is also working fine for sometime, but then getting same error like after 4 - 5hours :
events.js:167 throw er; // Unhandled 'error' event ^
Error: read ECONNRESET at TCP.onStreamRead
(internal/stream_base_commons.js:111:27) Emitted 'error' event at: at
emitErrorNT (internal/streams/destroy.js:82:8) at emitErrorAndCloseNT
(internal/streams/destroy.js:50:3)
But when run in Local Machine, its working fine. the error occurs only when run in remote Linux Machine.
somebody can help me how I can fix that problem?
Use fs.watchFile once , because fs.watch is not consistent across platforms,
https://nodejs.org/docs/latest/api/fs.html#fs_fs_watchfile_filename_options_listener
Change your code according to the requirement.
It has been happening since the users are closing the browser before the data request is received, leading to Connection Reset.
Used PM2 (http://pm2.keymetrics.io/) to run the application, and it is working great now .
I am beating my brains out trying to get this to work. I read all the other answers related to NodeJS UDP on SO already, but to no avail. I am on Windows 10.
Here is the error I am getting:
Uncaught Exception: Error: write ENOTSUP
at exports._errnoException (util.js:1022:11)
at ChildProcess.target._send (internal/child_process.js:654:20)
at ChildProcess.target.send (internal/child_process.js:538:19)
at sendHelper (cluster.js:751:15)
at send (cluster.js:534:12)
at cluster.js:509:7
at SharedHandle.add (cluster.js:99:3)
at queryServer (cluster.js:501:12)
at Worker.onmessage (cluster.js:450:7)
at ChildProcess.<anonymous> (cluster.js:765:8)
at emitTwo (events.js:111:20)
at ChildProcess.emit (events.js:191:7)
at process.nextTick (internal/child_process.js:744:12)
at _combinedTickCallback (internal/process/next_tick.js:67:7)
at process._tickDomainCallback [as _tickCallback] (internal/process/next_tick.js:122:9)
Here is my code:
let dgram = require('dgram'),
server = dgram.createSocket('udp4'),
multicastAddress = '239.255.255.250',
multicastPort = 1900,
myIp = '192.168.51.133';
server.bind(multicastPort, myIp, function () {
server.setBroadcast(true);
server.setMulticastTTL(128);
server.setInterface.getbyname(myIp);
server.addMembership(multicastAddress, myIp);
});
//wait for incoming messages and print ip address
server.on('message', function (data, rinfo) {
console.log(new Date() + ' RECEIVER received from ', rinfo.address, ':');
console.log(data.toString());
});
//Set up discovery message. Make sure to leave out any extra space in the message.
var discover_message = new Buffer('M-SEARCH * HTTP/1.1\r\nHost: 239.255.255.250:1900\r\nMan: ssdp:discover\r\nST: colortouch:ecp\r\n');
server.send(discover_message, 0, discover_message.length, 1900, multicastAddress);
Finally found an answer for this. The issue is due to being on Windows and using clusters in Node. The problem is on the server.bind call. Here is the correct, working code:
server.bind({port: 1900, exclusive: true}, function () {
console.log('PORT BIND SUCCESS');
server.setBroadcast(true);
server.setMulticastTTL(128);
server.addMembership(multicastAddress, myIp);
});
The fix was to pass in the object {port: 1900, exclusive: true}. Source: https://github.com/misterdjules/node/commit/1a87a95d3d7ccc67fd74145c6f6714186e56f571
I have a function in NodeJS using Mongoose driver like below:
Pseudocode:
function someFn(someParams) {
// Step 1: a couple of very fast mongo queries (in milliseconds)
// Step 2: HUGE CPU processing - think millions of data grouped, mapped, etc. (takes about a minute)
// Step 3: another mongo query which inserts the results from Step 2 into a collection
}
At step 3, I get the following error:
MongoError: connection 4 to cluster closed
at Function.MongoError.create (/home/some-user/my-repo/node_modules/mongodb-core/lib/error.js:29:11)
at TLSSocket.<anonymous> (/home/some-user/my-repo/node_modules/mongodb-core/lib/connection/connection.js:202:22)
at Object.onceWrapper (events.js:293:19)
at emitOne (events.js:101:20)
at TLSSocket.emit (events.js:191:7)
at _handle.close (net.js:513:12)
at Socket.done (_tls_wrap.js:332:7)
at Object.onceWrapper (events.js:293:19)
at emitOne (events.js:101:20)
at Socket.emit (events.js:191:7)
at TCP._handle.close [as _onclose] (net.js:513:12)
My MongoDB connection params are as follows:
mongoose.connect(connStr, {
server: {
socketOptions: {
keepAlive: 300000,
connectTimeoutMS: 300000,
socketTimeoutMS: 300000,
auto_reconnect: true
}
}
});
I don't understand why I'm getting this error at Step 3. Can someone help me out with this, please?
Figured out the issue after hours of debugging. My Step 3 mongoose query had too many documents(in the order of millions, from Step 2). The error from mongoose gives no reason why the connection is closing. A message like Too many documents or Too large query would've gone a long way in saving a lot of time.