I'm attempting to make a tcp game server in node.js but am having problems with data transmission. I want to get the length (in bytes) of the buffer i receive in the sockets DATA event so logically, under the assumption that the object is a buffer (from what i've read) i tried this :
var server = net.createServer();
server.on("connection", function(socket) {
socket.on("data", function(data) {
console.log(data.length);
});
});
But apparently it returns undefined. This confused me quite alot so i tried to see if i'm even handling a buffer here by doing this :
var server = net.createServer();
server.on("connection", function(socket) {
socket.on("data", function(data) {
console.log(Buffer.isBuffer(data));
});
});
And i got false! Could anybody explain what i'm doing wrong?
ps. i have no encoding set on the socket
This code is fine. I messed up somewhere else!
Related
Code
var websock = net.createServer(function(sock) {
sock.pipe(sock);
sock.setEncoding('utf8');
sock.setKeepAlive(true);
sock.on("data", function(d) {
console.log("websock", d);
});
sock.on('end', function() {
console.log('websock disconnected');
});
});
websock.listen(777, '127.0.0.1');
After few minutes ~15 mins the callback code in sock.on("data", function() {}) seems not to be working. why is it the case? I checked the console.log, there is no log with a string "websock disconnected".
if the socket is not disconnected and if there is no error, what has happened to the socket connection or the data stream?
On the other end, (Server side, data sender) seems to be streaming data continuously while client side (nodejs app) has stopped receiving data.
The issue arises from your use of the pipe mechanism to echo back data which is never consumed on the original side (communication is unidirectional):
sock.pipe(sock);
This makes your code work as an echo server. Your socket "sock" is a duplex stream (i.e. both readable - for the incoming data you receive, and writable - for outgoing data you send back).
A quick fix if you don't need to respond back and you just need to receive data is to simply delete the "sock.pipe(sock);" line. To find out the explanation, read ahead.
Most probably your data source (the MT5 application you mentioned) sends data continuously and it doesn't read what you send back at all. So, your code keeps echoing back the received data using sock.pipe(sock), filling the outgoing buffer which is never consumed. However, the pipe mechanism of Nodejs streams handles backpressure, which means that when two streams (a readable and a writable one) are connected by a pipe, if the outgoing buffer is filling (reaching a high watermark), the readable stream is paused, to prevent the "overflow" of the writable stream.
You can read more about backpressure in the Nodejs docs. This fragment particularly describes how streams are handling backpressure:
In Node.js the source is a Readable stream and the consumer is the Writable stream [...]
The moment that backpressure is triggered can be narrowed exactly to the return value of a Writable's .write() function. [...]
In any scenario where the data buffer has exceeded the highWaterMark or the write queue is currently busy, .write() will return false.
When a false value is returned, the backpressure system kicks in. It will pause the incoming Readable stream from sending any data and wait until the consumer is ready again.
Below you can find my setup to show where backpressure kicks in; there are two files, server.js and client.js. If you run them both, server will write to console "BACKPRESSURE" soon. As the server is not handling backpressure (it ignores that sock.write starts returning false at some point), the outgoing buffer is filled and filled, consuming more memory, while in your scenario, socket.pipe was handling backpressure and thus it paused the flow of the incoming messages.
The server:
// ----------------------------------------
// server.js
var net = require('net');
var server = net.createServer(function (socket) {
console.log('new connection');
// socket.pipe(socket); // replaced with socket.write on each 'data' event
socket.setEncoding('utf8');
socket.setKeepAlive(true);
socket.on("data", function (d) {
console.log("received: ", d);
var result = socket.write(d);
console.log(result ? 'write ok' : 'BACKPRESSURE');
});
socket.on('error', function (err) {
console.log('client error:', err);
});
socket.on('end', function () {
console.log('client disconnected');
});
});
server.listen(10777, '127.0.0.1', () => {
console.log('server listening...');
});
The client:
// ----------------------------------------
// client.js
var net = require('net');
var client = net.createConnection(10777, () => {
console.log('connected to server!' + new Date().toISOString());
var count = 1;
var date;
while(count < 35000) {
count++;
date = new Date().toISOString() + '_' + count;
console.log('sending: ', date);
client.write(date + '\n');
}
});
client.on('data', (data) => {
console.log('received:', data.toString());
});
client.on('end', () => {
console.log('disconnected from server');
});
Currently having an issue with "Broadcast" not seeming to function properly with a super simple websocket setup I started in Node. This is my first time working with websockets so I may be missing something pretty apparent, but after looking online for a while I wasn't able to find a resolution.
Basically I am just trying to have the ability to push some json out to all currently connected Clients.
I can confirm that that socket is working because I am able to see the static connection string on 'ws.send' in the 'connection' block when I connect at ws://localhost:3000, as well as seeing mulitple clients logged out from the broadcast method if I connect with multiple clients.
Any help as to what I may be missing would be greatly appreciated,
var WebSocketServer = require('uws').Server;
var wss = new WebSocketServer({ port: 3000 }); // ws://localhost:3000
// Static test var
var test_message = {
'test': 'Response',
'test2': 'Response2'
};
// Broadcast to all.
wss.broadcast = function broadcast(data) {
wss.clients.forEach(function each(client) {
console.log('IT IS GETTING INSIDE CLIENTS');
console.log(client);
// The data is coming in correctly
console.log(data);
client.send(data);
});
};
wss.on('connection', function(ws) {
ws.on('message', function(message) {
wss.broadcast(test_message);
console.log('Received: ' + message);
});
// TODO This is static just to check that the connection is properly working
ws.send('You successfully connected to the websocket.');
});
I tested your code with Smart Websocket Client. Your code is fine. If you broadcast data having string only, then you can see the reply in UI, but for javascript object, the client doesn't display although you can see response as Binary Frames (opcode = 2) in Chrome Developer Tools.
The reason behind this behavior is that the ws.send() method support normal strings, typed arrays or blobs, but sending typed arrays and blobs will result in the frame(s) received by the client as binary frames (opcode = 2).
You can try JSON.stringify the object,
wss.on('connection', function(ws) {
ws.on('message', function(message) {
wss.broadcast(JSON.stringify(test_message));
console.log('Received: ' + message);
});
ws.send('You successfully connected to the websocket.');
});
So I modified the accepted answer in this thread How do I shutdown a Node.js http(s) server immediately? and was able to close down my nodejs server.
// Create a new server on port 4000
var http = require('http');
var server = http.createServer(function (req, res) { res.end('Hello world!'); }).listen(4000);
// Maintain a hash of all connected sockets
var sockets = {}, nextSocketId = 0;
server.on('connection', function (socket) {
// Add a newly connected socket
var socketId = nextSocketId++;
sockets[socketId] = socket;
console.log('socket', socketId, 'opened');
// Remove the socket when it closes
socket.on('close', function () {
console.log('socket', socketId, 'closed');
delete sockets[socketId];
});
});
...
when (bw == 0) /*condition to trigger closing the server*/{
// Close the server
server.close(function () { console.log('Server closed!'); });
// Destroy all open sockets
for (var socketId in sockets) {
console.log('socket', socketId, 'destroyed');
sockets[socketId].destroy();
}
}
However, on the client side, the client throws a ConnectionException because of the server.close() statement. I want the client to throw a SocketTimeoutException, which means the server is active, but the socket just hangs. Someone else was able to do this with a jetty server from Java
Server jettyServer = new Server(4000);
...
when (bw == 0) {
server.getConnectors()[0].stop();
}
Is it possible to achieve something like that? I've been stuck on this for a while, any help is extremely appreciated. Thanks.
What you ask is impossible. A SocketTimeoutException is thrown when reading from a connection that hasn't terminated but hasn't delivered any data during the timeout period.
A connection closure does not cause it. It doesn't cause a ConnectionException either, as there is no such thing. It causes either an EOFException, a null return from readLine(), a -1 return from read(), or an IOException: connection reset if the close was abortive.
Your question doesn't make sense.
Can someone give me a working example on how to create a server listening on the stream and reply when there is a request coming through.
Here's what I have so far:
var port = 4567,
net = require('net');
var sockets = [];
function receiveData(socket, data) {
console.log(data);
var dataToReplyWith = ' ... ';
// ... HERE I need to reply somehow with something to the client that sent the initial data
}
function closeSocket(socket) {
var i = sockets.indexOf(socket);
if (i != -1) {
sockets.splice(i, 1);
}
}
var server = net.createServer(function (socket) {
console.log('Connection ... ');
sockets.push(socket);
socket.setEncoding('utf8');
socket.on('data', function(data) {
receiveData(socket, data);
})
socket.on('end', function() {
closeSocket(socket);
});
}).listen(port);
Will the socket.write(dataToReplyWith); do it?
Yes, you can just write to the socket whenever (as long as the socket is still writable of course). However the problem you may run into is that one data event may not imply a complete "request." Since TCP is just a stream, you can get any amount of data which may not align along your protocol message boundaries. So you could get half a "request" on one data event and then the other half on another, or the opposite: multiple "requests" in a single data event.
If this is your own custom client/server design and you do not already have some sort of protocol in place to determine "request"/"response" boundaries, you should incorporate that now.
I want to be able to handle all messages that are coming in from clients in a single handler.
Example client code:
var socket = io.connect('http://localhost');
socket.emit('news', { hello: 'test' });
socket.emit('chat', { hello: 'test' });
Example server code:
io.sockets.on('connection', function (socket) {
socket.on('message', function (data) {
console.log(data);
}); });
I'd like to be able to log every message even if its sent on news, chat or whatever other name using emit. Is this possible?
Note: The above server code does not work. There is nothing currently logged. I am just wondering if there is a single event which could be handled for all messages for every emit name.
That is possible by overriding socket.$emit function
//Original func
var x = socket.$emit;
socket.$emit = function(){
var event = arguments[0];
var feed = arguments[1];
//Log
console.log(event + ":" + feed);
//To pass listener
x.apply(this, Array.prototype.slice.call(arguments));
};
It's even easier on Socket.Io >3 using the socket.onAny(listener):
this.socket.onAny(m => {
..
});
This is supported out of the box now as of Socket-io 2.0.4, you simply need to register a middle ware (source from socketio-wildcard):
As of Socket.io v2.0.4 (commit), you can use a socket middleware to
catch every incoming Packet, which satisfies most of
socketio-wildcard's use cases.
io.on('connection', (socket) => {
socket.use((packet, next) => {
// Handler
next();
});
});
This is an open issue with Socket.IO.
At this time, if you really need it, you will probably have to fork Socket.IO. See 3rd-Edens comment for how to do it.