How to have express.post to wait for a response from a socket created using net client? net here means the package require("net")
Architecture
browser <-> express with built in unix domain client <-> some unix domain server
My express server serves the front end as usual. Sometimes, it gets information from other services running on the same machine. These services are outside my control. I use net to create a client to connect with them and this works fine.
Tried approaches
All the usual answers about express.post using, for example, a promise, are not applicable because they wait for a reply on the function you call.
For example. fs.readFile will return something related to the completion of fs.readFile and this can easily be promisified using the examples on the internet.
Node.js promise request return
https://www.intuz.com/blog/promises-in-node-js-with-examples
https://dzone.com/articles/how-to-interact-with-a-database-using-promises-in
https://www.turtle-techies.com/using-promises-with-express-js/
The problem
However, with net, the reply comes from somewhere else.
client.write is the method for sending. This method just returns true, not the response from the server.
We become aware of the response at the client.data event and I can't figure how to get a promise to watch the client.data event to fulfill the promise.
My best attempt
Using turtle-techies as an example (all above are similar though differing syntax):
const readFilePromise = (filename) => new Promise((resolve, reject) => {
fs.readFile(filename, 'utf-8', (err, data) => {
if (err) {
reject(err);
} else {
resolve(data);
}
});
});
const app = express();
//call the promisified long-running function. ONLY works if the function returns the data you need. client.write does not.
app.post('/get-file', (req, res) => {
readFilePromise('important_file.txt')
.then(data => res.end(data))
.catch(err => res.end(`could not serve data: ${err}`))
});
Now I try rewriting the promisified function or method to suit net.client:
var net = require('net');
var socketName = '/tmp/dp.sock';
var client = net.createConnection(socketName);
client.on("connect", function() {
client.write('client on connect');
});
client.on("data", function(data) {
console.log("client.on data: ", data.toString());
//I don't see a way to get a handle here on express.post req,res so
// I think the answer lies elswhere. Anyway, it would be messy to put it here.
});
client.on('close', function() {
console.log('client on close');
});
//========= end of client building ===========
//========= promisified client.write =========
const readFilePromise = (filename) => new Promise((resolve, reject) => {
client.write('my message'); //This just returns true which I don't care about
//now what to do here to become aware of client.ondata and handle its data?
});
I have a feeling the answer is in front of my face.
The answer is here: https://techbrij.com/node-js-tcp-server-client-promisify
In principle, wrap the client creation and client.write in a new class in which you can then do any promisification you like.
In case the link breaks, here's what they say (modified by me to be for unix domain socket; techbrij.com did tcp)
//in a module.js file
const net = require('net');
const DEF_socketName = '/tmp/echo.sock';
class Client {
constructor(socketName = '/tmp/echo.sock') {
this.socket = new net.createConnection(socketName);
this.address = socketName || DEF_socketName;
this.init();
}
init() {
var client = this;
client.socket.on("connect", function() {
client.socket.write('hello from unix client!');
});
client.socket.on("data", function(data) { //approahch 1 here we delete the function def and simply put dodata
console.log("client.on data: ", data.toString());
//client.destroy();
});
client.socket.on('close', function() {
console.log('Connection closed');
});
}
sendMessage(message) {
console.log('client.sendMessage: ' + message);
var client = this;
return new Promise((resolve, reject) => {
client.socket.write(message);
client.socket.on('data', (data) => {
resolve(data);
if (data.toString().endsWith('exit')) {
client.socket.destroy();
}
});
client.socket.on('error', (err) => {
reject(err);
});
});
}
}
module.exports = Client;
Then in a client.js file:
const Client = require('./client_mod');
const client = new Client();
client.sendMessage('A')
.then((data)=> { console.log(`Received: ${data}`); return client.sendMessage('B');} )
.then((data)=> { console.log(`Received: ${data}`); return client.sendMessage('C');} )
.then((data)=> { console.log(`Received: ${data}`); return client.sendMessage('exit');} )
.catch((err) =>{ console.error(err); })
Hello I'm trying to create a chat application, I googled around and I got some issues on this step. Would appreciate some help...
Server.js
var express = require("express");
var app = express();
var http = require("http").createServer(app);
var io = require("socket.io")(http);
var users = [];
io.on("connection", function (socket) {
console.log("User connected", socket.id);
socket.on("user_connected", function (username) {
users[username] = socket.id;
io.emit("user_connected", username);
});
socket.on("send_message", function (data) {
var socketId = users[data];
io.to(socketId).emit("new_message", data);
console.log(data);
});
});
http.listen(3000, function () {
console.log("Server Started");
});
chat.php
function sendMessage(){
var message = document.getElementById("message").value;
io.emit("send_message", {
sender: sender,
message: message
});
return false;
}
io.on("new_message", function (data) {
console.log(data);
//var html = "";
//html += "<li>" + data.sender + " says: " + data.message + "</li>";
//document.getElementById("messages").innerHTML += html;
});
So my problem is happening in chat.php where my console.log(data) isn't shown, however the data is shown in server.js. Why is this currently not working?
From what you said earlier it's possible that you make it more complicated than it actually is. No need to change anything in chat.php, however instead of creating the variable socketId you could just emit the data immediately like this:
io.on("connection", function (socket) {
console.log("User connected", socket.id);
socket.on("user_connected", function (username) {
users[username] = socket.id;
io.emit("user_connected", username);
});
socket.on("send_message", function (data) {
io.emit("new_message", data);
});
});
My nodejs server under heavy load for some reason, all i'm running is a small app using socket io. This app mainly gets text from a textarea on an html page and updates it to the people in the "room". Sort of like a live chat. However this is causing huge load on my server every time someone types something if theres 2 clients per room it will cause a lag spike.
server-side code
var app = require('http').createServer(handler)
var io = require('socket.io')(app);
var fs = require('fs');
app.listen(process.env.PORT);
function handler (req, res) {
fs.readFile(__dirname + '/index.html',
function (err, data) {
if (err) {
res.writeHead(500);
return res.end('Error loading index.html');
}
res.writeHead(200);
res.end(data);
});
}
io.on('connection', function (socket) {
socket.on('join_room',function(room){
socket.join(room);
socket.in(room).emit('joined_room', 'true');
});
socket.on('blob', function (room) {
socket.in(room.name).emit('new_text', room.update_text);
});
socket.on('blob-typing',function(room){
socket.in(room.name).emit('some_typing', {cur_loc:room.cursor_location});
});
});
client-side code
var socket = io.connect('x');
socket.emit('join_room',url_key);
setInterval(function(){
if (field_change === true) {
var post_data = {
name: url_key,
update_text: $('.blob').val()
};
socket.emit('blob', post_data);
post_data = {
name: url_key,
cursor_location:0
};
socket.emit('blob-typing', post_data);
}
field_change = false;
},1000);
$(".blob").on('keyup paste input',function () {
delay(function(){
field_change = true;
},500);
});
socket.on('joined_room',function(data){
});
var delay = (function(){
var timer = 0;
return function(callback, ms){
clearTimeout (timer);
timer = setTimeout(callback, ms);
};
})();
Please advise as to how i can fix the load, any technique or strategy would be much appreciated!
I'm working on chat, where facebook friends can talk only with each other. I'm using redis so save relation: fb_user_id - user_socket_id. This is how my implementation looks like:
getting friends from facebook;
selecting socket ids of my friends from redis, creating local friends-sockets list in my node client;
connecting to node server. Server saving my socket id to redis and notifying all my friends about new friend login (about me);
all my friends updating local friends-sockets list;
when someone sending chat message to server, this message comes with friends-sockets list, so server knows where need to send message (only for my friends).
Question: it's better solution to send friends-sockets every time to server, or it's better to get this relation on server from redis (or create array of sockets in server). How to adapt my task for high availability?
Any comments and suggestions are welcomed, thanks.
Here is my code (socket.io 1.2.0)
server.js
var app = require('express')();
var http = require('http').Server(app);
var io = require('socket.io')(http);
var redis = require("redis"), client = redis.createClient();
var parts;
client.select(2, function() {});
client.on("error", function (err) {
console.log("Error " + err);
});
process.on('uncaughtException', function (err) {
console.log(err);
});
app.get('/', function (req, res) {
res.sendFile(__dirname + '/index.html');
});
io.on('connection', function (socket) {
// on connect
socket.on("join", function (data)
{
if (data) {
// notify all friedns about new friend login
if (data.to) {
if (data.to.length > 0) {
for (x in data.to) {
io.to(data.to[x]['socket_id']).emit('new friend response', {uid: data.uid, sid: socket.id});
}
}
}
// save or update user socket id to redis
parts = split_id(data.uid);
client.hset(parts[1], parts[0], socket.id);
}
});
// disconnect
socket.on('disconnect', function () {
console.log("user disconnected");
});
// send message by friends-sockets list
socket.on('chat message', function (data) {
if (data.to.length > 0) {
for (x in data.to) {
var message = data.msg;
io.to(data.to[x]['socket_id']).emit('chat message response', {msg: message, uid: data.uid});
}
}
});
});
http.listen(3000, function () {
console.log('listening on *:3000');
});
// split facebook uid in 2 parts (for redis saving)
function split_id(str)
{
var n = str.length;
var res1 = str.substr(n - 2, 2);
var res2 = str.substr(0, n - 2);
return [res1, res2];
}
client.js
// friends socket list
var friends_sockets = [];
// my data from facebook
var my_data;
// my facebook uid
var my_uid;
function client() {
socket = io('http://server.com:3000');
// connect
socket.on('connect', function () {
// notify server about login
socket.emit('join', {uid: my_uid, to: friends_sockets, from: my_data, type: 'web'});
});
// send chat message to my friends
$('.enter_form button').click(function () {
if (friends_sockets.length > 0) {
socket.emit('chat message', {msg: $('#m').val(), to: friends_sockets, from: my_data, uid: my_uid});
}
// add message to my chat
$('#messages').append($('<li>').text(my_data.first_name + ' ' + my_data.last_name + ': ' + $('#m').val()));
$('#m').val('');
return false;
});
// new message listner (waiting for chat messages)
socket.on('chat message response', function (data) {
$('#messages').append($('<li>').text(data.msg));
});
// new friends lister (update list on friends login)
socket.on('new friend response', function (data) {
var found = false;
if (friends_sockets.length > 0) {
for (x in friends_sockets) {
if (friends_sockets[x]['uid'] == data.uid) {
friends_sockets[x]['socket_id'] = data.sid;
found = true;
}
}
}
if (found === false) {
friends_sockets.push(data);
}
});
}
Concerning your question regarding high availablity, have a look at
http://socket.io/docs/using-multiple-nodes/
for a configuration example using nginx, multiple Node processes and Redis as session store.
Could anyone provide a rock-solid, dead-simple unit test for Node.js using WebSockets (Socket.io)?
I'm using socket.io for Node.js, and have looked at socket.io-client for establishing the client connection to a server in the test. However, I seem to be missing something.
In the example below, "worked..." never gets printed out.
var io = require('socket.io-client')
, assert = require('assert')
, expect = require('expect.js');
describe('Suite of unit tests', function() {
describe('First (hopefully useful) test', function() {
var socket = io.connect('http://localhost:3001');
socket.on('connect', function(done) {
console.log('worked...');
done();
});
it('Doing some things with indexOf()', function() {
expect([1, 2, 3].indexOf(5)).to.be.equal(-1);
expect([1, 2, 3].indexOf(0)).to.be.equal(-1);
});
});
});
Instead, I simply get:
Suite of unit tests
First (hopefully useful) test
✓ Doing some things with indexOf()
1 test complete (26 ms)
Any suggestions?
After further poking and prodding, I found some incredibly useful information. In the author's example, he points out the critical step of establishing socket listeners in the before hooks.
This example works:
Assuming a server is listening for socket connections at localhost:3001, of course
var io = require('socket.io-client')
, assert = require('assert')
, expect = require('expect.js');
describe('Suite of unit tests', function() {
var socket;
beforeEach(function(done) {
// Setup
socket = io.connect('http://localhost:3001', {
'reconnection delay' : 0
, 'reopen delay' : 0
, 'force new connection' : true
});
socket.on('connect', function() {
console.log('worked...');
done();
});
socket.on('disconnect', function() {
console.log('disconnected...');
})
});
afterEach(function(done) {
// Cleanup
if(socket.connected) {
console.log('disconnecting...');
socket.disconnect();
} else {
// There will not be a connection unless you have done() in beforeEach, socket.on('connect'...)
console.log('no connection to break...');
}
done();
});
describe('First (hopefully useful) test', function() {
it('Doing some things with indexOf()', function(done) {
expect([1, 2, 3].indexOf(5)).to.be.equal(-1);
expect([1, 2, 3].indexOf(0)).to.be.equal(-1);
done();
});
it('Doing something else with indexOf()', function(done) {
expect([1, 2, 3].indexOf(5)).to.be.equal(-1);
expect([1, 2, 3].indexOf(0)).to.be.equal(-1);
done();
});
});
});
I found that the placement of done() in the beforeEach, socket.on('connect'...) listener was crucial to having the connection get established. For example, if you comment out done() in the listener, then add it one scope out (just before exiting the beforeEach), you'll see the "no connection to break..." message instead of the "disconnecting..." message. Like so:
beforeEach(function(done) {
// Setup
socket = io.connect('http://localhost:3001', {
'reconnection delay' : 0
, 'reopen delay' : 0
, 'force new connection' : true
});
socket.on('connect', function() {
console.log('worked...');
//done();
});
socket.on('disconnect', function() {
console.log('disconnected...');
});
done();
});
I'm new to Mocha, so there's probably a very obvious reason to the initiated for placing done() within the socket scope itself. Hopefully that little detail will save others in my shoes from hair pulling.
For me, the above test (with correct scoping of done()) outputs:
Suite of unit tests
First (hopefully useful) test
◦ Doing some things with indexOf(): worked...
✓ Doing some things with indexOf()
disconnecting...
disconnected...
◦ Doing something else with indexOf(): worked...
✓ Doing something else with indexOf()
disconnecting...
disconnected...
2 tests complete (93 ms)
Offering an extension of the accepted answer here. Has basic client to server communication useful as boilerplate for other future tests. Using mocha, chai, and expect.
var io = require('socket.io-client')
, io_server = require('socket.io').listen(3001);
describe('basic socket.io example', function() {
var socket;
beforeEach(function(done) {
// Setup
socket = io.connect('http://localhost:3001', {
'reconnection delay' : 0
, 'reopen delay' : 0
, 'force new connection' : true
, transports: ['websocket']
});
socket.on('connect', () => {
done();
});
socket.on('disconnect', () => {
// console.log('disconnected...');
});
});
afterEach((done) => {
// Cleanup
if(socket.connected) {
socket.disconnect();
}
io_server.close();
done();
});
it('should communicate', (done) => {
// once connected, emit Hello World
io_server.emit('echo', 'Hello World');
socket.once('echo', (message) => {
// Check that the message matches
expect(message).to.equal('Hello World');
done();
});
io_server.on('connection', (socket) => {
expect(socket).to.not.be.null;
});
});
});
Dealing with callbacks and promises yourself can be difficult and non trivial examples quickly become very complex and hard to read.
There is a tool called socket.io-await-test available via NPM that allows you to suspend/wait in a test until events have been triggered using the await keyword.
describe("wait for tests", () => {
it("resolves when a number of events are received", async () => {
const tester = new SocketTester(client);
const pongs = tester.on('pong');
client.emit('ping', 1);
client.emit('ping', 2);
await pongs.waitForEvents(2) // Blocks until the server emits "pong" twice.
assert.equal(pongs.get(0), 2)
assert.equal(pongs.get(1), 3)
})
})
Check out this boilerplate solution that's based on promises and good practice.
You can test your servers entire io events with it, no sweat.
You just need to copy a boilerplate test and add your own code as needed.
Checkout the repo on GitHub for full source code.
https://github.com/PatMan10/testing_socketIO_server
const io = require("socket.io-client");
const ev = require("../utils/events");
const logger = require("../utils/logger");
// initSocket returns a promise
// success: resolve a new socket object
// fail: reject a error
const initSocket = () => {
return new Promise((resolve, reject) => {
// create socket for communication
const socket = io("localhost:5000", {
"reconnection delay": 0,
"reopen delay": 0,
"force new connection": true
});
// define event handler for sucessfull connection
socket.on(ev.CONNECT, () => {
logger.info("connected");
resolve(socket);
});
// if connection takes longer than 5 seconds throw error
setTimeout(() => {
reject(new Error("Failed to connect wihtin 5 seconds."));
}, 5000);
}
);
};
// destroySocket returns a promise
// success: resolve true
// fail: resolve false
const destroySocket = socket => {
return new Promise((resolve, reject) => {
// check if socket connected
if (socket.connected) {
// disconnect socket
logger.info("disconnecting...");
socket.disconnect();
resolve(true);
} else {
// not connected
logger.info("no connection to break...");
resolve(false);
}
});
};
describe("test suit: Echo & Bello", () => {
test("test: ECHO", async () => {
// create socket for communication
const socketClient = await initSocket();
// create new promise for server response
const serverResponse = new Promise((resolve, reject) => {
// define a handler for the test event
socketClient.on(ev.res_ECHO, data4Client => {
//process data received from server
const { message } = data4Client;
logger.info("Server says: " + message);
// destroy socket after server responds
destroySocket(socketClient);
// return data for testing
resolve(data4Client);
});
// if response takes longer than 5 seconds throw error
setTimeout(() => {
reject(new Error("Failed to get reponse, connection timed out..."));
}, 5000);
});
// define data 4 server
const data4Server = { message: "CLIENT ECHO" };
// emit event with data to server
logger.info("Emitting ECHO event");
socketClient.emit(ev.com_ECHO, data4Server);
// wait for server to respond
const { status, message } = await serverResponse;
// check the response data
expect(status).toBe(200);
expect(message).toBe("SERVER ECHO");
});
test("test BELLO", async () => {
const socketClient = await initSocket();
const serverResponse = new Promise((resolve, reject) => {
socketClient.on(ev.res_BELLO, data4Client => {
const { message } = data4Client;
logger.info("Server says: " + message);
destroySocket(socketClient);
resolve(data4Client);
});
setTimeout(() => {
reject(new Error("Failed to get reponse, connection timed out..."));
}, 5000);
});
const data4Server = { message: "CLIENT BELLO" };
logger.info("Emitting BELLO event");
socketClient.emit(ev.com_BELLO, data4Server);
const { status, message } = await serverResponse;
expect(status).toBe(200);
expect(message).toBe("SERVER BELLO");
});
});
---- Foot Note ----
Depending on how you setup your server environment, you may experience environmental conflict between socket.io and socket.io-client running from the same project simultaneously. In which case it would be better to separate the project into a "test client" and a server. Checkout below repo if you get this issue.
https://github.com/PatMan10/testing_socketIO_server_v2
In OP's code,
socket.on('connect', function(done) {
console.log('worked...');
done();
});
the done was applied to the wrong callback. It should be removed from the socket.on callback and added to Mocha's it block callback:
it('First (hopefully useful) test', function (done) {
var socket = io.connect('http://localhost:3001');
socket.on('connect', function () {
console.log('worked...');
done();
});
});
A complete example
Existing answers are great but don't show the server ultimately being tested. Here's a complete version with console.logs to illustrate what's going on. Explanation follows.
src/server.js:
const express = require("express");
const createServer = (port=3000) => {
const app = express();
const http = require("http").Server(app);
const io = require("socket.io")(http);
io.on("connection", socket => {
console.log("[server] user connected");
socket.on("message", msg => {
console.log(`[server] received '${msg}'`);
socket.emit("message", msg);
});
socket.on("disconnect", () => {
console.log("[server] user disconnected");
});
});
http.listen(port, () =>
console.log(`[server] listening on port ${port}`)
);
return {
close: () => http.close(() =>
console.log("[server] closed")
)
};
};
module.exports = {createServer};
test/server.test.js:
const {expect} = require("chai");
const io = require("socket.io-client");
const {createServer} = require("../src/server");
const socketUrl = "http://localhost:3000";
describe("server", function () {
this.timeout(3000);
let server;
let sockets;
beforeEach(() => {
sockets = [];
server = createServer();
});
afterEach(() => {
sockets.forEach(e => e.disconnect())
server.close();
});
const makeSocket = (id=0) => {
const socket = io.connect(socketUrl, {
"reconnection delay": 0,
"reopen delay": 0,
"force new connection": true,
transports: ["websocket"],
});
socket.on("connect", () => {
console.log(`[client ${id}] connected`);
});
socket.on("disconnect", () => {
console.log(`[client ${id}] disconnected`);
});
sockets.push(socket);
return socket;
};
it("should echo a message to a client", done => {
const socket = makeSocket();
socket.emit("message", "hello world");
socket.on("message", msg => {
console.log(`[client] received '${msg}'`);
expect(msg).to.equal("hello world");
done();
});
});
it("should echo messages to multiple clients", () => {
const sockets = [...Array(5)].map((_, i) => makeSocket(i));
return Promise.all(sockets.map((socket, id) =>
new Promise((resolve, reject) => {
const msgs = [..."abcd"].map(e => e + id);
msgs.slice().forEach(e => socket.emit("message", e));
socket.on("message", msg => {
console.log(`[client ${id}] received '${msg}'`);
expect(msg).to.equal(msgs.shift());
if (msgs.length === 0) {
resolve();
}
});
})
));
});
});
In summary, the server exports a function that lets a server app be created from scratch, allowing each it block to be idempotent and avoid server state from carrying between tests (assuming no persistence on the server otherwise). Creating an app returns an object with a close function. socket.disconnect() must be called per socket in each test to avoid timeouts.
Given these requirements, the testing suite follows this per-test setup/teardown workflow:
let server;
let sockets;
beforeEach(() => {
sockets = [];
server = createServer();
});
afterEach(() => {
sockets.forEach(e => e.disconnect())
server.close();
});
makeSocket is an optional helper to reduce the repeated boilerplate of connecting and disconnecting a socket client. It does produce a side effect on the sockets array for cleanup later, but this is an implementation detail from the it block's perspective. Test blocks shoudn't touch server or sockets variables, although other workflows are likely depending on need. The critical takeaways are test case idempotency and closing all connections after each test case.
Options on the socket.connect object on the client let you choose transport and behavior of the socket. "force new connection": true creates a new Manager per socket instead of reusing an existing one and transports: ["websocket"] upgrades to WS protocol from long polling immediately.
Use it("should ... ", done => { /* tests */ }); and invoke done() after all work is completed in callbacks or return a promise (and omit the done parameter to the it callback). The example above shows both approaches.
Used in this post:
node: 12.19.0
chai: 4.2.0
express: 4.16.4
mocha: 5.2.0
socket.io: 2.2.0
socket.io-client: 2.2.0
I had this problem: How to do unit test with a "socket.io-client" if you don't know how long the server take to respond?.
I've solved so using mocha and chai:
var os = require('os');
var should = require("chai").should();
var socketio_client = require('socket.io-client');
var end_point = 'http://' + os.hostname() + ':8081';
var opts = {forceNew: true};
describe("async test with socket.io", function () {
this.timeout(10000);
it('Response should be an object', function (done) {
setTimeout(function () {
var socket_client = socketio_client(end_point, opts);
socket_client.emit('event', 'ABCDEF');
socket_client.on('event response', function (data) {
data.should.be.an('object');
socket_client.disconnect();
done();
});
socket_client.on('event response error', function (data) {
console.error(data);
socket_client.disconnect();
done();
});
}, 4000);
});
});