How to have express.post to wait for a response from a socket created using net client? net here means the package require("net")
Architecture
browser <-> express with built in unix domain client <-> some unix domain server
My express server serves the front end as usual. Sometimes, it gets information from other services running on the same machine. These services are outside my control. I use net to create a client to connect with them and this works fine.
Tried approaches
All the usual answers about express.post using, for example, a promise, are not applicable because they wait for a reply on the function you call.
For example. fs.readFile will return something related to the completion of fs.readFile and this can easily be promisified using the examples on the internet.
Node.js promise request return
https://www.intuz.com/blog/promises-in-node-js-with-examples
https://dzone.com/articles/how-to-interact-with-a-database-using-promises-in
https://www.turtle-techies.com/using-promises-with-express-js/
The problem
However, with net, the reply comes from somewhere else.
client.write is the method for sending. This method just returns true, not the response from the server.
We become aware of the response at the client.data event and I can't figure how to get a promise to watch the client.data event to fulfill the promise.
My best attempt
Using turtle-techies as an example (all above are similar though differing syntax):
const readFilePromise = (filename) => new Promise((resolve, reject) => {
fs.readFile(filename, 'utf-8', (err, data) => {
if (err) {
reject(err);
} else {
resolve(data);
}
});
});
const app = express();
//call the promisified long-running function. ONLY works if the function returns the data you need. client.write does not.
app.post('/get-file', (req, res) => {
readFilePromise('important_file.txt')
.then(data => res.end(data))
.catch(err => res.end(`could not serve data: ${err}`))
});
Now I try rewriting the promisified function or method to suit net.client:
var net = require('net');
var socketName = '/tmp/dp.sock';
var client = net.createConnection(socketName);
client.on("connect", function() {
client.write('client on connect');
});
client.on("data", function(data) {
console.log("client.on data: ", data.toString());
//I don't see a way to get a handle here on express.post req,res so
// I think the answer lies elswhere. Anyway, it would be messy to put it here.
});
client.on('close', function() {
console.log('client on close');
});
//========= end of client building ===========
//========= promisified client.write =========
const readFilePromise = (filename) => new Promise((resolve, reject) => {
client.write('my message'); //This just returns true which I don't care about
//now what to do here to become aware of client.ondata and handle its data?
});
I have a feeling the answer is in front of my face.
The answer is here: https://techbrij.com/node-js-tcp-server-client-promisify
In principle, wrap the client creation and client.write in a new class in which you can then do any promisification you like.
In case the link breaks, here's what they say (modified by me to be for unix domain socket; techbrij.com did tcp)
//in a module.js file
const net = require('net');
const DEF_socketName = '/tmp/echo.sock';
class Client {
constructor(socketName = '/tmp/echo.sock') {
this.socket = new net.createConnection(socketName);
this.address = socketName || DEF_socketName;
this.init();
}
init() {
var client = this;
client.socket.on("connect", function() {
client.socket.write('hello from unix client!');
});
client.socket.on("data", function(data) { //approahch 1 here we delete the function def and simply put dodata
console.log("client.on data: ", data.toString());
//client.destroy();
});
client.socket.on('close', function() {
console.log('Connection closed');
});
}
sendMessage(message) {
console.log('client.sendMessage: ' + message);
var client = this;
return new Promise((resolve, reject) => {
client.socket.write(message);
client.socket.on('data', (data) => {
resolve(data);
if (data.toString().endsWith('exit')) {
client.socket.destroy();
}
});
client.socket.on('error', (err) => {
reject(err);
});
});
}
}
module.exports = Client;
Then in a client.js file:
const Client = require('./client_mod');
const client = new Client();
client.sendMessage('A')
.then((data)=> { console.log(`Received: ${data}`); return client.sendMessage('B');} )
.then((data)=> { console.log(`Received: ${data}`); return client.sendMessage('C');} )
.then((data)=> { console.log(`Received: ${data}`); return client.sendMessage('exit');} )
.catch((err) =>{ console.error(err); })
I have been trying to write a test suite to my node.js API project and one of their requirements is to control when the server starts and stops. For that, I wrote this code below with two functions: initializeWebServer and stopWebServer.
express.js
const initializeWebServer = () => {
return new Promise((resolve, reject) => {
app = express();
/* some middlewares */
app.use('/', userRoutes);
httpServer = http.createServer(app);
httpServer.listen(3000, (err) => {
if (err) {
reject(err);
return;
}
resolve(app);
});
});
};
const stopWebServer = () => {
return new Promise((resolve, reject) => {
httpServer.close(() => { resolve(); });
});
};
Using mocha to run my tests, I choose to manage my server connection with before and after hooks, using async/await syntax.
user.spec.js
let axiosAPIClient;
before(async () => {
await initializeWebServer();
const axiosConfig = {
baseURL: `http://localhost:3000`,
validateStatus: () => true
};
axiosAPIClient = axios.create(axiosConfig);
});
after(async () => {
await stopWebServer();
});
describe('POST /api/user', () => {
it('when add a new user, then should get back approval with 200 response', async () => {
const userData = {
/* user props */
};
const response = await axiosAPIClient.post('/api/user', userData);
expect(response).to.containSubset({
status: 200,
data: { message: 'User signed up.' }
});
When axios (I tried fetch too) submit any HTTP request, mocha returns the following error: Error: Timeout of 2000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.. I tried to increase the timeout interval, but it didn't work.
If I execute my test suite without hooks, initializing my server with nodemon pointing to my index file with the same await initializeWebServer();, HTTP requests work as it should.
I really don't understand why this is happening, I think it's something with mocha.
I did some debug tests and figured out that I forgot to set up a db connection when I run my test suite, including in before and after hooks. It was a simple mistake.
user.spec.js
let axiosAPIClient;
before(async () => {
await initializeWebServer();
db.setUp();
/* axios config etc. */
after(async () => {
await stopWebServer();
await db.closeConnection();
});
I have a node.js data processing app that pulls some data from mssql. It runs fine and produces the expected results. However, the integration tests aren't working and I would like them to.
Below is the connection management and a test query function. I can see from output and running in the debugger that the test has run and failed before the database has connected. So it seems like my Mocha async setup isn't working, but it looks like everything I've seen in documentation.
node -v
v10.15.0
chai: "^4.2.0",
mocha: "^5.2.0"
mssql: "^4.3.0",
const config = require('./config')
const _ = require('underscore')
const sql = require('mssql')
sql.on('error', err => {
console.error('SQL Error', err)
})
let api = {}
api.connect = async dbConfig => {
return new sql.ConnectionPool(dbConfig).connect(
err => {
if (err)
console.error('Connection error', err)
else
console.log('connected')
})
}
var connecting = api.connect(config.sql)
api.simple = async () => {
let pool = await connecting
let result = await pool.request().query('select 1 as number')
return result.recordset[0].number
}
module.exports = api
Here is my mocha test for it that fails
const { expect } = require('chai')
const data = require('../src/data')
describe('data access', function () {
it('is simple', async function () {
const yo = await data.simple()
expect(yo).to.exist
expect(yo).to.equal(1)
})
})
I've also tried the older style of async mocha tests using done callbacks ala
it('is simple oldschool', function (done) {
data.simple()
.then(function(yo){
expect(yo).to.exist
expect(yo).to.equal(1)
done()
})
})
That times out no matter how long I set Mocha's timeout for (I tried as high as 60 seconds)
I'm at my wits end here, anyone see anything wrong?
api.connect can return before the connection is actually done. Rewriting it like this will make sure ConnectionPool.connect can finish before the api.connect promise resolves.
api.connect = dbConfig =>
new Promise((resolve, reject) => {
const pool = new sql.ConnectionPool(dbConfig);
pool.connect(err => {
if (err) {
console.error("Connection error", err);
return reject(err);
}
return resolve(pool);
});
});
Beyond that, I'm confused about let pool = await c3; there's no symbol c3 in the code you've pasted...
I think you are having a race condition with the database connection.
I do this in the before()
before((done) => {
server.on("serverStarted", function() {
done();
});
});
Then in my server (I'm using node), I emit when when I am connected:
var port = process.env.PORT || 3030;
var server = http.listen(port, function(){
console.log('listening on port: ' + port);
db.connect().then(() => {
console.log("Connect to database successful");
server.emit("serverStarted") // HERE IT IS
}).catch(err => {
console.error(err);
console.log("Cannot connect to database");
process.exit(1);
});
});
Hope this helps. I've pulled out some hair on this one.
I am trying to organise my mocha tests to run through a separate test runner. Whenever I run the test, console.log outputs a proper connection in the top level before block, but straight after outputs me null in the separate required file it block. The hook is being executed, and it sets the connection variable correctly, but somehow it does not get passed to the required file.
How come the connection does not get properly set? Confusingly, according to my debugger it block is executed before the before hook which contradicts the order of console.logs I see
describe.only('test-suite', async () => {
let connection; // undefinded at this point
before(async () => {
connection = await getConnection();
console.log(connection); -> proper connection instance
});
after(async () => {
await closeConnection();
});
require('./some/test')(
connection
);
});
./some/test.js
module.exports = async (
connection,
) => {
describe('my-method', async () => {
it('does things', async () => {
console.log(connection); // somehow undefined
});
});
};
This is because of the way JS handles references to objects. Reassigning the variable does not just change the value the reference points to, but instead creates a brand new reference pointing to a brand new value.
This is the workaround:
describe.only('test-suite', async () => {
let options = { connection: null};
before(async () => {
options.connection = await getConnection();
console.log(connection);
});
after(async () => {
await closeConnection();
});
require('./some/test')(
options
);
});
./some/test.js
module.exports = async (
options,
) => {
describe('my-method', async () => {
it('does things', async () => {
console.log(options.connection);
});
});
};
Could anyone provide a rock-solid, dead-simple unit test for Node.js using WebSockets (Socket.io)?
I'm using socket.io for Node.js, and have looked at socket.io-client for establishing the client connection to a server in the test. However, I seem to be missing something.
In the example below, "worked..." never gets printed out.
var io = require('socket.io-client')
, assert = require('assert')
, expect = require('expect.js');
describe('Suite of unit tests', function() {
describe('First (hopefully useful) test', function() {
var socket = io.connect('http://localhost:3001');
socket.on('connect', function(done) {
console.log('worked...');
done();
});
it('Doing some things with indexOf()', function() {
expect([1, 2, 3].indexOf(5)).to.be.equal(-1);
expect([1, 2, 3].indexOf(0)).to.be.equal(-1);
});
});
});
Instead, I simply get:
Suite of unit tests
First (hopefully useful) test
✓ Doing some things with indexOf()
1 test complete (26 ms)
Any suggestions?
After further poking and prodding, I found some incredibly useful information. In the author's example, he points out the critical step of establishing socket listeners in the before hooks.
This example works:
Assuming a server is listening for socket connections at localhost:3001, of course
var io = require('socket.io-client')
, assert = require('assert')
, expect = require('expect.js');
describe('Suite of unit tests', function() {
var socket;
beforeEach(function(done) {
// Setup
socket = io.connect('http://localhost:3001', {
'reconnection delay' : 0
, 'reopen delay' : 0
, 'force new connection' : true
});
socket.on('connect', function() {
console.log('worked...');
done();
});
socket.on('disconnect', function() {
console.log('disconnected...');
})
});
afterEach(function(done) {
// Cleanup
if(socket.connected) {
console.log('disconnecting...');
socket.disconnect();
} else {
// There will not be a connection unless you have done() in beforeEach, socket.on('connect'...)
console.log('no connection to break...');
}
done();
});
describe('First (hopefully useful) test', function() {
it('Doing some things with indexOf()', function(done) {
expect([1, 2, 3].indexOf(5)).to.be.equal(-1);
expect([1, 2, 3].indexOf(0)).to.be.equal(-1);
done();
});
it('Doing something else with indexOf()', function(done) {
expect([1, 2, 3].indexOf(5)).to.be.equal(-1);
expect([1, 2, 3].indexOf(0)).to.be.equal(-1);
done();
});
});
});
I found that the placement of done() in the beforeEach, socket.on('connect'...) listener was crucial to having the connection get established. For example, if you comment out done() in the listener, then add it one scope out (just before exiting the beforeEach), you'll see the "no connection to break..." message instead of the "disconnecting..." message. Like so:
beforeEach(function(done) {
// Setup
socket = io.connect('http://localhost:3001', {
'reconnection delay' : 0
, 'reopen delay' : 0
, 'force new connection' : true
});
socket.on('connect', function() {
console.log('worked...');
//done();
});
socket.on('disconnect', function() {
console.log('disconnected...');
});
done();
});
I'm new to Mocha, so there's probably a very obvious reason to the initiated for placing done() within the socket scope itself. Hopefully that little detail will save others in my shoes from hair pulling.
For me, the above test (with correct scoping of done()) outputs:
Suite of unit tests
First (hopefully useful) test
◦ Doing some things with indexOf(): worked...
✓ Doing some things with indexOf()
disconnecting...
disconnected...
◦ Doing something else with indexOf(): worked...
✓ Doing something else with indexOf()
disconnecting...
disconnected...
2 tests complete (93 ms)
Offering an extension of the accepted answer here. Has basic client to server communication useful as boilerplate for other future tests. Using mocha, chai, and expect.
var io = require('socket.io-client')
, io_server = require('socket.io').listen(3001);
describe('basic socket.io example', function() {
var socket;
beforeEach(function(done) {
// Setup
socket = io.connect('http://localhost:3001', {
'reconnection delay' : 0
, 'reopen delay' : 0
, 'force new connection' : true
, transports: ['websocket']
});
socket.on('connect', () => {
done();
});
socket.on('disconnect', () => {
// console.log('disconnected...');
});
});
afterEach((done) => {
// Cleanup
if(socket.connected) {
socket.disconnect();
}
io_server.close();
done();
});
it('should communicate', (done) => {
// once connected, emit Hello World
io_server.emit('echo', 'Hello World');
socket.once('echo', (message) => {
// Check that the message matches
expect(message).to.equal('Hello World');
done();
});
io_server.on('connection', (socket) => {
expect(socket).to.not.be.null;
});
});
});
Dealing with callbacks and promises yourself can be difficult and non trivial examples quickly become very complex and hard to read.
There is a tool called socket.io-await-test available via NPM that allows you to suspend/wait in a test until events have been triggered using the await keyword.
describe("wait for tests", () => {
it("resolves when a number of events are received", async () => {
const tester = new SocketTester(client);
const pongs = tester.on('pong');
client.emit('ping', 1);
client.emit('ping', 2);
await pongs.waitForEvents(2) // Blocks until the server emits "pong" twice.
assert.equal(pongs.get(0), 2)
assert.equal(pongs.get(1), 3)
})
})
Check out this boilerplate solution that's based on promises and good practice.
You can test your servers entire io events with it, no sweat.
You just need to copy a boilerplate test and add your own code as needed.
Checkout the repo on GitHub for full source code.
https://github.com/PatMan10/testing_socketIO_server
const io = require("socket.io-client");
const ev = require("../utils/events");
const logger = require("../utils/logger");
// initSocket returns a promise
// success: resolve a new socket object
// fail: reject a error
const initSocket = () => {
return new Promise((resolve, reject) => {
// create socket for communication
const socket = io("localhost:5000", {
"reconnection delay": 0,
"reopen delay": 0,
"force new connection": true
});
// define event handler for sucessfull connection
socket.on(ev.CONNECT, () => {
logger.info("connected");
resolve(socket);
});
// if connection takes longer than 5 seconds throw error
setTimeout(() => {
reject(new Error("Failed to connect wihtin 5 seconds."));
}, 5000);
}
);
};
// destroySocket returns a promise
// success: resolve true
// fail: resolve false
const destroySocket = socket => {
return new Promise((resolve, reject) => {
// check if socket connected
if (socket.connected) {
// disconnect socket
logger.info("disconnecting...");
socket.disconnect();
resolve(true);
} else {
// not connected
logger.info("no connection to break...");
resolve(false);
}
});
};
describe("test suit: Echo & Bello", () => {
test("test: ECHO", async () => {
// create socket for communication
const socketClient = await initSocket();
// create new promise for server response
const serverResponse = new Promise((resolve, reject) => {
// define a handler for the test event
socketClient.on(ev.res_ECHO, data4Client => {
//process data received from server
const { message } = data4Client;
logger.info("Server says: " + message);
// destroy socket after server responds
destroySocket(socketClient);
// return data for testing
resolve(data4Client);
});
// if response takes longer than 5 seconds throw error
setTimeout(() => {
reject(new Error("Failed to get reponse, connection timed out..."));
}, 5000);
});
// define data 4 server
const data4Server = { message: "CLIENT ECHO" };
// emit event with data to server
logger.info("Emitting ECHO event");
socketClient.emit(ev.com_ECHO, data4Server);
// wait for server to respond
const { status, message } = await serverResponse;
// check the response data
expect(status).toBe(200);
expect(message).toBe("SERVER ECHO");
});
test("test BELLO", async () => {
const socketClient = await initSocket();
const serverResponse = new Promise((resolve, reject) => {
socketClient.on(ev.res_BELLO, data4Client => {
const { message } = data4Client;
logger.info("Server says: " + message);
destroySocket(socketClient);
resolve(data4Client);
});
setTimeout(() => {
reject(new Error("Failed to get reponse, connection timed out..."));
}, 5000);
});
const data4Server = { message: "CLIENT BELLO" };
logger.info("Emitting BELLO event");
socketClient.emit(ev.com_BELLO, data4Server);
const { status, message } = await serverResponse;
expect(status).toBe(200);
expect(message).toBe("SERVER BELLO");
});
});
---- Foot Note ----
Depending on how you setup your server environment, you may experience environmental conflict between socket.io and socket.io-client running from the same project simultaneously. In which case it would be better to separate the project into a "test client" and a server. Checkout below repo if you get this issue.
https://github.com/PatMan10/testing_socketIO_server_v2
In OP's code,
socket.on('connect', function(done) {
console.log('worked...');
done();
});
the done was applied to the wrong callback. It should be removed from the socket.on callback and added to Mocha's it block callback:
it('First (hopefully useful) test', function (done) {
var socket = io.connect('http://localhost:3001');
socket.on('connect', function () {
console.log('worked...');
done();
});
});
A complete example
Existing answers are great but don't show the server ultimately being tested. Here's a complete version with console.logs to illustrate what's going on. Explanation follows.
src/server.js:
const express = require("express");
const createServer = (port=3000) => {
const app = express();
const http = require("http").Server(app);
const io = require("socket.io")(http);
io.on("connection", socket => {
console.log("[server] user connected");
socket.on("message", msg => {
console.log(`[server] received '${msg}'`);
socket.emit("message", msg);
});
socket.on("disconnect", () => {
console.log("[server] user disconnected");
});
});
http.listen(port, () =>
console.log(`[server] listening on port ${port}`)
);
return {
close: () => http.close(() =>
console.log("[server] closed")
)
};
};
module.exports = {createServer};
test/server.test.js:
const {expect} = require("chai");
const io = require("socket.io-client");
const {createServer} = require("../src/server");
const socketUrl = "http://localhost:3000";
describe("server", function () {
this.timeout(3000);
let server;
let sockets;
beforeEach(() => {
sockets = [];
server = createServer();
});
afterEach(() => {
sockets.forEach(e => e.disconnect())
server.close();
});
const makeSocket = (id=0) => {
const socket = io.connect(socketUrl, {
"reconnection delay": 0,
"reopen delay": 0,
"force new connection": true,
transports: ["websocket"],
});
socket.on("connect", () => {
console.log(`[client ${id}] connected`);
});
socket.on("disconnect", () => {
console.log(`[client ${id}] disconnected`);
});
sockets.push(socket);
return socket;
};
it("should echo a message to a client", done => {
const socket = makeSocket();
socket.emit("message", "hello world");
socket.on("message", msg => {
console.log(`[client] received '${msg}'`);
expect(msg).to.equal("hello world");
done();
});
});
it("should echo messages to multiple clients", () => {
const sockets = [...Array(5)].map((_, i) => makeSocket(i));
return Promise.all(sockets.map((socket, id) =>
new Promise((resolve, reject) => {
const msgs = [..."abcd"].map(e => e + id);
msgs.slice().forEach(e => socket.emit("message", e));
socket.on("message", msg => {
console.log(`[client ${id}] received '${msg}'`);
expect(msg).to.equal(msgs.shift());
if (msgs.length === 0) {
resolve();
}
});
})
));
});
});
In summary, the server exports a function that lets a server app be created from scratch, allowing each it block to be idempotent and avoid server state from carrying between tests (assuming no persistence on the server otherwise). Creating an app returns an object with a close function. socket.disconnect() must be called per socket in each test to avoid timeouts.
Given these requirements, the testing suite follows this per-test setup/teardown workflow:
let server;
let sockets;
beforeEach(() => {
sockets = [];
server = createServer();
});
afterEach(() => {
sockets.forEach(e => e.disconnect())
server.close();
});
makeSocket is an optional helper to reduce the repeated boilerplate of connecting and disconnecting a socket client. It does produce a side effect on the sockets array for cleanup later, but this is an implementation detail from the it block's perspective. Test blocks shoudn't touch server or sockets variables, although other workflows are likely depending on need. The critical takeaways are test case idempotency and closing all connections after each test case.
Options on the socket.connect object on the client let you choose transport and behavior of the socket. "force new connection": true creates a new Manager per socket instead of reusing an existing one and transports: ["websocket"] upgrades to WS protocol from long polling immediately.
Use it("should ... ", done => { /* tests */ }); and invoke done() after all work is completed in callbacks or return a promise (and omit the done parameter to the it callback). The example above shows both approaches.
Used in this post:
node: 12.19.0
chai: 4.2.0
express: 4.16.4
mocha: 5.2.0
socket.io: 2.2.0
socket.io-client: 2.2.0
I had this problem: How to do unit test with a "socket.io-client" if you don't know how long the server take to respond?.
I've solved so using mocha and chai:
var os = require('os');
var should = require("chai").should();
var socketio_client = require('socket.io-client');
var end_point = 'http://' + os.hostname() + ':8081';
var opts = {forceNew: true};
describe("async test with socket.io", function () {
this.timeout(10000);
it('Response should be an object', function (done) {
setTimeout(function () {
var socket_client = socketio_client(end_point, opts);
socket_client.emit('event', 'ABCDEF');
socket_client.on('event response', function (data) {
data.should.be.an('object');
socket_client.disconnect();
done();
});
socket_client.on('event response error', function (data) {
console.error(data);
socket_client.disconnect();
done();
});
}, 4000);
});
});