How to prevent repeated responses from Node.js server - node.js

We're running into a problem where we're getting multiple responses sent from our Node server to a web client which are connected by a socket server (socket.io). By listening with Docklight, I can see that we're really only getting a single response from the serial device, but for some reason the Node server is sending multiples, and they accumulate, so the first time you send a serial command (and it doesn't matter what commands) might only get a couple, next time a couple more, next time a couple more and so on. So if you run several serial commands, you'll get back lots of multiple responses.
Our environment is Windows 7 64 bit, Node V 4.5.0, serialport V 4.0.1. However, this needs to run on Windows, Mac & Linux when we're done. The dev team (me & one other guy) are both fairly new to Node, but otherwise capable developers.
I think what's happening is I'm not using the .flush() & .drain() functions properly and the serialport buffer still contains serial data. Our proprietary devices return either S>, or <Executed/> prompts when a command has completed, so I store the serial response in a buffer until I see one or the other, then process the data (in this example just providing a boolean response whether the device is responding with one or the other or not). For example, if I send a <CR><LF> to one of our devices, it should respond with S> (or <Executed/> depending).
The client calls into the server with this:
socket.on('getDeviceConnected', readDeviceResponse);
function readDeviceResponse(isDeviceResponding) {
console.log('getDeviceConnected');
console.log(isDeviceResponding);
}
function getDeviceConnected() {
console.log("Sending carriage return / line feed.");
socket.emit('getDeviceConnected', '\r\n');
}
And on the server, here's what I'm trying:
socket.on('getDeviceConnected', function (connectionData) {
//write over serial buffer before the write occurs to prevent command accumulation in the buffer.
serialBuffer = '';
sbeSerialPort.write(connectionData, function (err, results) {
//since there's no way to tell if the serial device hasn't responded, set a time out to return a false after allowing one second to elapse
setTimeout(function () {
console.log('Inside getDeviceConnected setTimeout');
console.log('Is serial device responding:', isSerialDeviceResponding);
if (!isSerialDeviceResponding) {
console.log('Serial device timed out.');
socket.emit('getDeviceConnected', false);
}
}, 1000);
if (err) {
console.log('Serial port error level:', err);
}
if (results) {
if (results === 2) {
console.log('Serial port is responding');
}
}
});
sbeSerialPort.on('data', function (serialData) {
isSerialDeviceResponding = true;
console.log('Does S> prompt exist?', serialData.lastIndexOf('S>'));
while(!serialData.lastIndexOf('S>') > -1 || !serialData.lastIndexOf('<Executed/>') > -1){
serialBuffer += serialData;
break;
}
if (isSerialDeviceResponding) {
socket.emit('getDeviceConnected', true);
isSerialDeviceResponding = true;
}
sbeSerialPort.flush(function (err, results) {
if (err) {
console.log(err);
return;
}
if(results){
console.log('Serial port flush return code:', results);
}
});
});
I'm not very sure about the .flush() implementation here, and I've omitted the .drain() part because neither of them seems to do much of anything (assuming they were correctly implemented).
How do I insure that there is no data left in the serialport buffer when the .write() command is complete? Or do you see other problems with how I'm handling the serial data?
Edit, Source code up on pastebin.com:
Server.js
Client.js
HTML

Related

Nodejs serial port to tcp

Is there a way to establish a host to stream a serial connection over tcp using nodejs - I want to stream the sensor data from the iot device on my computer to a connected computer to a web server. streaming of the raw data is fine - the remote computer will process it. I was looking into net and serialport npm packages - but im unsure of how to marry the two...
Thanks!
Preparation
Pretty much each vendor or device has its own serial communication protocol. Usually these devices also use packets with headers, checksums, but each device does this in a different way.
The first question is really, to what extend you want to forward the packet headers and checksum information. You may want to translate incoming packets to events or perhaps already to some kind of JSON message.
Assuming that you just want to forward the data in raw format without any pre-processing, it is still valuable to determine where a packet starts and ends. When you flush data over TCP/IP it's best not to do so halfway one of those serial packets.
For instance, it could be that your device is a barcode scanner. Most barcode scanners send a CR (carriage return) at the end of each scan. It would make sense to actively read incoming bytes looking for a CR character. Then each time a CR character is noticed you flush your buffer of bytes.
But well, it isn't always a CR. Some devices package their data between STX (0x02) and ETX (0x03) characters. And there are some that send fixed-length packages (e.g. 12 bytes per message).
Just for clarity, you could end up sending your data every 100 bytes while a message is actually 12 bytes. That would break some of the packet. Once in a while your TCP receiver would receive an incomplete packet. Having said all that. You could also add all this logic on the TCP receiving side. When an incomplete packet is received, you could keep it in a buffer in the assumption that the next incoming packet will contain the missing bytes.
Consider if it's worth it
Note that there are commercial RS232-to-ethernet devices that you can buy of the shelf and configure (~100EUR) that do exactly what you want. And often in the setup of that kind device you would have the option to configure a flush-character. (e.g. that CR). MOXA is probably the best you can get. ADAM also makes decent devices. These vendors have been making this kind of devices for ~30 years.
To get you started
But for the sake of exercise, here we go.
First of all, you would need something to communicate with your serial device.
I used this one:
npm install serialport#^9.1.0
You can pretty much blindly copy the following code. But obviously you need to set your own RS232 or USB port settings. Look in the manual of your device to determine the baudrate, databits, stopbits, parity and optionally RTS/DTR
import SerialPort from "serialport";
export class RS232Port {
private port: SerialPort;
constructor(private listener: (buffer: Buffer) => any, private protocol) {
this.port = new SerialPort("/dev/ttyS0", {
baudRate: 38400,
dataBits: 8,
stopBits: 1,
parity: "none",
});
// check your RTS/DTR settings.
// this.port.on('open', () => {
// this.port.set({rts: true, dtr: false}, () => {
// });
//});
const parser = this.port.pipe(this.protocol);
parser.on('data', (data) => {
console.log(`received packet:[${toHexString(data)}]`);
if (this.listener) {
this.listener(data);
}
});
}
sendBytes(buffer: Buffer) {
console.log(`write packet:[${toHexString(buffer)}]`);
this.port.write(buffer);
}
}
The code above continuously reads data from a serial device, and uses a "protocol" to determine where messages start/end. And it has a "listener", which is a callback. It can also send bytes with its sendBytes function.
That brings us to the protocol, which as explained earlier is something that should read until a separator is found.
Because I have no clue what your separator is. I will present you with an alternative, which just waits for a silence. It assumes that when there is no incoming data for a certain time, that the message will be complete.
export class TimeoutProtocol extends Transform {
maxBufferSize: number;
currentPacket: [];
interval: number;
intervalID: any;
constructor(options: { interval: number, maxBufferSize: number }) {
super()
const _options = { maxBufferSize: 65536, ...options }
if (!_options.interval) {
throw new TypeError('"interval" is required')
}
if (typeof _options.interval !== 'number' || Number.isNaN(_options.interval)) {
throw new TypeError('"interval" is not a number')
}
if (_options.interval < 1) {
throw new TypeError('"interval" is not greater than 0')
}
if (typeof _options.maxBufferSize !== 'number' || Number.isNaN(_options.maxBufferSize)) {
throw new TypeError('"maxBufferSize" is not a number')
}
if (_options.maxBufferSize < 1) {
throw new TypeError('"maxBufferSize" is not greater than 0')
}
this.maxBufferSize = _options.maxBufferSize
this.currentPacket = []
this.interval = _options.interval
this.intervalID = -1
}
_transform(chunk: [], encoding, cb) {
clearTimeout(this.intervalID)
for (let offset = 0; offset < chunk.length; offset++) {
this.currentPacket.push(chunk[offset])
if (this.currentPacket.length >= this.maxBufferSize) {
this.emitPacket()
}
}
this.intervalID = setTimeout(this.emitPacket.bind(this), this.interval)
cb()
}
emitPacket() {
clearTimeout(this.intervalID)
if (this.currentPacket.length > 0) {
this.push(Buffer.from(this.currentPacket))
}
this.currentPacket = []
}
_flush(cb) {
this.emitPacket()
cb()
}
}
Then finally the last piece of the puzzle is a TCP/IP connection. Here you have to determine which end is the client and which end is the server. I skipped that for now, because there are plenty of tutorials and code samples that show you how to set up a TCP/IP client-server connection.
In some of the code above I use a function toHexString(Buffer) to convert the content of a buffer to a hex format which makes it easier to print it to the console log.
export function toHexString(byteArray: Buffer) {
let s = '0x';
byteArray.forEach(function (byte) {
s += ('0' + (byte & 0xFF).toString(16)).slice(-2);
});
return s;
}

node.js and bluetooth barcode-scanner

I am struggling quite a while now to get a solid, long-term connection to a bluetooth barcode scanner from Inateck using node.js. The process is running in the background (linux, no input-focus) that's why I configured the scanner as a SPP device.
The connection is basically working as long as the scanner doesn't automatically switch off to save power, which is after about 5 minutes.
My first approach was to use the bluetooth-serial-port package. It discovers the scanner, reads the barcodes but when the scanner switches off, I don't know how to re-connect. I added an interval timer to check the connection and try to connect again if isOpen() returns false (which works once). When I press the button on the scanner it switches back on and I can re-connect but after a view seconds isOpen() returns false even if the connection is established, and I don't get any further readings. Here is my code:
var btSerial = new (require('bluetooth-serial-port')).BluetoothSerialPort();
var btInterval = null;
btSerial.on('found', function (address, name) {
btSerial.findSerialPortChannel(address, function (channel) {
if (address === '00:06:11:68:15:81') {
btSerial.connect(address, channel, function () {
console.log('connected to ' + address);
btInterval = setInterval(function () {
if (!btSerial.isOpen()) {
btSerial.close();
clearInterval(btInterval);
console.log('lost connection - try to reconnect');
btSerial.inquire();
}
}, 5000);
}, function () {
console.log('cannot connect to ' + address);
});
}
}, function () {
console.log('found nothing');
});
});
btSerial.on('finished', function () {
console.log('finished');
});
btSerial.on('data', function (buffer) {
console.log(buffer.toString('utf-8'));
});
btSerial.inquire();
The output looks like this:
finished
connected to 00:06:11:68:15:81
found nothing
... scanning works ...
lost connection - try to reconnect
finished
connected to 00:06:11:68:15:81
... scanning works ...
lost connection - try to reconnect
finished
... that's it - no more scans ...
^C
An other idea was to use nodes fs() an read directly from '/dev/rfcomm0'.
scanner = fs.createReadStream('/dev/rfcomm0', {bufferSize: 1});
scanner.on('open', function () {
logger.info('Scanner connected');
});
scanner.on('end', function () {
logger.info('End of data stream');
});
scanner.on('close', function () {
logger.info('Scanner disconnected');
});
scanner.on('error', function (error) {
logger.error('Scanner error');
});
scanner.on('data', function (chunk) {
logger.info(chunk.toString('ascii', 0, 13));
}
});
});
Connecting is done by the OS automatically when reading from the device and I do receive the codes via on('data',..). But I do have the same problem when the scanner switches off after a while. I do receive the on('close',..) event but reconnecting using fs.createReadStream() again doesn't work any more.
Maybe someone of you already had to deal which such a problem and can give me a hint how to handle this. I appreciate every suggestion.
Thanks,
Max
That's not the way I wanted to go for but a bash script to launch my node app when the scanner is available, does the job:
#!/bin/bash
echo "Press CTRL+C to stop..."
while :
do
if hcitool scan | grep -q "00:06:11:68:15:81"; then
# BT scanner found
node .
fi
sleep 1
done

How to handle CONTROL+C in node.js TCP server

how do I handle the CONTROL+C input in a node.js TCP server?
var server = net.createServer(function(c) {
c.on('end', function() {
console.log('Client disconnected');
});
c.on('data', function(data) {
if (data == "CONTROL+C") { // Here is the check
c.destroy();
}
});
}).listen(8124);
Control-C is a single byte, 0x03 (using an ASCII chart is kinda helpful).
However, whenever you're dealing with a socket connection you have to remember that you're going to receive data in a "chunked" fashion and the chunking does not necessarily correspond to the way the data was sent; you cannot assume that one send call on the client side corresponds to a single chunk on the server side. Therefore you can't assume that if the client sends a Control-C, it will be the only thing you receive in your data event. Some other data might come before it, and some other data might come after it, all in the same event. You will have to look for it inside your data.
From ebohlman's answer. It work.
c.on('data', function(data) {
if (data.toString().charCodeAt(0) === 3) {
c.destroy();
}
});

How to process a net.Stream using node.js?

I am trying to learn about streams in node.js!
server.js
var net = require("net");
var server = net.createServer(function(conn) {
conn.write("welcome!");
# echo the user input!
conn.pipe(conn);
});
server.listen("1111", function() {
console.log("port 1111 opened");
});
telnet test
The server currently echos the user's input
$ telnet localhost 1111
welcome!
hello
hello
desired output
To demonstrate where/how I should process the stream on the server side, I would like to wrap the user's input in {} before echoing it back
$ telnet localhost 1111
welcome!
hello
{hello}
This will basically accomplish the exact output you've requested:
var net = require('net');
var server = net.createServer(function(c) {
c.setEncoding('utf8');
c.on('data', function(d) {
c.write('{' + d.trim() + '}\n');
});
});
server.listen(9871);
First let me call your attention to c.setEncoding('utf8'). This will set a flag on the connection that will automatically convert the incoming Buffer to a String in the utf8 space. This works well for your example, but just note that for improved performance between Sockets it would be better to perform Buffer manipulations.
Simulating the entirety of .pipe() will take a bit more code.
.pipe() is a method of the Stream prototype, which can be found in lib/stream.js. If you take a look at the file you'll see quite a bit more code than what I've shown above. For demonstration, here's an excerpt:
function ondata(chunk) {
if (dest.writable) {
if (false === dest.write(chunk) && source.pause) {
source.pause();
}
}
}
source.on('data', ondata);
First a check is made if the destination is writable. If not, then there is no reason to attempt writing the data. Next the check if dest.write === false. From the documentation:
[.write] returns true if the entire data was flushed successfully to the kernel buffer. Returns false if all or part of the data was queued in user memory.
Since Streams live in kernel space, outside of the v8 memory space, it is possible to crash your machine by filling up memory (instead of just crashing the node app). So checking if the message has drained is a safety prevention mechanism. If it hasn't finished draining, then the source will be paused until the drain event is emitted. Here is the drain event:
function ondrain() {
if (source.readable && source.resume) {
source.resume();
}
}
dest.on('drain', ondrain);
Now there is a lot more we could cover with how .pipe() handles errors, cleans up its own event emitters, etc. but I think we've covered the basics.
Note: When sending a large string, it is possible that it will be sent in multiple packets. For this reason it may be necessary to do something like the following:
var net = require('net');
var server = net.createServer(function(c) {
var tmp = '';
c.setEncoding('utf8');
c.on('data', function(d) {
if (d.charCodeAt(d.length - 1) !== 10) {
tmp += d;
} else {
c.write('{' + tmp + d.trim() + '}\n');
tmp = '';
}
});
});
server.listen(9871);
Here we use the assumption that the string is ended by the new line character (\n, or ascii character code 10). We check the end of the message to see if this is the case. If not, then we temporarily store the message from the connection until the new line character is received.
This may not be a problem for your application, but thought it would be worth noting.
you can do something like
conn.on 'data', (d) ->
conn.write "{#{d}}"
the .pipe method is basically just attaching the data event of the input stream to write to the output stream
I'm not sure about net() actually, but I imagine it's quite similar to http:
http.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'text/event-stream'});
http.get(options, function(resp){
resp.on('data', function(chunk){
res.write("event: meetup\n");
res.write("data: "+chunk.toString()+"\n\n");
});
}).on("error", function(e){
console.log("Got error: " + e.message);
});
});
https://github.com/chovy/nodejs-stream

Node.js reply tcp server

I am trying to create a simple reply server in node.js
The problem I am having, is that when I telnet into the server, and send a hello, the if loop doesn't catch it, and it goes to the else.
Below is my code:
var net = require('net');
var server = net.createServer(function(socket) {
// Server start
socket.write('Welcome\n');
socket.on('data', function(data) {
dataReceived(socket, data);
});
});
server.listen(8250);
function dataReceived(socket, data) {
if(data == 'hello') {
socket.end('Hi');
} else {
socket.write(data);
socket.end('what??\n');
}
}
Thanks.
Data is a binary buffer, not a string. See http://nodejs.org/docs/v0.4.9/api/buffers.html.
Use the buffer.toString method to convert to a string.
Also, a new line will be added when hitting enter in telnet. Not sure if line endings vary by os, but in this case I'm stripping \r\n.
function dataReceived(socket, data) {
data = data.toString('utf8').replace(/\r\n/, '');
if(data == 'hello') {
socket.end('Hi');
} else {
socket.write(data);
socket.end('what??\n');
}
}
As mentioned, main problem is that you compare Buffer object with string.
There is another problem, most probably not visible in your example.
You don't have control how data is split into packets. 'Hello' sent to your server may result dataReceived called with 'Hel' + 'l' + 'o' buffer 3 times
Correct way to handle 'Hello' input us to create state machine or, more simple and less efficient - buffer all incoming data, look for 'Hello' at the beginning of buffered data, then cut handled data from buffer. There are modules aiming to help to unpack/unframe structured data from input stream, for example node-binary

Resources