I have my code that reads and writes to a serial port written in MFC. The programs works well but is a bit slow as there are many operations occuring (Read and writing). I have a timer that carries on the operations on the serial port. The timer is given below:
Loop_Timer = SetTimer(1,50,0);
The serial port transmission information is as follows:
BaudRate = 57600;
ByteSize = 8;
Parity = NOPARITY;
StopBits = ONESTOPBIT;
fAbortOnError = false;
The following write and read operation occurs when the timer starts:
Write(command);
Read(returned_message);
returned_message.Trim();
...
//finds a value from the returned string
...
So, this read and write operation occurs may be 1,2,3 or 4 times for a given selected option.
For Ex: Option 1 requires the above function to occurs 4 times in the given timer.
Option 2 requires the above function to occur 2 times. (as it has only two variables with return values). etc
...
Now, what I was trying to do is improving the speed of this overall operation making it robust and respond quickly. I tried changing the timer but it is still pretty slow. Any suggestions on improvement?
You'd do far better to run your actual serial port processing in a separate thread, and to use the WaitCommEvent rather than a timer for accepting incoming data. Append newly received data within a storage buffer local to that thread.
Retrieve data from your serial port thtread using a timer if you wish, or have your serial port thread communicate to your main app. when a complete message is received.
When sending data to the serial port thread you want a mechanism, whereby the data is stored locally to the serial port code and transmitted form there.
The thing to bear in mind is that compared to all other means of communications serial port transmission and reception is SLOW and by accessing the serial port on your main application thread you'll slow it down massively, especially when transmitting data.
If you find direct coding using the Win32 API and serial ports a pain then this class here I've found very useful.
Related
I have upto 30 nodes each capable of sending data upto 1000 messages/second. Each messages can have 256-512 bytes of data.
Each node use unique tcp port for communication. Each data received is pre-processed, inserted into database and post-processed.
Below are the approaches I have tried with observations :-
Case-1. using asyncio with processing data as soon as received.
async def process_packets(reader, writer, db):
while True:
data = reader.read(4096)
data = pre_process(data)
save_in_db(data)
post_process(data)
writer.close()
Observation:- For single packet, processing usually takes 10-20msec. But as packet frequency increases, tcp buffering starts to happen i.e single call to reader.read(), gets multiple packets.
This increases the processing of the current node as well as other nodes.
Case-2. using asyncio with data pushed in a queue and worker thread consuming this queue.
async def process_packets(reader, writer, q):
while True:
data = reader.read(4096)
q.put(data)
writer.close()
def worker_thread(q, db):
while True:
data = q.get()
data = pre_process(data)
save_in_db(data)
post_process(data)
Observation:- As no processing is done while receiving packets, all nodes are able put data in the queue as fast as possible. The issue comes in the worker thread where q.get() becomes very slow as time progress.
Case-3. creating socket server thread for each node
def server_thread(port, db):
s = socket.socket()
s.bind()
s.listen(1)
while True:
(conn, addr) = s.accept()
while True:
try:
data = conn.recv(4096)
except Exception:
conn.close()
break
data = pre_process(data)
save_in_db(data)
post_process(data)
Observation:- The advantage is this case is that each node has dedicated thread for receiving and processing data so other threads are not affected. But here I am facing multiple packets returned by socket.recv(). This increases processing time.
I need a way to process data from these node as fast as possible with application running 24x7 with no downtime.
OS = Ubuntu20.04-lts
System = intel i3 8th gen, 8GB ram, 4core
But here I am facing multiple packets returned by socket.recv()
TCP is a byte stream, i.e. there are no packets at this level. You likely mean application level messages. Your code MUST be able to deal with multiple or partial application messages itself since TCP does not provide a message syntax by itself. While you seem to get only full messages when reading fast enough there is no guarantee for this, eventually your application might stall for a short time (due to scheduling) and messages will accumulate.
Dealing with multiple messages returned from a socket.recv() can even be an advantage. Reading multiple messages at once means that a single system call returns more application data, which increases the efficiency of the application (less system calls needed for the same amount of work). So it is better to read as much as possible within a single recv instead of hoping to get only a single message.
As for the other design: the last approach with a thread per node scales best, since in this case the work (and this the load) is spread over multiple CPU cores. The other approaches only use a single CPU core. But none of the approaches actually guarantees that your specific system is able to process that much data. They only differ in how good they make use of the resources offered by the underlying system.
I have three bits of networked realtime data logging equipment that output lines of ASCII text via TCP sockets. They essentially just broadcast the data that they are logging - there are no requests for data from other machines on the network. Each piece of equipment is at a different location on my network and each has a unique IP address.
I'd like to combine these three streams into one so that I can log it to a file for replay or forward it onto another device to view in realtime.
At the moment I have a PHP script looping over each IP/port combination listening for up to 64Kb of data. As soon as the data is received or it gets an EOL then it forwards that on to another which that listens to the combined stream.
This works reasonably well but one of the data loggers outputs far more data than the others and tends to swamp the other machines so I'm pretty sure that I'm missing data. Presumably because it's not listening in parallel.
I've also tried three separate PHP processes writing to a shared file in memory (on /dev/shm) which is read and written out by a fourth process. Using file locking this seems to work but introduces a delay of a few seconds which I'd rather avoid.
I did find a PHP library that allows true multithreading using Pthreads called (I think) Amp but I'm still not sure how to combine the output. A file in RAM doesn't seem quick enough.
I've had a good look around on Google and can't see an obvious solution. There certainly doesn't seem to be a way to do this on Linux using command line tools that I've found unless I've missed something obvious.
I'm not too familiar with other languages but are there other languages that might be better suited to this problem ?
Based on the suggested solution below I've got the following code almost working however I get an error 'socket_read(): unable to read from socket [107]: Transport endpoint is not connected'. This is odd as I've set the socket to accept connections and made it non-blocking. What am I doing wrong ?:
// Script to mix inputs from multiple sockets
// Run forever
set_time_limit (0);
// Define address and ports that we will listen on
$localAddress='';
// Define inbound ports
$inPort1=36000;
$inPort2=36001;
// Create sockets for inbound data
$inSocket1=createSocket($localAddress, $inPort1);
$inSocket2=createSocket($localAddress, $inPort2);
// Define buffer of data to read and write
$buffer="";
// Repeat forever
while (true) {
// Build array streams to monitor
$readSockets=array($inSocket1, $inSocket2);
$writeSockets=NULL;
$exceptions=NULL;
$t=NULL;
// Count number of stream that have been modified
$modifiedCount=socket_select($readSockets, $writeSockets, $exceptions, $t);
if ($modifiedCount>0) {
// Process inbound arrays first
foreach ($readSockets as $socket) {
// Get up to 64 Kb from this socket
$buffer.=socket_read($socket, 65536, PHP_BINARY_READ);
}
// Process outbound socket array
foreach ($writeSockets as $socket) {
// Get up to 64 Kb from this socket and add it to any other data that we need to write out
//socket_write($socket, $buffer, strlen($buffer));
echo $buffer;
}
// Reset buffer
$buffer="";
} else {
echo ("Nothing to read\r\n");
}
}
function createSocket($address, $port) {
// Function to create and listen on a socket
// Create socket
$socket=socket_create(AF_INET, SOCK_STREAM, 0);
echo ("SOCKET_CREATE: " . socket_strerror(socket_last_error($socket)) . "\r\n");
// Allow the socket to be reused otherwise we'll get errors
socket_set_option($socket, SOL_SOCKET, SO_REUSEADDR, 1);
echo ("SOCKET_OPTION: " . socket_strerror(socket_last_error($socket)) . "\r\n");
// Bind it to the address and port that we will listen on
$bind=socket_bind($socket, $address, $port);
echo ("SOCKET_BIND: " . socket_strerror(socket_last_error($socket)) . " $address:$port\r\n");
// Tell socket to listen for connections
socket_listen($socket);
echo ("SOCKET_LISTEN: " . socket_strerror(socket_last_error($socket)) . "\r\n");
// Make this socket non-blocking
socket_set_nonblock($socket);
// Accept inbound connections on this socket
socket_accept($socket);
return $socket;
}
You don't necessary need to switch languages, it just sounds like you're not familiar with the concept of IO multiplexing. Check out some documentation for the PHP select call here
The concept of listening to multiple data inputs and not knowing which one some data will come from next is a common one and has standard solutions. There are variations on exactly how its implemented but the basic idea is the same: you tell the system that you're interested in receiving data from multiple source simultaneously (TCP sockets in your case), and run a loop waiting for this data. On every iteration of the loop the system the system tells you which source is ready for reading. In your case that means you can piecemeal-read from all 3 of your sources without waiting for an individual one to reach 64KB before moving on to the next.
This can be done in lots of languages, including PHP.
UPDATE: Looking at the code you posted in your update, the issue that remains is that you're trying to read from the wrong thing, namely from the listening socket rather than the connection socket. You are ignoring the return value of socket_accept in your createSocket function which is wrong.
Remove these lines from createSocket:
// Accept inbound connections on this socket
socket_accept($socket);
Change your global socket creation code to:
// Create sockets for inbound data
$listenSocket1=createSocket($localAddress, $inPort1);
$listenSocket2=createSocket($localAddress, $inPort2);
$inSocket1=socket_accept($listenSocket1);
$inSocket2=socket_accept($listenSocket2);
Then your code should work.
Explanation: when you create a socket for binding and listening, its sole function then becomes to accept incoming connections and it cannot be read from or written to. When you accept a connection a new socket is created, and this is the socket that represents the connection and can be read/written. The listening socket in the meantime continues listening and can potentially accept other connections (this is why a single server running on one http port can accept multiple client connections).
Before everybody marks this as a dup let me state that I know my fair share of network programming and this question is my attempt to solve something that riddles me even after finding the "solution".
The setup
I've spend the last weeks writing some glue code to incorporate a big industrial system into our current setup. The system is controlled by a Windows XP computer (PC A) which is controlled from a Ubuntu 14.04 system (PC B) by sending a steady stream of UDP packets at 2000 Hz. It responds with UDP packets containing the current state of the system.
Care was taken to ensure that the the 2000 Hz rate was held because there is a 3ms timeout after which the system faults and returns to a safe state. This involves measuring and accounting for inaccuracies in std::this_thread::sleep_for. Measurements show that there is only a 0.1% derivation from the target rate.
The observation
Problems started when I started to receive the state response from the system. The controlling side on PC B looks roughly like this:
forever at 2000Hz {
send current command;
if ( socket.available() >= 0 ) {
receive response;
}
}
edit 2: Or in real code:
auto cmd_buf = ...
auto rsp_buf = ...
while (true) {
// prepare and send command buffer
cmd_buf = ...
socket.send(cmd_buf, endpoint);
if (socket.available() >= 0) {
socket.receive(rsp_buf);
// the results are then parsed and stored, nothing fancy
}
// time keeping
}
Problem is that, whenever the receiving portion of the code was present on PC B, PC A started to run out of memory within seconds when trying to allocate receive buffers. Additionally it raised errors stating that the timeout was missed, which was probably due to packets not reaching the control software.
Just to highlight the strangeness: PC A is the pc sending UDP packets in this case.
Edit in response to EJP: this is the (now) working setup. It started out as:
forever at 2000Hz {
send current command;
receive response;
}
But by the time the response was received (blocking) the deadline was missed. Therefore the availability check.
Another thing that was tried was to receive in a seperate thread:
// thread A
forever at 2000Hz {
send current command;
}
// thread B
forever {
receive response;
}
Which displays the same behavior as the first version.
The solution
The solution was to set the socket on PC B to non blocking mode. One line and all problems were gone.
I am pretty sure that even in blocking mode the deadline was met. There should be no performance difference between blocking and non-blocking mode when there is just one socket involved. Even if checking the socket for available data takes some microseconds more than in non-blocking mode it shouldn't make a difference when the overall deadline is met accuratly.
Now ... what is happening here?
If I read your code correctly and referring to this code:
forever at 2000Hz {
send current command;
receive response;
}
Examine the difference between the blocking and not blocking socket. With blocking socket you send current command and then you are stuck waiting for the response. By this time I would guess you already miss the 2kHz goal.
Now in non blocking socket you send the current command, try to received whatever is in receive buffers, but if there is nothing there you return immediately and continue your tight 2kHz loop of sending. This explains to me why your industrial control system works fine in non-blocking code.
I am benchmarking a Java UDP client that continuously sends datagrams with a payload of 100 bytes as fast as it can. It was implemented using java.nio.*. Tests show that it's able to achieve a steady throughput of 220k datagrams per second. I am not testing with a server; the client just sends the datagrams to some unused port on localhost.
I decided to run the same test in Node.js to compare both technologies and it was surprisingly sad to see that Node.js performed 10 times slower than Java. Let me walk you through my code.
First, I create a UDP socket using Node.js's dgram module:
var client = require('dgram').createSocket("udp4");
Then I create a function that sends a datagram using that socket:
function sendOne() {
client.send(message, 0, message.length, SERVER_PORT, SERVER_ADDRESS, onSend);
}
The variable message is a buffer created from a string with a hundred characters when the application starts:
var message = new Buffer(/* string with 100 chars */);
The function onSend just increments a variable that holds how many datagrams were sent so far. Next I have a function that constantly calls sendOne() using setImmediate():
function sendForever() {
sendOne();
setImmediate(sendForever);
}
Initially I tried to use process.nextTick(sendForever) but I found out that it always puts itself at the tip of the event queue, even before IO events, as the docs says:
It runs before any additional I/O events (including timers) fire in subsequent ticks of the event loop.
This prevents the send IO events from ever happening, as nextTick is constantly putting sendForever at the tip of the queue at every tick. The queue grows with unread IO events until it makes Node.js crash:
fish: Job 1, 'node client' terminated by signal SIGSEGV (Address boundary error)
On the other hand, setImmediate fires after I/O events callbacks, so that's why I'm using it.
I also create a timer that once every 1 second prints to the console how many datagrams were sent in the last second:
setInterval(printStats, 1000);
And finally I start sending:
sendForever();
Running on the same machine as the Java tests ran, Node.js achieved a steady throughput of 21k datagrams per second, ten times slower than Java.
My first guess was to put two sendOne's for every tick to see if it would double the throughput:
function sendForever() {
send();
send(); // second send
setImmediate(sendForever);
}
But it didn't change the throughput whatsoever.
I have a repository available on GitHub with the complete code:
https://github.com/luciopaiva/udp-perf-js
Simply clone it to your machine, cd into the folder and run:
node client
I want to open a discussion about how this test could be improved in Node.js and if there's some way we can increase Node.js's throughput. Any ideas?
P.S.: for those interested, here is the Java part.
That test is overfly flawed. UDP doesn't guarantee the delivery of anything and doesn't guarantee that it would give any error in case of error.
Your application could send 1000k datagram/s at 1GB/s from the Java application, yet 90% of datagrams never reached the destination... the destination might not even be running.
If you want to do any sort of UDP testing, you need two applications, one on each end. Send numbered datagrams 1, 2, 3... and check what's sent and what's received. Note that UDP doesn't guarantee any ordering of messages.
Kernels manage the localhost network in special ways. There are huge buffers dedicated to it and higher limits, no traffic ever goes through any network cards or drivers. It's very different from sending packets for real.
Tests might seem somewhat okay when they're only done on localhost. Expect everything to fail miserably when it's going through any physical infrastructure for real.
PC1 <-----> switch <-----> PC2
Let's say, there are two computers in the same room linked by a switch. It would be no small feat to achieve 10k/s UDP datagrams on that simple setup, without loosing messages randomly.
And that's just two computers in the same room. It can be a lot worse on the Internet and long distance.
If all you want is to make the performance test go faster, removing the setImmediate call and executing the next send once the first has completed i.e. in the send callback increased its performance to ~100k requests per second on my slowish laptop.
function send(socket, message) {
socket.send(message, SERVER_PORT, (err) => {
send(socket, message);
});
}
const socket = require('dgram').createSocket('udp4');
const message = new Buffer('dsdsddsdsdsjkdshfsdkjfhdskjfhdskjfhdsfkjsdhfdskjfhdskjfhsdfkjdshfkjdshfkjdsfhdskjfhdskjfhdkj');
send(socket, message);
Let's say client sent 100 bytes of data but somehow server only received 90 bytes. How do I handle this case? If server calls the "read" function inside of while loop checking the total received data then the server will wait forever for the pack last 10 bytes..
Also, it could happen that client got disconnected in the middle of data transfer. In this case also server will wait forever until it receives all the data which won't arrive..
I am using tcp but in real world network environment, this situation could happen. Thanks in advance...
You do not call the read() function in a loop until you receieve the number of bytes you require. Instead, you set the socket to nonblocking and call the read() function in a loop until it returns 0 (indicating end of stream) or an error.
In the normal case the loop will terminate by read() returning -1, with errno set to EAGAIN. This indicates that the connection hasn't been closed, but no more data is available at the current time. At this point, if you do not have enough data from the client yet, you simply save the data that you do have for later, and return to the main epoll() loop.
If and when the remainder of the data arrives, the socket will be returned as readable by epoll(), you will read() the rest of the data, retreieve the saved data and process it all.
This means that you need space in your per-socket data structure to store the read-but-not-processed-yet data.
You must carefully check the return value of read. It can return any of three things:
A positive number, indicating some bytes were read.
Zero, indicating the other end has gracefully closed the connection.
-1, meaning an error occurred. (If the socket is non-blocking, then the error EAGAIN or EWOULDBLOCK means the connection is still open but no data is ready for you right now, so you need to wait until epoll says there is more data for you.)
If your code is not checking for each of these three things and handling them differently, then it is almost certainly broken.
These cover all of the cases you are asking about, like a client sending 90 bytes then closing or rudely breaking the connection (because read() will return 0 or -1 for those cases).
If you are worried that a client might send 90 bytes and then never send any more, and never close the connection, then you have to implement your own timeouts. For that your best bet is non-blocking sockets and putting a timeout on select() / poll() / epoll(), ditching the connection if it is idle for too long.
TCP connection is a bi-directional stream layered on top of packet-based network. It's a common occurrence to read only part of what the other side sent. You have to read in a loop, appending until you have a complete message. For that you need an application level protocol - types, structure, and semantics of messages - that you use on top of TCP (FTP, HTTP, SMTP, etc. are such protocols).
To answer the specific second part of the question - add EPOLLRDHUP to the set of epoll(7) events to get notified when connection drops.
In addition to what caf has said, I'd recommend just subscribing EPOLLRDHUP because this is the only safe way to figure out whether a connection was closed (read() == 0 is not reliable as, caf mentioned this too, may be true in case of an error). EPOLLERR is always subscribed to, even if you didn't specifically asked for it. The correct behaviour is to close the connection using close() in case of EPOLLRDHUP and probably even when EPOLLERR is set.
For more information, I've given a similar answer here: epoll_wait() receives socket closed twice (read()/recv() returns 0)