I have a program that uses ZMQ to send and receive between a C++ application and a Python GUI. The Python sends all commands to the C++ app to do the work, the C++ app periodically sends back status to update the GUI.
The C++ is multi-theaded but we never made the call zmq_send thread safe, so in 1 out of 100,000 runs we'd get an unhandled exception or segmentation fault if two threads tried to send status back to the gui simultaneously. This took longer than I care to admit figuring out since it was so sporadic. This was easily solved with a mutex around zmq_send because the socket is managed by a singleton.
In addition to the processing threads, there is one thread that just idly waits to receive and dispatch commands from the gui using zmq_poll and then zmq_msg_recv when something is available.
The question, can I safely poll the same socket while a send is happening? Most of the time the receive thread is sitting in zmq_poll with a timeout, and sends seem to be happening without issue. I can't seem to find any good documentation about this. I assume a mutex needs to protect zmq_send and zmq_msg_recv from occurring simultaneously, but I am not sure about the safety of polling while sending.
Details about the setup: using PAIR interface with a single client and server. All messages are small (<1KB). There is only one socket shared for sending and receiving.
This is a large, decade old application I'd like to avoid redesigning if possible.
Related
The following situation:
Web client: Using JavaScript socketio to listen for incoming messages (= JavaScript).
Web server: Using flask-socketio with eventlet to send data (= Python).
Everything works if the client sends a message to the server. The server receives the messages. Example:
socketio = SocketIO(app, engineio_logger=True, async_mode="eventlet")
#socketio.on("mymsg")
def handle_event(message):
print("received message: " + str(message))
Unfortunately the other way around does not work - to some extent. I have a thread producing live data about 5 to 10 times a second the web frontend should display. It should be sent to the client.
First: It does not work at all if the thread producing the data tries to invoke sockeito.emit() directly. The reason for that is unclear to me but somehow plausible as flask-socketio with eventlet follows different async models, as the documentation says.
Second: Decoupling classic threads from the async model of flask/eventlet works to some extent. I attempt to use an eventlet queue for that. All status data my thread produces is put into the queue like this:
statusQueue.put(statusMsg)
This works fine. Debugging messages show that this is performed all the time, adding data after data to the queue.
As the documentation of flasks tells I'm adviced to use socketio.start_background_task() in order to get a running "thread" in a compatible mode to the async model socketio uses. So I am using this code:
def emitStatus():
print("Beginning to emit ...")
while True:
msg = statusQueue.get()
print("Sending status packet: " + str(msg))
socketio.emit("status", msg, broadcast=True)
statusQueue.task_done()
print("Sending status packet done.")
print("Terminated.")
socketio.start_background_task(emitStatus)
The strange thing where I'm asking you for help is this: The first call to statusQueue.get() blocks as expected as initially the queue is empty. The first message is taken from the queue and sent via socketio. Debug messages at the client show that the web client receives this message. Debug messages at the server show that the message is sent successfully. But: As soon as the next statusQueue.get() is invoked, the call blocks indefinitely, regardless of how many messages get put into the queue.
I'm not sure if this helps but some additional information: The socketio communication is perfectly intact. If the client sends data, everything works. Additionally I can see the ping-pongs both client and server play to keep the connections alive.
My question is: How can I properly implement a server that is capable of sending messages to the client asynchronously?
Have a look at https://github.com/jkpubsrc/experiment-python-flask-socketio for a minimalistic code example featuring the Python-Flask server process and a JQuery based JavaScript client.
(FYI: As these are status messages not necessarily every message needs to arrive. But I very much would like to receive at least some messages not only the very first message and then no other message.)
Thank you for your responses.
I left two solutions to make the code work as pull requests.
Basically, the answer is: you choose one technology and stick a process with it:
Going async_mode=threading? Great, use stdlib Queue. Don't import eventlet unless you have to.
Going async_mode=eventlet? Also great, use eventlet Queue and don't forget that stdlib time.sleep or socket IO will block everything else, fix with eventlet.monkey_patch()
If you must use both eventlet and threading, the best approach is to let them live in separate OS processes and communicate via local socket. It's extra work, but it is very robust and you know how it works and why it will not break.
With good knowledge of both eventlet and native threads you can carefully mix them into working code. As of 2018-09, mixing doesn't work in friendly obvious way, as you already found. Sorry. Patches are welcome.
I am wondering if a ZeroMQ REP socket is allowed to be poll()-ed on incoming data in one thread and used to send data from the other thread.
The idea I am trying to follow is the following:
A REP socket is not going to receive anything, as long as it did not send a reply to the incoming request. Thus if a zmq_poll() was called for such a socket, it'd just block (until timeout or forever).
Now, while this socket is a part of the zmq_poll() call for incoming data, what happens if another thread prepared a reply and uses this socket to send this reply.
Is is safe to do so or are race conditions possible than?
ZeroMQ has been built on a set of a few maxims.
Zero-sharing is one of these core maxims.
While a user can at her/his own risk experiment with sharing, ZeroMQ best practices avoid doing that, except for very few and very specific cases and not on a socket-level. Sockets are knowingly not thread-safe, for the sake of the higher overall performance and lower latency.
This is the reason why a question "What happens if another thread ..." may sound legitimate, but not inside the domain of ZeroMQ Best Practices zone.
I'm working with a Node library that doesn't explicitly close sockets after it's done with them. Instead it tries to clean up by deleting reference to the socket and letting them be garbage collected.
Googling is failing me: I don't think that is possible for the GC to clean up unclosed sockets. That is, I think that any socket descriptors will still be in use, from the OS's perspective.
Additionally, assuming that I as the library consumer have access to the socket objects, what is the best way for me to close them? I have played around with end(), close(), and destroy() with limited success. Sometimes they seem to block into perpetuity (end/destroy), and other times it seems like the callback is never made (close).
It could be due to the fact that your socket sent a FIN package and hangs up on the connection while waiting for the other end to send the FIN2 message. In cases when the socket on the other side is not nicely closed, your one won't receive any package, thus hanging up forever.
Actually, end sends a FIN packet and does not shutdown the socket.
A possible solution could be to wait for a while on it by means of setTimeout when you invoke the end function, then explicitly destroy it by means of the function destroy. This won't affect your socket if the other end has correctly closed the connection, otherwise it will force the shutdown and all resources should be released.
assume we have an application which uses winsock to implement tcp communication.
for each socket we create a thread and block-receiving on it.
when data arrives, we would like to notify other threads (listening threads).
i was wondering what is the best way to implement this:
move away from this design and use a non-blocking socket, then the listening thread will have to iterate constantly and call a non-blocking receive, thus making it thread safe (no extra threads for the sockets)
use asynchronous procedure calls to notify listening threads - which again will have to alert-wait for apc to queue for them.
implement some thread safe message queue, where each socket thread will post messages to it, and the listener, again, will go over it every interval and pull data from it.
also, i read about WSAAsyncSelect, but i saw that this is used to send messages to a window. isnt there something similar for other threads? (well i guess apcs are...)
Thanks!
Use I/O completion ports. See the CreateIoCompletionPort() and the GetQueuedCompletionStatus() functions of the Win32 API (under File Management functions). In this instance, the socket descriptors are used in place of file handles.
You'll always be better off abstracting the mechanics of socket API (listening, accepting, reading & writing) in a separate layer from the application logic. Have an object that captures the state of a connection, which is created during an incoming connection and you can maintain buffers in this object for the incoming and outgoing traffic. This will allow your network interface layer to be independent of the application code. This will also make the code cleaner by separating the application functionality from the underlying communication mechanism.
Blocking or non-blocking socket decision depends on the level of scalability that your applications needs to achieve. If your application needs to support hundreds of incoming connections, adopting a thread-per-socket approach is not going to be very wise. You'll be better off going for an Io ports based implementation, which will make your app immensely scaleable at added code complexity. However, if you only foresee a few 10s of connections at any point in time, you can go for an asynchronous sockets model using Win32 events or messages. Win32 events based approach doesn't scale very well beyond a certain limit as you would have to manage multiple threads if the number of concurrent sockets exceed 63 (as WaitForMultipleObjects can only support a max of 64 sockets). Windows message based mechanism doesn't have this limitation though. OHOH, Win32 event based approach does not require a GUI window to work.
Check out WSAEventSelect along with WSAAsyncSelect API documentation in MSDN.
You might want to take a look at boost::asio package as well. It provides a neat (though a little complex) C++ abstraction over sockets API.
I have a simple application on a OpenWRT style router. It's written in C++ currently. The router (embedded Linux) has very limited disk space and RAM. For example there is not enough space to install Python.
So, I want to control this daemon app via the network. I have read some tutorials on creating sockets, and listening to the port for activity. But I haven't been able to integrate the flow into a C++ class. And I haven't been able to figure out how to decode the information received, or how to send a response.
All the tutorials I've read are dead ends, they show you how to make a server that basically just blocks until it receives something, and then returns a message when it got something.
Is there something a little more higher level that can be used for this sort of thing?
Sounds like what you are asking is "how do I build a simple network service that will accept requests from clients and do something in response?" There are a bunch of parts to this -- how do you build a service framework, how do you encode and decode the requests, how do you process the requests and how do you tie it all together?
It sounds like you're having problems with the first and last parts. There are two basic ways of organizing a simple service like this -- the thread approach and the event approach.
In the thread approach, you create a thread for each incoming connection. That thread reads the messages (requests) from that connection (file descriptor), processes them, and writes back responses. When a connection goes away, the thread exits. You have a main 'listening' thread that accepts incoming connections and creates new threads to handle each one.
In the event approach, each incoming request becomes an event. You then have event handlers that processes these events, sending back responses. Its important that the event handlers NOT block and complete promptly, otherwise the service may appear to lock up. Your program has a main event loop that waits for incoming events (generally blocking on a single poll or select call) and reads and dispatches each event as appropriate.
I installed python-mini package with opkg, which has socket and thread support.
Works like a charm on a WRT160NL with backfire/10.03.1.