I'm pretty new with ZMQ and I'm working with the NodeJS binding. I have an application that uses PUSH/PULL sockets. On one side I PUSH data to some nodes that through the PULL socket receive and process it. Sometimes I have to kill one or more nodes of my application, and it can happen that these nodes still have some data in the PULL socket to be processed. I don't want to lose this data, so I was wondering if there is a way to access ZMQ's PULL socket queue to check if there are still messages to be processed.
I actually couldn't find anything in the specs of ZMQ and the NodeJS binding, so maybe I'm getting the whole concept wrong.
If you kill a process then any data in that processes buffers will be lost.
Instead of killing the process forcefully, you should always find a way to allow processes to shut-down gracefully. Here, you can send a "KILL" message to the PULL socket; the process can then read that and exit when it receives it. If you can flush the socket buffer (depends if there are other processes still sending to it), you can do that and then exit when there are no more messages to read.
I'm posting the solution I found. It's not really a solution as I'm not using the ZMQ socket to check that there are no more messages in the queue, it's just a workaround/hack that came to my mind to make the thing work. I don't have time to write the queue handling by myself, so here's how I solved the problem:
Whenever the processes receive messages to process, they store a timestamp through new Date().getTime(). Whenever a process needs to be killed a kill message is sent to it. As the process receives the message, it starts a timeout with setInterval. Every x seconds (I put 10, can be more or less) the timeout fires a function that checks if the last received message is old enough (takes a timestamp, subtract this ts with the last one saved and if the result is greater that y, which in my case is 100 seconds, it is old enough). If it is, it means no more messages have been received (no more messages in the queue) so it kills the process, otherwise does nothing.
Related
I have around 5700 messages (each message is a 100x100 image as a Base64 string) which I emit from the server to the client from within a for-loop, pretty fast:
[a pretty big array].forEach((imgAsBase64) => {
io.emit('newImgFromServer', imgAsBase64)
})
The client only receives from 1700 to 3000 of them in total, before I get a:
disconnected due to = transport error
socket connected
Once the socket re-connects (and the for-loop has not ended) the emission of new messages from within the loop resumes but I have lost those previous ones forever.
How can I make sure that the client receives all of the messages every time ?
This question is an interesting example of "starving the event loop". If you're in a tight for loop for some period of time with no await in the loop, then you don't let the event loop process any other events during the duration of the for loop. If some events need to be processed during that time for things to work properly, you get problems. Read on for how that applies to this case.
Both client and server need some occasional cycles to process housekeeping pings and pongs in the socket.io protocol. If you firehose messages from one end to the other in a non-stop for loop, you can starve the ability to process those housekeeping messages and it will think that it has timed out (not received the housekeeping messages when it should have which is usually a sign of a lost or inoperative connection). In reality, the housekeeping messages are sitting in the event loop waiting to be processed, but if you never give the event loop a chance to process them, some other code running in the for loop will think that they never arrived.
So, you have to make sure you give both ends enough occasional cycles to process those housekeeping messages. The typical way to do that is to just make sure that you aren't fire hosing messages. Send N messages, then pause for a short period of time (enough time for the event loop to be able to service any incoming network events). Then send N more, pause, etc...
In addition, you could make this whole process a lot more efficient by combining a number of the Base64 strings into a single message. You can probably just put them into an array of 100 of them and send that array of 100 and repeat until they are all sent. Then, obviously change the client to expect an array of Base64 strings instead of just a single one. This will obviously result in a lot fewer messages to send (which is more efficient), but you will still need to pause every so often to let the server process things in the event loop.
Exactly how many messages to send before pausing is something that could be figured out via trial and error, but if you put 100 images into a single message and send 10 of these larger messages (which sends 1,000 images) and then pause for even just 50ms, that should be enough time for the event loop to service any inbound ack messages from socket.io to avoid the timeout. Any sort of pause using setTimeout() makes the setTimeout() get in line behind most other messages that are waiting in the event loop so even a short pause with setTimeout() tends to accomplish the goal of letting the event loop process the things that were waiting to be run.
If end-to-end time was super important, you could experiment with sending more messages at once and/or changing the pause time, but you don't want to end with a setting that is close to where you get a timeout (you want some safety factor).
I use three different threads to read can messages from socket CAN raw and to write can messages to another socket CAN raw, The message read every 2 seconds are put onto queue and retrieved from queue on another thread, mutex thread functions are used, The idea is to write the CAN message every 2 seconds as read. But CAN messages are written every 0.3 millisecond by constantly retrieving from the queue. when the message is only read and put onto queue on every 2 second, so queue the message that is retrieved is not getting removed and remains stagnant in the queue. Every time I try to increase the msgsize, the program shows segmentation fault or automatically killed by the OS. How do i go about debugging this issue. please help, thank you
I'm using pika on python3 to consume a queue on RabbitMQ. My consumer is supposed to stop/be killed on certain events, and thus it should be able to handle the closing of channels and connections by itself. After some running, I found out that I had a lot of "zombie consumers" (for want of a better term, they are registered consumers on the Rabbit, with their unacked messages lying around, but often do not have a matching process anymore) left hanging around.
After some experiments I found out that, when I try to run a channel.cancel() the process just hangs until something else comes to kill it and then the consumer is considered still active by Rabbit for some time (I think about 20 minutes).
My code works like this:
def do_some_work(method_frame, body):
# something happens here
if condition:
logging.info("Closing up...")
requeue = channel.cancel()
logging.info("Consumer stopped, {0} messages sent back".format(requeue))
for method_frame, properties, body in channel.consume(args.queue):
do_some_work(method_frame, body)
When condition is met, I can see the first log line and the requests to get more messages from Rabbit stop (at least this is what I can tell from Rabbit), but the process effectively hangs without closing the channels and connection, until clean ups from Rabbit and the OS happen.
The title says it all, but here's a more in-depth explanation:
I made a chat server for some of my friends and I, but one of the last issues I need to iron out is that when one of them disconnects, there's no available indication of this to the others connected to the server. I'm planning to start a separate thread that makes sure some specific data is sent to the server every minute or so (sending of data also automated on the client side) to keep each client in-check. If one were to not send data for a certain amount of time, it would be discarded as "disconnected."
The problem is, the way my program is set up, it would be impossible to discern whether they both were receiving data without dismantling most of the code already there.
Help is greatly appreciated,
~P
Two recv() threads, non-blocking, same socket: do both receive a sent buffer?
No, but they would both receive the EOS indication (return value is zero).
I am trying to write a Node.js program to execute and monitor javascript programs. I am looking for a way to find out whether the monitored program is still "running" i.e. doing anything useful.
In my current approach, when receiving code to test, I start a new child process and hand the code to it. The child process instruments the code creates a Sandbox using Contextify and executes the code using this sandbox.
After the sandbox.run(code) call returned I know that the blocking part of the code finished and can show that in the UI. However, I don't now whether the code registered any timers using setTimeouts or created any other event sources that would cause parts of the code to be exited later. So I don't know whether it's really "finished" yet.
Is there a way in Node.js to check whether there are still events on the event loop to be handled (or even better, how many are left)?
I found this other question, but it only talks about how to monitor the event loop to find out whether the performance of node is still fine. But I'm not interested in the performance (I don't care if the executing code is blocking for 10s or only doing something for 1ms every 2 minutes) and I don't want to use outside tools but find out about the state of the event loop from inside node itself. Is that possible?
I solved my problem in a way, although I didn't find a general answer to the question.
The idea here is that the process will exit by itself if it has executed all the code it was started with and no EventEmitters are registered anymore. This was basically what I wanted, since I wanted to be notified when the process was "done" and now I could just listen to the "exit" event of the child_process.
But my process that executed the code didn't exit by itself. This had two reasons:
I used a timer to regularly send the data gathered about the execution to the parent process. If such a timer is registered the process won't exit. You could unref the timer but I was afraid that would lead to data loss since the process could quit before the last bit of data was sent out (since it wouldn't wait for the timer to execute). So I changed my code to only schedule a timer if there was data to be sent out instead of regularly checking for data to be sent.
I used fork to create the child process. This also creates a communication channel between the parent and child process and since I needed to send the code to execute to the child_process I registered the child process for messages received from the parent using process.on("message", callback). However, now we have another EventEmitter registered which prevents the process from quitting. Luckily, I realized that I only needed one message from the parent process and no further messages so I could remove the event emitter after receiving that message. So instead of process.on() I used process.once() which will execute the callback only once and automatically remove the event emitter after that. Exactly what I wanted. Alternatively you could use process.removeListener().
Now I just wait for the child_process to exit and thus know that everything is finished and can notify the client.
So the solution here is to make sure none of your own EventEmitters keep the process alive and then just wait for it to exit.