I have three Linux boxes, each running my program.
The program needs to call a certain callback at regular intervals, and each call must happen at the exact same time across the three boxes. I don't need any other synchronization except for the calls.
If it helps, the three boxes have their clocks synchronized by NTP (one of the boxes is the master).
Is there a way to accomplish this with good precision? Preferably non Linux specific. To make things simple, the callback must be called each N ms even if a previous call hasn't completed yet.
How about you send a request to execute the function far enough ahead of time including a timestamp when the function should be executed? The receiving application would sleep/wait the remaining time (some time is lost due to latency), then execute the function at the precise timestamp you requested.
If the called function itself takes longer than your interval, you should probably consider using threads. If the function executes quickly but transfer of the results takes longer then you should get away with something like select() without additional threads.
Related
Premise: I have a calendar-like system that allows the creation/deletion of 'events' at a scheduled time in the future. The end goal is to perform an action (send message/reminder) prior to & at the start of the event. I've done a bit of searching & have narrowed down to what seems to be my two most viable choices
Unix Cron Jobs
Bree
I'm not quite sure which will best suit my end goal though, and additionally, it feels like there must be some additional established ways to do things like this that I just don't have proper knowledge of, or that I'm entirely skipping over.
My questions:
If, theoretically, the system were to be handling an arbitrarily large amount of 'events', all for arbitrary times in the future, which of these options is more practical system-resource-wise? Is my concern in this regard even valid?
Is there any foreseeable problem with filling up a crontab with a large volume of jobs - or, in bree's case, scheduling a large amount of jobs?
Is there a better idea I've just completely missed so far?
This mainly stems from bree's use of node 'worker threads'. I'm very unfamiliar with this concept
and concerned that since a 'worker thread' is spawned per every job, I could very quickly tie up all of my available threads and grind... something, to a halt. This, however, sounds somewhat silly & possibly wrong(possibly indicative of my complete lack of knowledge here), & thus, my question.
Thanks, Stark.
For a calendar-like system, it seems you could query your database to find all events occuring in the next hour, then create a setTimeout() for each one of those. Then, an hour later, do the same thing again. Then, upon any server restart, do the same thing again. You don't really need to worry about events that aren't imminent. They can just sit in the database until shortly before their time. You will just need an efficient way to query the database to find events that are imminent and user a timer for them.
WorkerThreads are fairly heavy weight items in nodejs as they create a whole separate heap and a whole new instance of a V8 interpreter. You would definitely not want a separate WorkerThread for each event.
I should add that timers in nodejs are very lightweight items and it is not problem to have lots of them. They are just stored in a sorted linked list and only the insertion of a new timer takes a little bit more time (to do an insertion sort as it is added to the list) as the list gets longer. There is no continuous run-time overhead because there are lots of timers. The event loop, then just checks the first item in the linked list to see if it's time yet for the next timer to fire. If so, it removes it from the head of the list and calls its callback. If not, it goes about the rest of the event loop work items and will check the first item in the list again the next through the event loop.
Simply put, I want to manipulate two motors in parallel, then when both are ready, continue with a 3rd thread.
Below is image of what I have now. In two top threads, it sets motors B and C to "unlimited", then waits until both trigger the switches, then sets a separate boolean variable for both.
Then in 3rd thread, I poll these two variables with 1 second interval, until AND operation gives true to the loop termination condition.
This is embedded system and all, so it may be ok here, but in "PC programming", this kind of polling loop would be rather horrible thing to do.
Question: Can I do either of both of
wait for variable without this kind of polling loop?
wait for a thread to finish without using a variable at all?
Your question is a bit vague on what you actually want to achieve and using which language. As I understood you want to be able to implement a similar multithreaded motor control mechanism in Labview?
If so, then the answer to both of your questions is yes, you can implement the wait without an explicitly defined variable (other than the error cluster, which you probably would be passing around anyway). The easiest method is to pass an error cluster to both your loops and then use Merge errors to combine the generated errors once the loops are finished. Merge errors will wait until both inputs have data, merges the errors, and passes the merged error cluster on. By wiring the merged error cluster to your teardown function you effectively achieve the thread synchronization you described. If you require thread synchronization for the two control loops, you would however still have to use semaphores, rendezvous', notifiers, and other built-in synch methods.
In the image there's an init function that opens two serial devices (purple wire) and passes them to the control loops, which both runs until an error (yellow-black wire) occurs. The errors from both are merged and passed to the teardown function that releases the serial devices. Notice that in this particular example the synchronization would occur at the end of program as long as there's at least one wire coming from each loop to the teardown function.
Similar functionality in a text based programming language would necessitate the use of more elaborate mechanisms, though some specialised language for parallel programming might help here.
I am currently working with three matlab functions to make them run near simultaneously in single Matlab session(as I known matlab is single-threaded), these three functions are allocated with individual tasks, it might be difficult for me to explain all the detail of each function here, but try to include as much information as possible.
They are CONTROL/CAMERA/DATA_DISPLAY tasks, The approach I am using is creating Timer objects to have all the function callback continuously with different callback period time.
CONTROL will sending and receiving data through wifi with udp port, it will check the availability of package, and execute callback constantly
CAMERA receiving camera frame continuously through tcp and display it, one timer object T1 for this function to refresh the capture frame
DATA_DISPLAY display all the received data, this will refresh continuously, so another timer T2 for this function to refresh the display
However I noticed that the timer T2 is blocking the timer T1 when it is executed, and slowing down the whole process. I am working on a system using a multi-core CPU and I would expect MATLAB to be able to execute both timer objects in parallel taking advantage of the computational cores.
Through searching the parallel computing toolbox in matlab, it seems not able to deal with infinite loop or continuous callback, since the code will not finish and display nothing when execute, probably I am not so sure how to utilize this toolbox
Or can anyone provide any good idea of re-structuring the code into more efficient structure.
Many thanks
I see a problem using the parallel computing toolbox here. The design implies that the jobs are controlled via your primary matlab instance. Besides this, the primary instance is the only one with a gui, which would require to let your DISPLAY_DATA-Task control everything. I don't know if this is possible, but it would result in a very strange architecture. Besides this, inter process communication is not the best idea when processing large data amounts.
To solve the issue, I would use Java to display your data and realise the 'DISPLAY_DATA'-Part. The connection to java is very fast and simple to use. You will have to write a small java gui which has a appendframe-function that allows your CAMERA-Job to push new data. Obviously updating the gui should be done parallel without blocking.
Is it possible, or even practical to create a node program (or sub program/loop) that executes independently of the connected clients.
So in my specific use case, I would like to make a mulitplayer game, where each turn a player preforms actions. And at the end of that turn those actions are computed. Is it possible to perform those computations at a specific time regardless of the client/players connecting?
I assume this involves the use of threads somewhere.
Possibly an easier solution would be to compute the outcome when it is observed, but this could cause difficulties if it has an influence in with other entities. But this problem has been a curiosity of mine for a while.
Well, basically, the easiest solution would probably to run the computation onto a cluster. This is spawning a thread who's running independent task and communicating with messages with the main thread.
If you wish however to run a completely separate process (I probably wouldn't, but it is an option), this can happen too. You then just need a communication protocol between the two process. Usually this would be handled by a messaging or a task queue system. A popular queue solving this issue is RabbitMQ.
If the computations each turn is not to heavy you could solve the issue with a simple setTimeout()
function turnCalculations(){
//do loads of stuff every 30 seconds
}
setTimout(turnCalculations,30000)
//normal node server stuff here
This would do the turn calculations every 30 seconds regardless of users connected, but if the calculations take to long they might block your server.
I have 2 questions :
Q1) Can i implement an asynchronous timer in a single threaded application i.e i want a functionality like this.
....
Timer mytimer(5,timeOutHandler)
.... //this thread is doing some other task
...
and after 5 seconds, the timeOutHandler function is invoked.
As far as i can think this cannot be done for a single threaded application(correct me if i am wrong). I don't know if it can be done using select as the demultiplexer, but even if select could be used, the event loop would require one thread ? Isn't it ?
I also want to know whether i can implement a timer(not timeout) using select.
Select only waits on set of file descriptors, but i want to have a list of timers in ascending order of their expiry timeouts and want select to tell me when the first timer expires and so on. So the question boils down to can a asynchronous timer be implemented using select/poll or some other event demultiplexer ?
Q2) Now lets come to my second question. This is my main question.
Now i am using a dedicated thread for checking timeouts i.e i have a min heap of timers(expiry times) and this thread sleeps till the first timer expires and then invokes the callback.
i.e code looks something like this
lock the mutex
check the time of the first timer
condition timed wait for that time(and wake up if some other thread inserts a timer with expiry time less than the first timer) Condition wait unlocks the lock.
After the condition wait ends we have the lock. So unlock it, remove the timer from the heap and invoke the callback function.
go to 1
I want the time complexity of such asynchronous timer. From what i see
Insertion is lg(n)
Expiry is lg(n)
Cancellation
:( this is what makes me dizzy ) the problem is that i have a min heap of timers according to their times and when i insert a timer i get a unique id. So when i need to cancel the timer, i need to provide this timer id and searching for this timer id in the heap would take in the worst case O(n)
Am i wrong ?
Can cancellation be done in O(lg n)
Please do take care of some multithreading issues. I would elaborate on what i mean by my previous sentence once i get some responses.
It's definitely possible (and usually preferable) to implement timers using a single thread, if we can assume that the thread will be spending most of its time blocking in select().
You could check out using signal() and SIGALRM to implement the functionality under POSIX, but I'd recommend against it (Unix signals are ugly hacks, and when the signal callback function runs there is very little that you can do inside it safely, since it is running asynchronously to your app thread)
Your idea about using select()'s timeout to implement your timer functionality is a good one -- that is a very common technique and it works well. Basically you keep a list of pending/upcoming events that is sorted by timestamp, and just before you call select() you subtract the current time from the first timestamp in the list, and pass in that time-delta as the timeout value to select(). (note: if the time-delta is negative, pass in zero as the timeout value!) When select() returns, you compare the current time with the time of the first item in the list; if the current time is greater than or equal to the event time, handle the timer-event, pop the first item off the head of the list, and repeat.
As for efficiency, your big-O times will depend entirely on the data structure you use to store your ordered list of timers. If you use a priority queue (or a similar ordered tree type structure) you can have O(log N) times for all of your operations. You can even go further and store the events-list in both a hash table (keyed on the event IDs) and a linked list (sorted by time stamp), and that can give you O(1) times for all operations. O(log N) is probably sufficiently efficient though, unless you plan to have a really large number of events pending at once.
man pthread_cond_timedwait
man pthread_cond_signal
If you are a windows App, you can trigger a WM_TIMER message to be sent to you at some point in the future, which will work even if your app is single threaded. However, the accuracy of the timing will not be great.
If your app runs in a constant loop (like a game, rendering at 60Hz), you can simply check each time around the loop to see if triggered events need to be called.
If you want your app to basically be interrupted, your function to be called, then execution to return to where it was, then you may be out of luck.
If you're using C#, System.Timers.Timer will do what you want. You specify an event handler method that the timer calls when it expires, which can be in the class that you invoke the timer from. Note that when the timer calls the event handler, it will do it on a separate thread, which you need to take into account if you're updating the user interface, or use its SynchronizingObject property to run it on the UI thread.