I have an experiment in which I present stimuli using PsychoPy / PyGaze and track eye movements with an EyeTribe eye tracker. In this experiment I update the size of two visual stimuli on each frame (at 60 Hz). I prepare each frame beforehand and afterwards loop through all of the screen objects and present them. Meanwhile, a continuous sound is playing. When I run this experiment in dummy mode (mouse movement is used as a simulation for gaze position), there are no timing issues for the visual presentation. However, when I run the experiment while performing eye tracking, the timing of the visual presentation is no longer accurate (higher variability in duration of frames).
I tried looking into the multi threading more, but in the pytribe script of PyGaze I can't find any evidence that one thread is waiting for an event coming from the eye tracking thread. So, I have no idea how to figure out what is causing the timing issues or how to solve this? (I hope I explained the problem sufficiently specific).
It's worse than just needing a separate thread for eyetrack versus stimulus rendering. What you really need is a separate process that avoids the python Global Interpreter Lock (GIL). The GIL prevents different threads from running on different processors.
For improved temporal precision I would really recommend you switch from pygaze to iohub (which also has support for eyetribe I believe). iohub does run genuinely on a different core of the machine where possible so that your stimuli and eye data can be processed independently in time, and it handles all the sync stuff for you.
Adding to Jon's answer: Hanne also emailed about the problem, and it turns out she was running her experiments from Spyder. When run from the command prompt, there shouldn't be any timing issues. (Obviously, the GIL is still around, but in practice this doesn't seem to affect screen timing.)
To prevent any issues in the future, I've added a class that allows for running the EyeTribe in a parallel Process. See: https://github.com/esdalmaijer/PyTribe/blob/master/pytribe.py#L365
Example use:
if __name__ == "__main__":
from pygaze.display import Display
from pygaze.screen import Screen
from pytribe import ParallelEyeTribe
disp = Display()
scr = Screen()
scr.draw_fixation(fixtype='cross')
tracker = ParallelEyeTribe()
tracker.start_recording()
disp.fill(scr)
disp.show()
tracker.log("Stimulus onset")
time.sleep(10)
disp.show()
tracker.log("Stimulus offset")
tracker.stop_recording()
tracker.close()
disp.close()
Related
First of all, I attach some of the general specifications of my project at the end of this post.
The main objective of my project is to detect the use of face mask via camera vision and then actuate certain actions accordingly. For example, if it detects a person not wearing a face mask, a buzzer will start to buzz continuously, Red LED start to flash and the gate will not open.
So far I manage to implement the object detection process, where it able to detect the use of face mask sufficiently. The object detection process should run continuously in a infinite while loop without any delays and will stop until I press a specific key.
The problem is when I try to incorporate delays for the actuation process in the same loop, for instance a blinking LED. The video stream for the object detection process froze because of the delays.
Thus, I try a few things to ensure that the output actuation process will not interrupt the object detection process such as by implementing multiprocessing along with pickle file that act as a buffer memory storage that stores information produce by the object detection process. But still, I did not manage to solve this problem. I have an issue with writing/reading the pickle file simultaneously from two different processes.
Requirement of the processes are as listed below.
Process 1 (Main Process)
In an infinite loop
No delays, speed of the iteration is limited by the hardware and the OS
Able to write output signal as soon as it detect the face mask
Process 2 (Secondary Process)
Start to run the program once received signal from the main process
Able to read output signal from the main process
Able to operate with delays without interrupting the main process
Able to delete/edit the output signal from the main process
Killed once the main process is terminated
Therefore, I wonder if there is any method/library/function that able to run two process simultaneously and independently, with different timing, and able to retrieve/transfer information within those processes. If it is necessary to share my codes, please do inform me.
Thank you.
General specifications of my project:
Programming language, Python3
Text editor/compiler, Code-OSS
Hardware, Nvidia Jetson Nano 2GB
OS, Linux/Nvidia JetPack
Pre-trained model, SSD-Mobilenet V2
After reading and searching more about multiprocessing I manage to find something that are useful for my project, it is the method of "Sharing data using server process" and "Process synchronization" more details about this features you may refer to the YouTube video below. It is highly recommended for you to watch the full playlist, so that you have broader understanding on multiprocessing which might simplify your work.
Sharing data using server process
https://youtu.be/v5u5zSYKhFs
Process synchronization
https://youtu.be/-zJ1x2QHTKE
Both of this method successfully solve my problem, I think my previous problem raised due to the issue of simultaneous writing and reading of the pickle file from both of the processes.
I'm developing a game in pyglet, that scheduled by a simple text file like :
0:00:01;event1
0:00:02;event2
0:00:03;event3
The fact is that, among these events, some might be blocking (for instance event2 might consist in displaying instructions until a key is pressed). As a consequence, event3 might not be executed at the proper time (i.e., during the event2). For now, my strategy is to schedule one event after the other :
Execute the first event
Once the first event is finished, compute the remaining duration between the first and the second event (delta_duration)
Schedule the second event with a delay of delta_duration
... and so on
For now, I did not succeed in implementing properly a blocking event with this strategy. It seems that anything blocking the event_loop (like a sleep call during event2) is preventing even the graphical elements of event2 (text instructions) to be displayed. On the other hand, if I do not put any blocking routine (sleep) in the event2, I'm able to see the vertices, but the scheduler keeps on scheduling (!), and so the event3 comes too soon.
My question is : what would be a general strategy, in pyglet, to articulate non-blocking to blocking events ? More precisely, is it possible (desirable) to use multiple clocks for that purpose ? The pyglet documentation mentions that multiple clocks can be used but it is not very well explained.
I don't want a solution that is specific to my events example but, rather, general indications about the way to go.
It's really up to your program on what blocks. If you are using input from Python for the console window, then yes that will block because it's blocking execution of Python in general. If you have a label popup in the window that is waiting for input from an on_key_press window event, then that is completely different as it's not blocking the pyglet loop, it was scheduled within it.
If your event is a 20 second long math calculation, then that should probably be ran in a thread. You will probably have to separate the types of events in order to differentiate how they should be ran. It's hard to say because without a runnable example or sample of code, I am just guessing at your intentions.
Although it sounds more like you are wanting some sort of callback system. When execution of func1 is declared done, go to func2. There is nothing built into pyglet like this, you would have to have a clever use of scheduling. There are examples of this using pure python though. I personally use Twisted Deferred's for this.
I develop a SCADA in python3 with the help of PyQt. I expect my program to continuously indicate various parameters received through RS-485 interface, however, within several minutes from the start (it is always a different time), the GUI stops to update itself. At the same time, the GUI stays responsive and if I were to click on one of the animated QAbstractButtons it has, the GUI starts to work as intended again for a short period of time. The problem occurs on both Linux and Windows.
The program has several worker threads: one for RS-485 exchange, one for reading/writing various data to disk, one for decoding received package and refreshing data in memory, one for request queue management, etc. They all work in loops of while (True): time.sleep(...) - do the job. The GUI is implemented in the main thread.
The data is indicated with QLabels. The QLabel.setText is added to painterEvent() of QWidgets containing the QLabels.
When the GUI stops updating, the other threads are up and running: the exchange is functioning, the request queue is forming, etc. Despite not being updated, the GUI stays responsive and reacts to QAbstractButton clicks.
I tried adding gui.update() or app.processEvents() into one of the worker threads, tried force updating through QTimer in the main thread. The result is the same: it works for a short while and then stops.
I tried increasing the time.sleep of the refresh thread and force updating the main widget over longer intervals of time (0.5 to 5 seconds) and it seems to help the situation a lot, this way it can run for several minutes, but it still does not solve the problem.
I would love to show the code, but the whole thing is way too bulky to post here and if I could narrow it down to a at least a hundred lines, I would have already solved the issue by now. So if any of you could at least share some general considerations on what to look for, I would be very happy.
update:
This seems to work, I'll leave it running for a few hours tomorrow to confirm:
update_timer = QTimer()
update_timer.setSingleShot(False)
update_timer.timeout.connect(self.gui.repaint)
update_timer.setInterval(500)
update_timer.start()
I assume that self.gui.update() did not work because dataChanged() was probably not emitted and control passed instead of doing repaint(). As far as I understand, the solution above is not the right way to update widgets.
So, the question actually boils down to the following:
What is the right way to update the main QWidget and how do I let the program know it needs to be redrawn, probably using dataChanged() signal?
The answer is simple. Never do anything related to PyQt from outside the main thread. Even if it seems the only logical way to do something.
After I 'fixed' the issue with the gui not being updated, I stumbled into a more serious problem: the program crashed every now and then. As it turns out, the GUI not being updated was only the top of the iceberg. The uptime was usually 2 to 7 minutes, I received a few glibc errors ("corrupted double-linked list" and "double free or corrution"). As it turns out, the source of the problem was inside refresh thread that has several hundreds of lines updating gui:
self.gui.screen_name.device_name.setCrash()
where setCrash() changes widget color and palette, or even more direct things like
self.gui.screen_name.label_name.setText(str(value))
I tracked down everything even loosely related to gui from the main file (the place where all the threading took place) and restyled it. Now it only has one worker thread - the RS-485, the rest are wrapped into separate QTimers.
This is what it was:
class Main():
def __init__():
self.plots_thread = Thread(target = self.plots_refresh)
self.plots_thread.start()
def plots_refresh():
while True:
time.sleep(0.5)
#do stuff
And this is what it is now:
class Main():
def __init__():
self.plots_refresh_timer = QTimer()
self.plots_refresh_timer.setSingleShot(False)
self.plots_refresh_timer.timeout.connect(self.plots_refresh)
self.plots_refresh_timer.setInterval(499) #prime number
self.plots_refresh_timer.start()
def plots_refresh():
#do stuff
This time the program has been running for over an hour and never crashed. After coming to conclusion that this was THE fix, I refactored the really messy test file and resumed testing - again, no troubles for half an hour.
I have a large data acquisition and control program written in Python3.4.2 using GUI mostly developed on Glade 3.18.3, Gtk3.0 GObject running Debian 8 with XFCE.
There are timers that keep doing things, and these work fine. After startup, the program runs for some 3 - 7 days, then suddenly, all of the windows go blank and stay blank. Other applications are not affected. Memory and CPU usage is modest.
There are no indications of problems prior to the windows going blank. The windows show their title bars and respond normally to minimize, restore, move to another Workspace, etc. It looks like they are not getting repainted - no widgets are visible at all. The code is way too large to post here, and I am not able to isolate a specific problem area for lack of obvious symptoms other than the blank screens. There are no error messages or warnings.
The timers continue to run, acquire data and control things. This happens whether the program is run from the command line or under PyDev in Eclipse.
Things I have tried:
In the main timer loop, I put code to look for a file, and then exec the command in it, printing the results, so I have been able to mess with the program in real time:
Replace the usual Gtk.main() with a while loop whose variable, if not made false, will re-execute the Gtk.main(). Executing Gtk.main.quit() stops Gtk.main and starts it again. Windows still blank. Did this repeatedly to no avail.
Experimented with garbage collection with GC. Collecting garbage makes no different. Windows still blank.
Put in code to print percent of time consumed by the timer loops. Fairly steady around 18 - 20% of available CPU time, so nothing is hogging the CPU preventing re-paint.
I have a button that clears a label. I read the label, then executed a builder.get_object(...).activate command to the button. I re-read the label and it was now properly blank. So events and widgets appear to be working normally, at least to some extent.
Finally, if I click on the close X on the title bar, XFCE asks me if I want to wait or close now. So it seems as though there may be a disconnection or problem with signals and the OS, even though XXX.activate() works.
Web searching is in vain. Does anybody have ideas of what might be happening, useful diagnostics, or other suggestions? Many thanks!
April 27, 2017 Update:
I have taken two substantial steps to mostly work around problems. First, partly in response to a couple of Gtk crashes over the last few months, instead of ending the main program with:
Gtk.main()
I end with:
while wannalive:
try:
Gtk.main()
except:
pass
wannalive is True until user does a quit, so recovery is instant.
Second, I grouped all of the code for each window setup and initial population of static items into two functions. I also made another function for closing a window. These functions propagate to children, grandchildren windows. A function in the top window first, closes, then re-creates all windows, with one call. In operation, there are overlaps in what windows exist, but that is not a problem.
Above, I describe that I can inject code with an external program. The external program has a button that injects a call to that third function. In about five seconds or less, the result of a single button click is to replace all blank windows with functional windows. For my purposes in a controlled environment with a trained operator, this is acceptable.
Next, let me address the relationship between the timer loops and GUI events processing. I do use GObject.timeout_add(ms, somefunction). Experiment shows that a button that calls time.sleep(5) stalls the timer. Experiment shows that injecting time.sleep(5) in the timer loop stalls the GUI. This is consistent with my belief (correct me if I am wrong) that Python runs on a single thread. Therefore, bad code caught in an infinite loop should stall both the GUI and the timer loop. (The program has one timeout_add call.)
I'm writing an app that embeds a bunch of matplotlib figures into a PyQt GUI. The updating of these figures can take up to a few seconds, so I would like to introduce a waiting indicator to display while the plots are being drawn. I've moved all the data processing code into its own thread, but it seems the actual plotting calls are often making up the majority of processing time.
I have written a waiting indicator that uses a QTimer instance to trigger paintEvent on the widget. This works just fine when all the intensive processing can be pushed into another thread. The problem is that these calls to construct the matplotlib plots cannot be moved outside of the main thread due to the way Qt is designed, and so block the updating of the waiting indicator, rendering it kind of useless.
I've introduced some calls to QCoreApplication.processEvents() after the updating of each figure, which improves the performance a little. I've also toyed with the idea of monkeypatching a bunch of methods of matplotlib.axes.Axes to include calls to QCoreApplication.processEvents(), but I can see that getting messy. Is this the best I can do? Is there any way to interrupt the main thread at regular intervals and force it to process new events?
It should also help a big deal to do the actual drawing on a QPixmap in a thread. Drawing that pixmap with QPainters drawPixmap() method is very fast. And you need to recreate the pixmap only when really needed (e.g. after Zooming or so). In the meantime you just have to reuse that already drawn pixmap. The actual paintEvents using drawPixmap() will cost close to nothing and your GUI will be completely responsive.
Clobbering the code with processEvent() is not only ugly but can cause very nasty and very hard to debug malfunctions. E.g. it might cause premature deletion of objects which are still in use but were scheduled for deletion using deleteLater().
This Answer might be also of use: Python - matplotlib - PyQT: plot to QPixmap
I havn't used matplotlib, yet. But in case it uses directly QWidgets and can not be used without it won't be so easy as you mentioned above. But you could do the drawing in another process started by your GUI which uses matplotlib as in the link above and stores the pixmap to disk and your gui loads whenever a new pixmap is ready. QFileSystemWatcher might help here to avoid polling.