I have a GUI in PyQt with a function addImage(image_path). Easy to imagine, it is called when a new image should be added into a QListWidget. For the detection of new images in a folder, I use a threading.Thread with watchdog to detect file changes in the folder, and this thread then calls addImage directly.
This yields the warning that QPixmap shouldn't be called outside the gui thread, for reasons of thread safety.
What is the best and most simple way to make this threadsafe? QThread? Signal / Slot? QMetaObject.invokeMethod? I only need to pass a string from the thread to addImage.
You should use the built in QThread provided by Qt. You can place your file monitoring code inside a worker class that inherits from QObject so that it can use the Qt Signal/Slot system to pass messages between threads.
class FileMonitor(QObject):
image_signal = QtCore.pyqtSignal(str)
#QtCore.pyqtSlot()
def monitor_images(self):
# I'm guessing this is an infinite while loop that monitors files
while True:
if file_has_changed:
self.image_signal.emit('/path/to/image/file.jpg')
class MyWidget(QtGui.QWidget):
def __init__(self, ...)
...
self.file_monitor = FileMonitor()
self.thread = QtCore.QThread(self)
self.file_monitor.image_signal.connect(self.image_callback)
self.file_monitor.moveToThread(self.thread)
self.thread.started.connect(self.file_monitor.monitor_images)
self.thread.start()
#QtCore.pyqtSlot(str)
def image_callback(self, filepath):
pixmap = QtGui.QPixmap(filepath)
...
I believe the best approach is using the signal/slot mechanism. Here is an example. (Note: see the EDIT below that points out a possible weakness in my approach).
from PyQt4 import QtGui
from PyQt4 import QtCore
# Create the class 'Communicate'. The instance
# from this class shall be used later on for the
# signal/slot mechanism.
class Communicate(QtCore.QObject):
myGUI_signal = QtCore.pyqtSignal(str)
''' End class '''
# Define the function 'myThread'. This function is the so-called
# 'target function' when you create and start your new Thread.
# In other words, this is the function that will run in your new thread.
# 'myThread' expects one argument: the callback function name. That should
# be a function inside your GUI.
def myThread(callbackFunc):
# Setup the signal-slot mechanism.
mySrc = Communicate()
mySrc.myGUI_signal.connect(callbackFunc)
# Endless loop. You typically want the thread
# to run forever.
while(True):
# Do something useful here.
msgForGui = 'This is a message to send to the GUI'
mySrc.myGUI_signal.emit(msgForGui)
# So now the 'callbackFunc' is called, and is fed with 'msgForGui'
# as parameter. That is what you want. You just sent a message to
# your GUI application! - Note: I suppose here that 'callbackFunc'
# is one of the functions in your GUI.
# This procedure is thread safe.
''' End while '''
''' End myThread '''
In your GUI application code, you should create the new Thread, give it the right callback function, and make it run.
from PyQt4 import QtGui
from PyQt4 import QtCore
import sys
import os
# This is the main window from my GUI
class CustomMainWindow(QtGui.QMainWindow):
def __init__(self):
super(CustomMainWindow, self).__init__()
self.setGeometry(300, 300, 2500, 1500)
self.setWindowTitle("my first window")
# ...
self.startTheThread()
''''''
def theCallbackFunc(self, msg):
print('the thread has sent this message to the GUI:')
print(msg)
print('---------')
''''''
def startTheThread(self):
# Create the new thread. The target function is 'myThread'. The
# function we created in the beginning.
t = threading.Thread(name = 'myThread', target = myThread, args = (self.theCallbackFunc))
t.start()
''''''
''' End CustomMainWindow '''
# This is the startup code.
if __name__== '__main__':
app = QtGui.QApplication(sys.argv)
QtGui.QApplication.setStyle(QtGui.QStyleFactory.create('Plastique'))
myGUI = CustomMainWindow()
sys.exit(app.exec_())
''' End Main '''
EDIT
Mr. three_pineapples and Mr. Brendan Abel pointed out a weakness in my approach. Indeed, the approach works fine for this particular case, because you generate / emit the signal directly. When you deal with built-in Qt signals on buttons and widgets, you should take another approach (as specified in the answer of Mr. Brendan Abel).
Mr. three_pineapples adviced me to start a new topic in StackOverflow to make a comparison between the several approaches of thread-safe communication with a GUI. I will dig into the matter, and do that tomorrow :-)
Related
I have a script that uses user input and based on that it generates some data (code below). The script uses threads so the GUI does not freeze while the background tasks are running.
My question is if it is possible to pause the worker thread, ask the user for some input, send it to the worker thread and then to continue the operation?
from tkinter.constants import LEFT, RIGHT, S
import tkinter.messagebox
import tkinter as tk, time, threading, random, queue
from tkinter import NORMAL, DISABLED, filedialog
class GuiPart(object):
def __init__(self, master, client_instance):
canvas = tk.Canvas(master, height=600, width=1020, bg="#263D42")
canvas.pack()
self.button1 = tk.Button(master, text="Mueller Matrix - MMP mode", padx=10,
pady=5, bd = 10, font=12, command=client_instance.start_thread1)
self.button1.pack(side = LEFT)
def files(self):
self.filena = filedialog.askopenfilenames(initialdir="/", title="Select File",
filetypes = (("comma separated file","*.csv"), ("all files", "*.*")))
return self.filena
def processing_done(self):
tkinter.messagebox.showinfo('Processing Done', 'Process Successfully Completed!')
return
class ThreadedClient(object):
def __init__(self, master):
self.running=True
self.gui = GuiPart(master, self)
def start_thread1(self):
thread1=threading.Thread(target=self.worker_thread1)
thread1.start()
def worker_thread1(self):
if self.running:
self.gui.button1.config(state=DISABLED) # can I disable the button from the main thread?
user_input = self.gui.files() # can I pause the Worker thread here so ask the user_input from main loop?
time.sleep(5) # to simulate a long-running process
self.gui.processing_done()
self.gui.button1.config(state=NORMAL)
root = tk.Tk()
root.title('test_GUI')
client = ThreadedClient(root)
root.mainloop()
I am asking this as from time to time my script ends abruptly saying that the GUI is not part of the main loop. Thank you for your help!
In general, you should run the GUI from the main thread.
There is some confusion whether tkinter is thread-safe. That is, if one can call tkinter functions and methods from any but the main thread. The documentation for Python 3 says “yes”. Comments in the C source code for tkinter say “its complicated” depending on how tcl is built.
You should be able to call tkinter functions and methods from additional threads in Python 3 if your tkinter was built with thread support. AFAIK, the tkinter that comes with the downloads from python.org have threading enabled.
If you want to be able to pause additional threads, you should create a global variable, e.g. pause = False.
The extra threads should regularly read the value of this and pause their calculations when set to true, e.g.:
def thread():
while run:
while pause:
time.sleep(0.2)
# do stuff
Meanwhile the GUI running in the main thread should be the only one to write the value of pause, e.g. in a widget callback.
Some sample callbacks:
def start_cb():
pause = False # So it doesn't pause straight away.
run = True
def stop_cb():
pause = False
run = False
def pause_cb():
pause = True
def continue_cb():
pause = False
If you have more complicated global data structures that can be modified be several threads, you should definitely guard access to those with a Lock.
Edit1
Threaded tkinter programs without classes:
Simple threaded tkinter example program
Real-world example for unlocking ms-excel files
Edit2:
From the python source code for _tkinter:
If Tcl is threaded, this approach won't work anymore. The Tcl
interpreter is only valid in the thread that created it, and all Tk
activity must happen in this thread, also. That means that the
mainloop must be invoked in the thread that created the
interpreter. Invoking commands from other threads is possible;
_tkinter will queue an event for the interpreter thread, which will
then execute the command and pass back the result. If the main
thread is not in the mainloop, and invoking commands causes an
exception; if the main loop is running but not processing events,
the command invocation will block.
You can read the whole comment here.
I working on a gui application which plots real time data as it comes in. There is a main class which is a plot window. The window shows the plot of the data itself as well as a number of buttons, checkboxes, textboxes, etc. that a user can interact with to modify the plot output. All of these widgets configure the "settings" for the plot. The plotting of new data itself is resource intensive. The loading and plotting of the data may take around a second. I want the user to be able to modify the settings in the GUI without worrying about the lag of loading and plotting so I want to execute the loading and plotting in a separate thread.
I've come up with, what seems to me, to be a weird way to accomplish this but which works. It is something like this:
from PyQt5 import QtCore, QtWidgets
from PyQt5.QtCore import QThread, pyqtSignal
class PlotWorker(QtCore.QObject):
def __init__(self, plotwindow):
self.plotwindow = plotwindow
self.plot_thread = QThread()
self.plot_thread.start()
self.moveToThread(self.plot_thread)
def worker_update:
self.plotwindow.updating = True
self.plotwindow.plot()
self.plotwindow.updating = False
class PlotWindow(Ui_PlotWindow, QtWidgets.QMainWindow):
update_signal = pyqtSignal()
def __init__(self):
self.setupUi(self)
self.settings = self.settings_lineEdit.text()
self.worker = PlotWorker(self)
self.update_signal.connect(self.worker.worker_update)
self.update_settings_pushButton.clicked.connect(self.update_settings)
self.updating = False
self.refresh_timer = QtCore.QTimer(self)
self.refresh_timer.timeout.connect(self.update)
self.update_settings()
self.plot()
self.refresh_time.start(1)
def update_setting(self):
self.settings = self.settings_lineEdit.text()
def update(self)
if self.updating:
print('Already updating, cannot update until previous update complete')
elif not self.updating:
self.update_signal.emit()
def plot(self):
# Plot the data, slow. Uses self.settings
self.load()
...
def load(self):
# Load the data from the appropriate directory/file, slow
...
This gets the rough gist of what I am going for. Ui_PlotWindow implements setupUi. In practices there self.settings stands for a long list of settings variables which can be manipulated by the user. The check on self.updating ensures that requests to update the plot which arrive while the plot is currently updating are terminated rather than added to a queue in the worker thread event loop.
Though my code works, I feel like the pattern I am using with the worker thread is a bit strange. Basically ALL of the information needed is in the PlotWindow class, but since I want to call one of the methods of the PlotWindow class in a separate thread I feel I need a different QObject which can live in a different QThread to house the slot which will run the expensive method.
It just feels a bit roundabout to have a whole seperate class and object just to call a function which already exists in the first class.. However, I am new to threading applications and perhaps this is a normal pattern and not to be worried about?
I would appreciate any advice for how I might be able to make the flow of this function more clear.
The answer is to use a different pattern for threading. Instead of the self.thread = and self.moveToThread for the worker the worker should simply BE a QThread. The thread can be "started" in which case its run method is initiated in a new thread. If a request is made in the main thread to "start" the worker thread again while it is already running nothing will happen. Below is a revision of the above code which implements this pattern.
WARNING: This code is still a bad idea because the plotting involving redrawing GUI objects (the plots). This should not happen in a thread different than the main thread. For posterity I'll state that code in the format in the original question did in fact run but I think that was by luck that the plotting worked in the worker thread. The best pattern would be to have any data loading and processing happen in the worker thread but to then finally emit the data to a plotting method of the main thread to update the GUI with the new data.
class PlotWorker(QtCore.QThread):
def __init__(self, plotwindow):
self.plotwindow = plotwindow
def run(self):
self.plotwindow.plot()
class PlotWindow(Ui_PlotWindow, QtWidgets.QMainWindow):
update_signal = pyqtSignal()
def __init__(self):
self.setupUi(self)
self.settings = self.settings_lineEdit.text()
self.worker = PlotWorker(self)
self.update_settings_pushButton.clicked.connect(self.update_settings)
self.refresh_timer = QtCore.QTimer(self)
self.refresh_timer.timeout.connect(self.update)
self.update_settings()
self.plot()
self.refresh_time.start(1)
def update_setting(self):
self.settings = self.settings_lineEdit.text()
def update(self)
if self.worker.started():
print('Already updating, cannot update until previous update complete')
elif not self.worker.started():
self.worker.start()
def plot(self):
# Plot the data, slow. Uses self.settings
self.load()
...
def load(self):
# Load the data from the appropriate directory/file, slow
...
I am creating a new thread to separate the UI interface from the data processing logic., but for reasons completely unfathomable to me, when I build and run my app , the app promptly crashes with the exception:
File "capnp/lib/capnp.pyx", line 2150, in capnp.lib.capnp._DynamicCapabilityClient._send
capnp.lib.capnp.KjException: src/kj/async.c++:53: failed: expected loop != nullptr; No event loop is running on this thread.
stack: 0x7f9bd4774489 0x7f9bd477845d 0x7f9bd4c0aacc 0x7f9bd49a81cb 0x7f9bd49a8207 0x7f9bd49a8304 0x7f9bd54d0341 0x7f9bd4edb93e 0x7f9bd4edba96 0x7f9bd4ec305e 0x7f9bd4ec43e9 0x7f9bd4eb0651 0x7f9bd54b5ad8 0x7f9bd5485844 0x55b92dc620c5 0x55b92dd55107
I'm not sure what in the build system could cause this runtime error,here is my code ,My best guess is some kind of arcane linking quirk, but I have no idea what or how to fix it.
import sys
import os
from PyQt5.QtCore import *
from PyQt5.QtGui import QGuiApplication
from cnc import Cnc
cnc = Cnc(os.path.join(os.getenv("HOME"), "cnc"), "192.168.7.98")
class workthread(QThread):
def __init__(self):
super(workthread, self).__init__()
def run(self):
self.cnc_axis_names = {axis.id: axis.name.absolute for axis in cnc.axes}
self.coors = {self.cnc_axis_names[id_]: coor for id_, coor in cnc.axes.coors().items()}
data = list(self.coors.values())
print(data)
self.sleep(1)
self.run()
self.exic_()
if __name__ == '__main__':
thread1 = workthread()
thread1.start()
app = QGuiApplication(sys.argv)
sys.exit(app.exec_())
Any advice on how to troubleshoot this is welcome.
I don't know what cnc is, but my guess is that it uses Cap'n Proto RPC. Cap'n Proto RPC uses a single-threaded event loop concurrency model, so RPC objects originally created on one thread cannot be manipulated on another thread.
In your program, you are creating the cnc object at startup, in the main thread. You then create a new thread and try to access the object from there. This won't work. You need to construct and use the object on a single thread.
I wrote a PyQt5 GUI (Python 3.5 on Win7). While it is running, its GUI is not responsive. To still be able to use the GUI, I tried to use QThread from this answer: https://stackoverflow.com/a/6789205/5877928
The module is now designed like this:
class MeasureThread(QThread):
def __init(self):
super().__init__()
def get_data(self):
data = auto.data
def run(self):
time.sleep(600)
# measure some things
with open(outputfile) as f:
write(things)
class Automation(QMainWindow):
[...]
def measure(self):
thread = MeasureThread()
thread.finished.connect(app.exit)
for line in open(inputfile):
thread.get_data()
thread.start()
measure() gets called once per measurement but starts the thread once per line in inputfile. The module now exits almost immediately after starting it (I guess it runs all thread at once and does not sleep) but I only want it to do all the measurements in another single thread so the GUI can still be accessed.
I also tried to apply this to my module, but could not connect the methods used there to my methods: http://www.xyzlang.com/python/PyQT5/pyqt_multithreading.html
The way I used to use the module:
class Automation(QMainWindow):
[...]
def measure(self):
param1, param2 = (1,2)
for line in open(inputfile):
self.measureValues(param1, param2)
def measureValues(self, param1, param2):
time.sleep(600)
# measure some things
with open(outputfile) as f:
write(things)
But that obviously used only one thread. Can you help me to find the right method to use here( QThread, QRunnable) or to map the example of the second link to my module?
Thanks!
I have a loading widget that consists of two labels, one is the status label and the other one is the label that the animated gif will be shown in. If I call show() method before heavy stuff gets processed, the gif at the loading widget doesn't update itself at all. There's nothing wrong with the gif btw(looping problems etc.). The main code(caller) looks like this:
self.loadingwidget = LoadingWidgetForm()
self.setCentralWidget(self.loadingwidget)
self.loadingwidget.show()
...
...
heavy stuff
...
...
self.loadingwidget.hide()
The widget class:
class LoadingWidgetForm(QWidget, LoadingWidget):
def __init__(self, parent=None):
super().__init__(parent=parent)
self.setupUi(self)
self.setWindowFlags(self.windowFlags() | Qt.FramelessWindowHint)
self.setAttribute(Qt.WA_TranslucentBackground)
pince_directory = SysUtils.get_current_script_directory() # returns current working directory
self.movie = QMovie(pince_directory + "/media/loading_widget_gondola.gif", QByteArray())
self.label_Animated.setMovie(self.movie)
self.movie.setScaledSize(QSize(50, 50))
self.movie.setCacheMode(QMovie.CacheAll)
self.movie.setSpeed(100)
self.movie.start()
self.not_finished=True
self.update_thread = Thread(target=self.update_widget)
self.update_thread.daemon = True
def showEvent(self, QShowEvent):
QApplication.processEvents()
self.update_thread.start()
def hideEvent(self, QHideEvent):
self.not_finished = False
def update_widget(self):
while self.not_finished:
QApplication.processEvents()
As you see I tried to create a seperate thread to avoid workload but it didn't make any difference. Then I tried my luck with the QThread class by overriding the run() method but it also didn't work. But executing QApplication.processEvents() method inside of the heavy stuff works well. I also think I shouldn't be using seperate threads, I feel like there should be a more elegant way to do this. The widget looks like this btw:
Processing...
Full version of the gif:
Thanks in advance! Have a good day.
Edit: I can't move the heavy stuff to a different thread due to bugs in pexpect. Pexpect's spawn() method requires spawned object and any operations related with the spawned object to be in the same thread. I don't want to change the working flow of the whole program
In order to update GUI animations, the main Qt loop (located in the main GUI thread) has to be running and processing events. The Qt event loop can only process a single event at a time, however because handling these events typically takes a very short time control is returned rapidly to the loop. This allows the GUI updates (repaints, including animation etc.) to appear smooth.
A common example is having a button to initiate loading of a file. The button press creates an event which is handled, and passed off to your code (either via events directly, or via signals). Now the main thread is in your long-running code, and the event loop is stalled — and will stay stalled until the long-running job (e.g. file load) is complete.
You're correct that you can solve this with threads, but you've gone about it backwards. You want to put your long-running code in a thread (not your call to processEvents). In fact, calling (or interacting with) the GUI from another thread is a recipe for a crash.
The simplest way to work with threads is to use QRunner and QThreadPool. This allows for multiple execution threads. The following wall of code gives you a custom Worker class that makes it simple to handle this. I normally put this in a file threads.py to keep it out of the way:
import sys
from PyQt5.QtCore import QObject, QRunnable
class WorkerSignals(QObject):
'''
Defines the signals available from a running worker thread.
error
`tuple` (exctype, value, traceback.format_exc() )
result
`dict` data returned from processing
'''
finished = pyqtSignal()
error = pyqtSignal(tuple)
result = pyqtSignal(dict)
class Worker(QRunnable):
'''
Worker thread
Inherits from QRunnable to handler worker thread setup, signals and wrap-up.
:param callback: The function callback to run on this worker thread. Supplied args and
kwargs will be passed through to the runner.
:type callback: function
:param args: Arguments to pass to the callback function
:param kwargs: Keywords to pass to the callback function
'''
def __init__(self, fn, *args, **kwargs):
super(Worker, self).__init__()
# Store constructor arguments (re-used for processing)
self.fn = fn
self.args = args
self.kwargs = kwargs
self.signals = WorkerSignals()
#pyqtSlot()
def run(self):
'''
Initialise the runner function with passed args, kwargs.
'''
# Retrieve args/kwargs here; and fire processing using them
try:
result = self.fn(*self.args, **self.kwargs)
except:
traceback.print_exc()
exctype, value = sys.exc_info()[:2]
self.signals.error.emit((exctype, value, traceback.format_exc()))
else:
self.signals.result.emit(result) # Return the result of the processing
finally:
self.signals.finished.emit() # Done
To use the above, you need a QThreadPool to handle the threads. You only need to create this once, for example during application initialisation.
threadpool = QThreadPool()
Now, create a worker by passing in the Python function to execute:
from .threads import Worker # our custom worker Class
worker = Worker(fn=<Python function>) # create a Worker object
Now attach signals to get back the result, or be notified of an error:
worker.signals.error.connect(<Python function error handler>)
worker.signals.result.connect(<Python function result handler>)
Then, to execute this Worker, you can just pass it to the QThreadPool.
threadpool.start(worker)
Everything will take care of itself, with the result of the work returned to the connected signal... and the main GUI loop will be free to do it's thing!