I am creating a new thread to separate the UI interface from the data processing logic., but for reasons completely unfathomable to me, when I build and run my app , the app promptly crashes with the exception:
File "capnp/lib/capnp.pyx", line 2150, in capnp.lib.capnp._DynamicCapabilityClient._send
capnp.lib.capnp.KjException: src/kj/async.c++:53: failed: expected loop != nullptr; No event loop is running on this thread.
stack: 0x7f9bd4774489 0x7f9bd477845d 0x7f9bd4c0aacc 0x7f9bd49a81cb 0x7f9bd49a8207 0x7f9bd49a8304 0x7f9bd54d0341 0x7f9bd4edb93e 0x7f9bd4edba96 0x7f9bd4ec305e 0x7f9bd4ec43e9 0x7f9bd4eb0651 0x7f9bd54b5ad8 0x7f9bd5485844 0x55b92dc620c5 0x55b92dd55107
I'm not sure what in the build system could cause this runtime error，here is my code ,My best guess is some kind of arcane linking quirk, but I have no idea what or how to fix it.
import sys
import os
from PyQt5.QtCore import *
from PyQt5.QtGui import QGuiApplication
from cnc import Cnc
cnc = Cnc(os.path.join(os.getenv("HOME"), "cnc"), "192.168.7.98")
class workthread(QThread):
def __init__(self):
super(workthread, self).__init__()
def run(self):
self.cnc_axis_names = {axis.id: axis.name.absolute for axis in cnc.axes}
self.coors = {self.cnc_axis_names[id_]: coor for id_, coor in cnc.axes.coors().items()}
data = list(self.coors.values())
print(data)
self.sleep(1)
self.run()
self.exic_()
if __name__ == '__main__':
thread1 = workthread()
thread1.start()
app = QGuiApplication(sys.argv)
sys.exit(app.exec_())
Any advice on how to troubleshoot this is welcome.
I don't know what cnc is, but my guess is that it uses Cap'n Proto RPC. Cap'n Proto RPC uses a single-threaded event loop concurrency model, so RPC objects originally created on one thread cannot be manipulated on another thread.
In your program, you are creating the cnc object at startup, in the main thread. You then create a new thread and try to access the object from there. This won't work. You need to construct and use the object on a single thread.
Related
I have a GUI in PyQt with a function addImage(image_path). Easy to imagine, it is called when a new image should be added into a QListWidget. For the detection of new images in a folder, I use a threading.Thread with watchdog to detect file changes in the folder, and this thread then calls addImage directly.
This yields the warning that QPixmap shouldn't be called outside the gui thread, for reasons of thread safety.
What is the best and most simple way to make this threadsafe? QThread? Signal / Slot? QMetaObject.invokeMethod? I only need to pass a string from the thread to addImage.
You should use the built in QThread provided by Qt. You can place your file monitoring code inside a worker class that inherits from QObject so that it can use the Qt Signal/Slot system to pass messages between threads.
class FileMonitor(QObject):
image_signal = QtCore.pyqtSignal(str)
#QtCore.pyqtSlot()
def monitor_images(self):
# I'm guessing this is an infinite while loop that monitors files
while True:
if file_has_changed:
self.image_signal.emit('/path/to/image/file.jpg')
class MyWidget(QtGui.QWidget):
def __init__(self, ...)
...
self.file_monitor = FileMonitor()
self.thread = QtCore.QThread(self)
self.file_monitor.image_signal.connect(self.image_callback)
self.file_monitor.moveToThread(self.thread)
self.thread.started.connect(self.file_monitor.monitor_images)
self.thread.start()
#QtCore.pyqtSlot(str)
def image_callback(self, filepath):
pixmap = QtGui.QPixmap(filepath)
...
I believe the best approach is using the signal/slot mechanism. Here is an example. (Note: see the EDIT below that points out a possible weakness in my approach).
from PyQt4 import QtGui
from PyQt4 import QtCore
# Create the class 'Communicate'. The instance
# from this class shall be used later on for the
# signal/slot mechanism.
class Communicate(QtCore.QObject):
myGUI_signal = QtCore.pyqtSignal(str)
''' End class '''
# Define the function 'myThread'. This function is the so-called
# 'target function' when you create and start your new Thread.
# In other words, this is the function that will run in your new thread.
# 'myThread' expects one argument: the callback function name. That should
# be a function inside your GUI.
def myThread(callbackFunc):
# Setup the signal-slot mechanism.
mySrc = Communicate()
mySrc.myGUI_signal.connect(callbackFunc)
# Endless loop. You typically want the thread
# to run forever.
while(True):
# Do something useful here.
msgForGui = 'This is a message to send to the GUI'
mySrc.myGUI_signal.emit(msgForGui)
# So now the 'callbackFunc' is called, and is fed with 'msgForGui'
# as parameter. That is what you want. You just sent a message to
# your GUI application! - Note: I suppose here that 'callbackFunc'
# is one of the functions in your GUI.
# This procedure is thread safe.
''' End while '''
''' End myThread '''
In your GUI application code, you should create the new Thread, give it the right callback function, and make it run.
from PyQt4 import QtGui
from PyQt4 import QtCore
import sys
import os
# This is the main window from my GUI
class CustomMainWindow(QtGui.QMainWindow):
def __init__(self):
super(CustomMainWindow, self).__init__()
self.setGeometry(300, 300, 2500, 1500)
self.setWindowTitle("my first window")
# ...
self.startTheThread()
''''''
def theCallbackFunc(self, msg):
print('the thread has sent this message to the GUI:')
print(msg)
print('---------')
''''''
def startTheThread(self):
# Create the new thread. The target function is 'myThread'. The
# function we created in the beginning.
t = threading.Thread(name = 'myThread', target = myThread, args = (self.theCallbackFunc))
t.start()
''''''
''' End CustomMainWindow '''
# This is the startup code.
if __name__== '__main__':
app = QtGui.QApplication(sys.argv)
QtGui.QApplication.setStyle(QtGui.QStyleFactory.create('Plastique'))
myGUI = CustomMainWindow()
sys.exit(app.exec_())
''' End Main '''
EDIT
Mr. three_pineapples and Mr. Brendan Abel pointed out a weakness in my approach. Indeed, the approach works fine for this particular case, because you generate / emit the signal directly. When you deal with built-in Qt signals on buttons and widgets, you should take another approach (as specified in the answer of Mr. Brendan Abel).
Mr. three_pineapples adviced me to start a new topic in StackOverflow to make a comparison between the several approaches of thread-safe communication with a GUI. I will dig into the matter, and do that tomorrow :-)
I have a python3 program that starts a second thread (besides the main thread) for handling some events asynchronously. Ideally, my program works without a flaw and never has an unhandled exceptions. But stuff happens. When/if there is an exception, I want the whole interpreter to exit with an error code as if it had been a single thread. Is that possible?
Right now, if an exception occurs on the spawned thread, it prints out the usual error information, but doesn't exit. The main thread just keeps going.
Example
import threading
import time
def countdown(initial):
while True:
print(initial[0])
initial = initial[1:]
time.sleep(1)
if __name__ == '__main__':
helper = threading.Thread(target=countdown, args=['failsoon'])
helper.start()
time.sleep(0.5)
#countdown('THISWILLTAKELONGERTOFAILBECAUSEITSMOREDATA')
countdown('FAST')
The countdown will eventually fail to access [0] from the string because it's been emptied causing an IndexError: string index out of range error. The goal is that whether the main or helper dies first, the whole program dies alltogether, but the stack trace info is still output.
Solutions Tried
After some digging, my thought was to use sys.excepthook. I added the following:
def killAll(etype, value, tb):
print('KILL ALL')
traceback.print_exception(etype, value, tb)
os.kill(os.getpid(), signal.SIGKILL)
sys.excepthook = killAll
This works if the main thread is the one that dies first. But in the other case it does not. This seems to be a known issue (https://bugs.python.org/issue1230540). I will try some of the workarounds there.
While the example shows a main thread and a helper thread which I created, I'm interested in the general case where I may be running someone else's library that launches a thread.
Well, you could simply raise an error in your thread and have the main thread handle and report that error. From there you could even terminate the program.
For example on your worker thread:
try:
self.result = self.do_something_dangerous()
except Exception as e:
import sys
self.exc_info = sys.exc_info()
and on main thread:
if self.exc_info:
raise self.exc_info[1].with_traceback(self.exc_info[2])
return self.result
So to give you a more complete picture, your code might look like this:
import threading
class ExcThread(threading.Thread):
def excRun(self):
pass
#Where your core program will run
def run(self):
self.exc = None
try:
# Possibly throws an exception
self.excRun()
except:
import sys
self.exc = sys.exc_info()
# Save details of the exception thrown
# DON'T rethrow,
# just complete the function such as storing
# variables or states as needed
def join(self):
threading.Thread.join(self)
if self.exc:
msg = "Thread '%s' threw an exception: %s" % (self.getName(), self.exc[1])
new_exc = Exception(msg)
raise new_exc.with_traceback(self.exc[2])
(I added an extra line to keep track of which thread is causing the error in case you have multiple threads, it's also good practice to name them)
My solution ended up being a happy marriage between the solution posted here and the SIGKILL solution piece from above. I added the following killall.py submodule to my package:
import threading
import sys
import traceback
import os
import signal
def sendKillSignal(etype, value, tb):
print('KILL ALL')
traceback.print_exception(etype, value, tb)
os.kill(os.getpid(), signal.SIGKILL)
original_init = threading.Thread.__init__
def patched_init(self, *args, **kwargs):
print("thread init'ed")
original_init(self, *args, **kwargs)
original_run = self.run
def patched_run(*args, **kw):
try:
original_run(*args, **kw)
except:
sys.excepthook(*sys.exc_info())
self.run = patched_run
def install():
sys.excepthook = sendKillSignal
threading.Thread.__init__ = patched_init
And then ran the install right away before any other threads are launched (of my own creation or from other imported libraries).
Just wanted to share my simple solution.
In my case I wanted the exception to display as normal but then immediately stop the program. I was able to accomplish this by starting a timer thread with a small delay to call os._exit before raising the exception.
import os
import threading
def raise_and_exit(args):
threading.Timer(0.01, os._exit, args=(1,)).start()
raise args[0]
threading.excepthook = raise_and_exit
Python 3.8 added threading.excepthook which makes it possible to handle this more cleanly.
I wrote the package "unhandled_exit" to do just that. It basically adds os._exit(1) to after the default handler. This means you get the normal backtrace before the process exits.
Package is published to pypi here: https://pypi.org/project/unhandled_exit/
Code is here: https://github.com/rfjakob/unhandled_exit/blob/master/unhandled_exit/\_\_init__.py
Usage is simply:
import unhandled_exit
unhandled_exit.activate()
I wrote a PyQt5 GUI (Python 3.5 on Win7). While it is running, its GUI is not responsive. To still be able to use the GUI, I tried to use QThread from this answer: https://stackoverflow.com/a/6789205/5877928
The module is now designed like this:
class MeasureThread(QThread):
def __init(self):
super().__init__()
def get_data(self):
data = auto.data
def run(self):
time.sleep(600)
# measure some things
with open(outputfile) as f:
write(things)
class Automation(QMainWindow):
[...]
def measure(self):
thread = MeasureThread()
thread.finished.connect(app.exit)
for line in open(inputfile):
thread.get_data()
thread.start()
measure() gets called once per measurement but starts the thread once per line in inputfile. The module now exits almost immediately after starting it (I guess it runs all thread at once and does not sleep) but I only want it to do all the measurements in another single thread so the GUI can still be accessed.
I also tried to apply this to my module, but could not connect the methods used there to my methods: http://www.xyzlang.com/python/PyQT5/pyqt_multithreading.html
The way I used to use the module:
class Automation(QMainWindow):
[...]
def measure(self):
param1, param2 = (1,2)
for line in open(inputfile):
self.measureValues(param1, param2)
def measureValues(self, param1, param2):
time.sleep(600)
# measure some things
with open(outputfile) as f:
write(things)
But that obviously used only one thread. Can you help me to find the right method to use here( QThread, QRunnable) or to map the example of the second link to my module?
Thanks!
In my PyQt4-based program, QSliders (with signals sliderMoved and sliderReleased connected to callables) sometimes "freeze", i.e. they don't move anymore when trying to drag them with the mouse, even though sliderMoved and sliderReleased are still emitted.
This behaviour happens seemingly randomly, sometimes after running the program for hours -- making it more or less impossible to reproduce and test.
Any help to solve this issue would be welcome.
EDIT: This is with PyQt 4.10.4 on Python 3.4 and Windows 7.
After some debugging I am pretty sure that this was due to calling a GUI slot from a separate thread, which (I knew) is forbidden. Fixing this to use a proper signal-slot approach seems to have fixed the issue.
After calling the patch function defined below, all slot calls are wrapped by a wrapper that checks that they are called only from the GUI thread -- a warning is printed otherwise. This is how I found the culprit.
import functools
import sys
import threading
import traceback
from PyQt4.QtCore import QMetaMethod
from PyQt4.QtGui import QWidget
SLOT_CACHE = {}
def patch():
"""Check for calls to widget slots outside of the main thread.
"""
qwidget_getattribute = QWidget.__getattribute__
def getattribute(obj, name):
attr = qwidget_getattribute(obj, name)
if type(obj) not in SLOT_CACHE:
meta = qwidget_getattribute(obj, "metaObject")()
SLOT_CACHE[type(obj)] = [
method.signature().split("(", 1)[0]
for method in map(meta.method, range(meta.methodCount()))
if method.methodType() == QMetaMethod.Slot]
if (isinstance(attr, type(print)) and # Wrap builtin functions only.
attr.__name__ in SLOT_CACHE[type(obj)]):
#functools.wraps(
attr, assigned=functools.WRAPPER_ASSIGNMENTS + ("__self__",))
def wrapper(*args, **kwargs):
if threading.current_thread() is not threading.main_thread():
print("{}.{} was called out of main thread:".format(
type(obj), name), file=sys.stderr)
traceback.print_stack()
return attr(*args, **kwargs)
return wrapper
else:
return attr
QWidget.__getattribute__ = getattribute
I wish to query a SQL database via QSqlQueryModel (PyqQt 5/Qt 5.2) asynchronously, so that the GUI doesn't block. How can this be accomplished? Maybe through multithreading? Please provide code of how to do this. If using QSqlQueryModel asynchronously isn't practical, feel free to provide alternatives (should be usable with QTableView though).
My (synchronous) code currently looks as shown beneath. The main script bin/app.py loads gui/__init__.py and executes its main method. That in turn uses gui.models.Table to load data from the database. The problem is that gui.models.Table queries the database synchronously and locks up the GUI in the meantime.
bin/app.py:
import os.path
import sys
sys.path.insert(0, os.path.abspath(os.path.join(
os.path.dirname(__file__), "..")))
import gui
if __name__ == "__main__":
gui.main()
gui/__init__.py:
import sys
import os.path
from PyQt5 import uic
from PyQt5 import QtCore, QtWidgets
from gui import models
class MainWindow(QtWidgets.QMainWindow):
def __init__(self):
super(MainWindow, self).__init__()
uic.loadUi(os.path.join(os.path.dirname(__file__), 'app.ui'), self)
self.tableView.setModel(models.Table(self))
def main():
app = QtWidgets.QApplication(sys.argv)
w = MainWindow()
w.show()
app.exec_()
gui/models.py:
import os.path
from PyQt5.QtCore import *
from PyQt5.QtSql import *
class Table(QSqlQueryModel):
def __init__(self, parent=None):
super(Table, self).__init__(parent)
pth = os.path.abspath(os.path.join(os.path.dirname(__file__), "..",
"test.sqlite"))
db = QSqlDatabase.addDatabase("QSQLITE")
db.setDatabaseName(pth)
if not db.open():
raise Exception("Couldn't open database '{}'".format(pth))
try:
self.setQuery("select * from Test")
finally:
db.close()
Unfortunately, a typical database driver that Qt (or anyone else, really) uses is synchronous. Qt views unfortunately don't know how to deal with models in foreign threads.
The solution thus requires a shim proxy model, subclassing QIdentityProxyModel. The first step in the implementation is to shim all of the source model's method calls with blocking QMetaObject::invokeMethod calls. This is needed just to be correct, if not asynchronous just yet. It' just to expose a safe interface to a model that lives in another thread.
The next step is to provide an asynchronous veneer over some of the functionality. Suppose that you want to make the data method asynchronous. What you do is:
For each role, have a cache of variant values keyed by the model index.
On the dataChanged signal from the source model, cache all the values that were changed, across all roles. The data call needs to be queued in the model's thread - more on that later.
In data, if there's a cache hit, return it. Otherwise return a null variant and queue the data call in the model's thread.
Your proxy should have a private method called cacheData that will be called from the queued calls. In another answer, I've detailed how to queue functor calls in another thread. Leveraging that, your data call queuing method can look like:
void ThreadsafeProxyModel::queueDataCall(const QModelIndex & index, int role) {
int row = index.row();
int column = index.column();
void * data = index.internalPointer();
postMetacall(sourceModel()->thread(), [this, row, column, data, role]{
QVariant data = sourceModel()->data(createIndex(row, column, data), role);
QMetaObject::invoke(this, "cacheData",
Q_ARG(QVariant, data), Q_ARG(int, role),
Q_ARG(int, row), Q_ARG(int, column), Q_ARG(void*, data));
});
}
This is just a sketch. It'd be fairly involved, but certainly doable, and still maintaining the semantics of a real model.