I have attempted to purloin some of the code shipped with IP in Action and, following issues, I have even gone to the lengths of reading the book!
I am getting the error 'expect Delegate, got Function' when I use the following code. FYI I am passing in a reference to a WPF textBox so I should have a dispatcher on my UI element
I have removed all of the threading pipe reading stuff just to leave 'test' code:
import System
import System.IO
import Avacta.Optim.Server.WebServices
import Avacta.Optim.Server.DataModel
import sys
import clr
import time
from System import Console
from System.Threading import Thread, ThreadStart
def SetDispatcher(ui_element):
global dispatcher # needed else "Exception: 'NoneType' object has no attribute 'BeginInvoke'"
dispatcher = ui_element.Dispatcher
def Dispatch(function, *args):
dispatcher.BeginInvoke(lambda *_: function(*args))
def GetDispatchFunction(function):
return lambda *args: Dispatch(function, *args)
class ListOutput:
def __init__(self, textbox):
self.textbox = textbox
def write(self, string):
Dispatch(self.addText, string) # error: "expect Delegate, got Function"
#self.addText(string) # ok works fine w-w/o dispatcher stuff
def addText(self, string):
textbox.AppendText(string)
if textbox != None:
listout = ListOutput(textbox)
sys.stdout = listout
SetDispatcher(textbox)
print "Define running"
#running = True
Thread.Sleep(0)
time.sleep(2)
print "Start The Comms Thread..."
#comms_t = Thread(ThreadStart(run_comms))
#comms_t.Start()
Thread.Sleep(0)
time.sleep(2)
Any clues appreciated.
AndyF.
Thanks to Dino Viehland
Changing my dispatcher code to call the dispatcher directly fixes this issue.
dispatcher.BeginInvoke(System.Action(lambda *_: function(*args)))
Unfortunately I no longer get real-time output from my print statments to my 'console' - it all appears when the script completes. Remove the dispatcher and it reverts to real-time...
There is a set of dispatcher static methods (extension methods) are provided by way of DispatcherExtensions which take an Action as the parameter.
The code sample below demonstrates the usage of the WPF dispatcher. More information is available here http://msdn.microsoft.com/en-us/library/cc647497.aspx
import clr
clr.AddReference('WindowsBase')
clr.AddReference('System.Windows.Presentation')
from System import Action
from System.Windows.Threading import DispatcherExtensions, Dispatcher
dispatcher = Dispatcher.CurrentDispatcher
def workCallBack():
print 'working'
DispatcherExtensions.BeginInvoke(dispatcher, Action(workCallBack))
Related
I have a question for pytest
I would like to run same pytest script with multiple threads.
But,i am not sure how to create and run thread which is passing more than one param. (And running thread with pytest..)
for example I have
test_web.py
from selenium import webdriver
import pytest
class SAMPLETEST:
self.browser = webdriver.Chrome()
self.browser.get(URL)
self.browser.maximize_window()
def test_title(self):
assert "Project WEB" in self.browser.title
def test_login(self):
print('Testing Login')
ID_BOX = self.broswer.find_element_by_id("ProjectemployeeId")
PW_BOX = self.broswer.find_element_by_id("projectpassword")
ID_BOX.send_keys(self.ID) # this place for ID. This param come from thread_run.py
PW_BOX.send_keys(self.PW) # this place for PW. It is not working. I am not sure how to get this data from threa_run.py
PW_BOX.submit()
IN thread_run.py
import threading
import time
from test_web import SAMPLETEST
ID_List = ["0","1","2","3","4","5","6","7"]
PW_LIST = ["0","1","2","3","4","5","6","7"]
threads = []
print("1: Create thread")
for I in range(8):
print("Append thread" + str(I))
t = threading.Thread(target=SAMPLETEST, args=(ID_List[I], PW_LIST[I]))
threads.append(t)
for I in range(8):
print("Start thread:" + str(I))
threads[I].start()
i was able to run thread to run many SAMPLETEST class without pytest.
However, it is not working with pytest.
My question is.
First, how to initialize self.brower in insde of SAMPLETEST? I am sure below codes will not be working
self.browser = webdriver.Chrome()
self.browser.get(URL)
self.browser.maximize_window()
Second, in thread_run.py, how can i pass the two arguments(ID and Password) when I run thread to call SAMPLTEST on test_web.py?
ID_BOX.send_keys(self.ID) # this place for ID. This param come from thread_run.py
ID_BOX.send_keys(self.ID)
PW_BOX.send_keys(self.PW)
I was trying to build constructor (init) in SAMPLETEST class but it wasn't working...
I am not really sure how to run threads (which passing arguments or parameter ) with pytest.
There are 2 scenarios which i could read from this:
prepare the test data and pass in parameters to your test method which could be achieved by pytest-generate-tests and parameterise concept. You can refer to the documentation here
In case of running pytest in multi threading - Pytest-xdist or pytest-parallel
I had a similar issue and got it resolved by passing an argument in form of a list.
Like:,
I replaced below line
thread_1 = Thread(target=fun1, args=10)
with
thread_1 = Thread(target=fun1, args=[10])
I am creating a new thread to separate the UI interface from the data processing logic., but for reasons completely unfathomable to me, when I build and run my app , the app promptly crashes with the exception:
File "capnp/lib/capnp.pyx", line 2150, in capnp.lib.capnp._DynamicCapabilityClient._send
capnp.lib.capnp.KjException: src/kj/async.c++:53: failed: expected loop != nullptr; No event loop is running on this thread.
stack: 0x7f9bd4774489 0x7f9bd477845d 0x7f9bd4c0aacc 0x7f9bd49a81cb 0x7f9bd49a8207 0x7f9bd49a8304 0x7f9bd54d0341 0x7f9bd4edb93e 0x7f9bd4edba96 0x7f9bd4ec305e 0x7f9bd4ec43e9 0x7f9bd4eb0651 0x7f9bd54b5ad8 0x7f9bd5485844 0x55b92dc620c5 0x55b92dd55107
I'm not sure what in the build system could cause this runtime error，here is my code ,My best guess is some kind of arcane linking quirk, but I have no idea what or how to fix it.
import sys
import os
from PyQt5.QtCore import *
from PyQt5.QtGui import QGuiApplication
from cnc import Cnc
cnc = Cnc(os.path.join(os.getenv("HOME"), "cnc"), "192.168.7.98")
class workthread(QThread):
def __init__(self):
super(workthread, self).__init__()
def run(self):
self.cnc_axis_names = {axis.id: axis.name.absolute for axis in cnc.axes}
self.coors = {self.cnc_axis_names[id_]: coor for id_, coor in cnc.axes.coors().items()}
data = list(self.coors.values())
print(data)
self.sleep(1)
self.run()
self.exic_()
if __name__ == '__main__':
thread1 = workthread()
thread1.start()
app = QGuiApplication(sys.argv)
sys.exit(app.exec_())
Any advice on how to troubleshoot this is welcome.
I don't know what cnc is, but my guess is that it uses Cap'n Proto RPC. Cap'n Proto RPC uses a single-threaded event loop concurrency model, so RPC objects originally created on one thread cannot be manipulated on another thread.
In your program, you are creating the cnc object at startup, in the main thread. You then create a new thread and try to access the object from there. This won't work. You need to construct and use the object on a single thread.
I have a GUI in PyQt with a function addImage(image_path). Easy to imagine, it is called when a new image should be added into a QListWidget. For the detection of new images in a folder, I use a threading.Thread with watchdog to detect file changes in the folder, and this thread then calls addImage directly.
This yields the warning that QPixmap shouldn't be called outside the gui thread, for reasons of thread safety.
What is the best and most simple way to make this threadsafe? QThread? Signal / Slot? QMetaObject.invokeMethod? I only need to pass a string from the thread to addImage.
You should use the built in QThread provided by Qt. You can place your file monitoring code inside a worker class that inherits from QObject so that it can use the Qt Signal/Slot system to pass messages between threads.
class FileMonitor(QObject):
image_signal = QtCore.pyqtSignal(str)
#QtCore.pyqtSlot()
def monitor_images(self):
# I'm guessing this is an infinite while loop that monitors files
while True:
if file_has_changed:
self.image_signal.emit('/path/to/image/file.jpg')
class MyWidget(QtGui.QWidget):
def __init__(self, ...)
...
self.file_monitor = FileMonitor()
self.thread = QtCore.QThread(self)
self.file_monitor.image_signal.connect(self.image_callback)
self.file_monitor.moveToThread(self.thread)
self.thread.started.connect(self.file_monitor.monitor_images)
self.thread.start()
#QtCore.pyqtSlot(str)
def image_callback(self, filepath):
pixmap = QtGui.QPixmap(filepath)
...
I believe the best approach is using the signal/slot mechanism. Here is an example. (Note: see the EDIT below that points out a possible weakness in my approach).
from PyQt4 import QtGui
from PyQt4 import QtCore
# Create the class 'Communicate'. The instance
# from this class shall be used later on for the
# signal/slot mechanism.
class Communicate(QtCore.QObject):
myGUI_signal = QtCore.pyqtSignal(str)
''' End class '''
# Define the function 'myThread'. This function is the so-called
# 'target function' when you create and start your new Thread.
# In other words, this is the function that will run in your new thread.
# 'myThread' expects one argument: the callback function name. That should
# be a function inside your GUI.
def myThread(callbackFunc):
# Setup the signal-slot mechanism.
mySrc = Communicate()
mySrc.myGUI_signal.connect(callbackFunc)
# Endless loop. You typically want the thread
# to run forever.
while(True):
# Do something useful here.
msgForGui = 'This is a message to send to the GUI'
mySrc.myGUI_signal.emit(msgForGui)
# So now the 'callbackFunc' is called, and is fed with 'msgForGui'
# as parameter. That is what you want. You just sent a message to
# your GUI application! - Note: I suppose here that 'callbackFunc'
# is one of the functions in your GUI.
# This procedure is thread safe.
''' End while '''
''' End myThread '''
In your GUI application code, you should create the new Thread, give it the right callback function, and make it run.
from PyQt4 import QtGui
from PyQt4 import QtCore
import sys
import os
# This is the main window from my GUI
class CustomMainWindow(QtGui.QMainWindow):
def __init__(self):
super(CustomMainWindow, self).__init__()
self.setGeometry(300, 300, 2500, 1500)
self.setWindowTitle("my first window")
# ...
self.startTheThread()
''''''
def theCallbackFunc(self, msg):
print('the thread has sent this message to the GUI:')
print(msg)
print('---------')
''''''
def startTheThread(self):
# Create the new thread. The target function is 'myThread'. The
# function we created in the beginning.
t = threading.Thread(name = 'myThread', target = myThread, args = (self.theCallbackFunc))
t.start()
''''''
''' End CustomMainWindow '''
# This is the startup code.
if __name__== '__main__':
app = QtGui.QApplication(sys.argv)
QtGui.QApplication.setStyle(QtGui.QStyleFactory.create('Plastique'))
myGUI = CustomMainWindow()
sys.exit(app.exec_())
''' End Main '''
EDIT
Mr. three_pineapples and Mr. Brendan Abel pointed out a weakness in my approach. Indeed, the approach works fine for this particular case, because you generate / emit the signal directly. When you deal with built-in Qt signals on buttons and widgets, you should take another approach (as specified in the answer of Mr. Brendan Abel).
Mr. three_pineapples adviced me to start a new topic in StackOverflow to make a comparison between the several approaches of thread-safe communication with a GUI. I will dig into the matter, and do that tomorrow :-)
I have a python3 program that starts a second thread (besides the main thread) for handling some events asynchronously. Ideally, my program works without a flaw and never has an unhandled exceptions. But stuff happens. When/if there is an exception, I want the whole interpreter to exit with an error code as if it had been a single thread. Is that possible?
Right now, if an exception occurs on the spawned thread, it prints out the usual error information, but doesn't exit. The main thread just keeps going.
Example
import threading
import time
def countdown(initial):
while True:
print(initial[0])
initial = initial[1:]
time.sleep(1)
if __name__ == '__main__':
helper = threading.Thread(target=countdown, args=['failsoon'])
helper.start()
time.sleep(0.5)
#countdown('THISWILLTAKELONGERTOFAILBECAUSEITSMOREDATA')
countdown('FAST')
The countdown will eventually fail to access [0] from the string because it's been emptied causing an IndexError: string index out of range error. The goal is that whether the main or helper dies first, the whole program dies alltogether, but the stack trace info is still output.
Solutions Tried
After some digging, my thought was to use sys.excepthook. I added the following:
def killAll(etype, value, tb):
print('KILL ALL')
traceback.print_exception(etype, value, tb)
os.kill(os.getpid(), signal.SIGKILL)
sys.excepthook = killAll
This works if the main thread is the one that dies first. But in the other case it does not. This seems to be a known issue (https://bugs.python.org/issue1230540). I will try some of the workarounds there.
While the example shows a main thread and a helper thread which I created, I'm interested in the general case where I may be running someone else's library that launches a thread.
Well, you could simply raise an error in your thread and have the main thread handle and report that error. From there you could even terminate the program.
For example on your worker thread:
try:
self.result = self.do_something_dangerous()
except Exception as e:
import sys
self.exc_info = sys.exc_info()
and on main thread:
if self.exc_info:
raise self.exc_info[1].with_traceback(self.exc_info[2])
return self.result
So to give you a more complete picture, your code might look like this:
import threading
class ExcThread(threading.Thread):
def excRun(self):
pass
#Where your core program will run
def run(self):
self.exc = None
try:
# Possibly throws an exception
self.excRun()
except:
import sys
self.exc = sys.exc_info()
# Save details of the exception thrown
# DON'T rethrow,
# just complete the function such as storing
# variables or states as needed
def join(self):
threading.Thread.join(self)
if self.exc:
msg = "Thread '%s' threw an exception: %s" % (self.getName(), self.exc[1])
new_exc = Exception(msg)
raise new_exc.with_traceback(self.exc[2])
(I added an extra line to keep track of which thread is causing the error in case you have multiple threads, it's also good practice to name them)
My solution ended up being a happy marriage between the solution posted here and the SIGKILL solution piece from above. I added the following killall.py submodule to my package:
import threading
import sys
import traceback
import os
import signal
def sendKillSignal(etype, value, tb):
print('KILL ALL')
traceback.print_exception(etype, value, tb)
os.kill(os.getpid(), signal.SIGKILL)
original_init = threading.Thread.__init__
def patched_init(self, *args, **kwargs):
print("thread init'ed")
original_init(self, *args, **kwargs)
original_run = self.run
def patched_run(*args, **kw):
try:
original_run(*args, **kw)
except:
sys.excepthook(*sys.exc_info())
self.run = patched_run
def install():
sys.excepthook = sendKillSignal
threading.Thread.__init__ = patched_init
And then ran the install right away before any other threads are launched (of my own creation or from other imported libraries).
Just wanted to share my simple solution.
In my case I wanted the exception to display as normal but then immediately stop the program. I was able to accomplish this by starting a timer thread with a small delay to call os._exit before raising the exception.
import os
import threading
def raise_and_exit(args):
threading.Timer(0.01, os._exit, args=(1,)).start()
raise args[0]
threading.excepthook = raise_and_exit
Python 3.8 added threading.excepthook which makes it possible to handle this more cleanly.
I wrote the package "unhandled_exit" to do just that. It basically adds os._exit(1) to after the default handler. This means you get the normal backtrace before the process exits.
Package is published to pypi here: https://pypi.org/project/unhandled_exit/
Code is here: https://github.com/rfjakob/unhandled_exit/blob/master/unhandled_exit/\_\_init__.py
Usage is simply:
import unhandled_exit
unhandled_exit.activate()
In my PyQt4-based program, QSliders (with signals sliderMoved and sliderReleased connected to callables) sometimes "freeze", i.e. they don't move anymore when trying to drag them with the mouse, even though sliderMoved and sliderReleased are still emitted.
This behaviour happens seemingly randomly, sometimes after running the program for hours -- making it more or less impossible to reproduce and test.
Any help to solve this issue would be welcome.
EDIT: This is with PyQt 4.10.4 on Python 3.4 and Windows 7.
After some debugging I am pretty sure that this was due to calling a GUI slot from a separate thread, which (I knew) is forbidden. Fixing this to use a proper signal-slot approach seems to have fixed the issue.
After calling the patch function defined below, all slot calls are wrapped by a wrapper that checks that they are called only from the GUI thread -- a warning is printed otherwise. This is how I found the culprit.
import functools
import sys
import threading
import traceback
from PyQt4.QtCore import QMetaMethod
from PyQt4.QtGui import QWidget
SLOT_CACHE = {}
def patch():
"""Check for calls to widget slots outside of the main thread.
"""
qwidget_getattribute = QWidget.__getattribute__
def getattribute(obj, name):
attr = qwidget_getattribute(obj, name)
if type(obj) not in SLOT_CACHE:
meta = qwidget_getattribute(obj, "metaObject")()
SLOT_CACHE[type(obj)] = [
method.signature().split("(", 1)[0]
for method in map(meta.method, range(meta.methodCount()))
if method.methodType() == QMetaMethod.Slot]
if (isinstance(attr, type(print)) and # Wrap builtin functions only.
attr.__name__ in SLOT_CACHE[type(obj)]):
#functools.wraps(
attr, assigned=functools.WRAPPER_ASSIGNMENTS + ("__self__",))
def wrapper(*args, **kwargs):
if threading.current_thread() is not threading.main_thread():
print("{}.{} was called out of main thread:".format(
type(obj), name), file=sys.stderr)
traceback.print_stack()
return attr(*args, **kwargs)
return wrapper
else:
return attr
QWidget.__getattribute__ = getattribute