I am creating a multiple threads to execute a function that generates PDF. This process takes a lot of time, so the user has a choice to cancel the execution.
To stop a thread, I know that I can use threading.Event() to check if it will be set. However, the process of the function I am executing in my event loop is straight forward/linear (There is no loop to check regularly if the Event is set).
--threading class--
def execute_function(self, function_to_execute, total_executions, execution_time, controller):
self.event = threading.Event()
self.event_list.append(self.event)
self.loop = asyncio.get_event_loop()
self.future = self.loop.run_in_executor(self._executor, function_to_execute, self.event, total_executions,
execution_time, controller)
def stop_executor(self):
for event in self.event_list:
event.set()
self.event = None
if self._executor:
self._executor.shutdown(wait=False)
def *function_to_execute*(self, event, total_execution, seconds=SECONDS_DEFAULT, controller=None):
self.event = event
self.controller = controller
...
My problem is that, I can't implement the Event to interrupt the threads without looping to regularly check the Event.
Is there any other way around to stop all those threads?
Or if I will still use the Event, is there any other logic to implement it?
Thanks in advance!
Related
I got a coroutine running to catch events sent by a footswitch device. This coroutine is launched from the main window as following.
class FootswitchMonitor(wx.Frame):
def __init__(self, parent, title):
super().__init__(parent, title=title,
size=(350, 150))
self.InitUI()
StartCoroutine(self.footswitch_callback, self)
self.Bind(EVT_FOOTSWITCH, self.pedal)
def on_footswitch(self, evt):
print(f"eid: {evt.GetId()} code: {evt.code} ")
async def footswitch_callback(self):
device = get_footswitch_device()
device.grab()
key_pressed = None
async for ev in device.async_read_loop():
if ev.type == 1 and ev.value == 1: # only key events and key_down
if key_pressed != ev.code:
event = footswitch_event(code=ev.code)
wx.PostEvent(self, event)
key_pressed = ev.code
else:
key_pressed = None
def setup_footswitch(self, event):
cDialog = ConfigDialog(None)
cDialog.ShowModal()
cDialog.Destroy()
In order to setup the device I use a ConfigDialog dialog box where I'd like to bind the same EVT_FOOTSWITCH.
class ConfigDialog(wx.Dialog):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.InitUI()
self.SetSize(350, 450)
self.SetTitle("Configuration")
self.Bind(EVT_FOOTSWITCH, self.footswitch_setup_callback)
And this is where it hurts. The main coroutine has FootswitchMonitor frame as target: wx.PostEvent(self, event). So the ConfigDialog windows won't get it.
I cannot run another coroutine with this device in the parent windows because it is grabed, which is locked by the main coroutine.
So my question is how can I catch the EVT_FOOTSWITCH in ConfigDialog a parent windows. Is it possible to post the event globaly? Is it possible to stop the main coroutine to run anotheone in the modal window?
I have something similar where I run reading the footpedal in a separate thread.
If the user wishes to reconfigure the pedal settings, I simply stop the thread, for the duration of the reconfiguration, then restart it.
Stopping the thread also ungrabs the device.
I do insist the program is restarted, as I only read the configuration of the footpedal's settings at program start. I found it less complicated this way.
Your other option, if you wish to attack it from a different angle, is to Unbind the event i.e. self.Unbind(EVT_FOOTSWITCH) and ungrab the device.
Then Bind it to your reconfiguration routine, which Unbind's when it finishes and finally rebind the event back to the original, whilst regrabbing the device, when you return to the main app.
My rather vast code can be perused at https://sourceforge.net/projects/footswitch2 , if you need further clues.
When browsing the file fs2.py search for PedalThread
Two coroutintes in code below, running in different threads, cannot communicate with each other by asyncio.Queue. After the producer inserts a new item in asyncio.Queue, the consumer cannot get this item from that asyncio.Queue, it gets blocked in method await self.n_queue.get().
I try to print the ids of asyncio.Queue in both consumer and producer, and I find that they are same.
import asyncio
import threading
import time
class Consumer:
def __init__(self):
self.n_queue = None
self._event = None
def run(self, loop):
loop.run_until_complete(asyncio.run(self.main()))
async def consume(self):
while True:
print("id of n_queue in consumer:", id(self.n_queue))
data = await self.n_queue.get()
print("get data ", data)
self.n_queue.task_done()
async def main(self):
loop = asyncio.get_running_loop()
self.n_queue = asyncio.Queue(loop=loop)
task = asyncio.create_task(self.consume())
await asyncio.gather(task)
async def produce(self):
print("id of queue in producer ", id(self.n_queue))
await self.n_queue.put("This is a notification from server")
class Producer:
def __init__(self, consumer, loop):
self._consumer = consumer
self._loop = loop
def start(self):
while True:
time.sleep(2)
self._loop.run_until_complete(self._consumer.produce())
if __name__ == '__main__':
loop = asyncio.get_event_loop()
print(id(loop))
consumer = Consumer()
threading.Thread(target=consumer.run, args=(loop,)).start()
producer = Producer(consumer, loop)
producer.start()
id of n_queue in consumer: 2255377743176
id of queue in producer 2255377743176
id of queue in producer 2255377743176
id of queue in producer 2255377743176
I try to debug step by step in asyncio.Queue, and I find after the method self._getters.append(getter) is invoked in asyncio.Queue, the item is inserted in queue self._getters. The following snippets are all from asyncio.Queue.
async def get(self):
"""Remove and return an item from the queue.
If queue is empty, wait until an item is available.
"""
while self.empty():
getter = self._loop.create_future()
self._getters.append(getter)
try:
await getter
except:
# ...
raise
return self.get_nowait()
When a new item is inserted into asycio.Queue in producer, the methods below would be invoked. The variable self._getters has no items although it has same id in methods put() and set().
def put_nowait(self, item):
"""Put an item into the queue without blocking.
If no free slot is immediately available, raise QueueFull.
"""
if self.full():
raise QueueFull
self._put(item)
self._unfinished_tasks += 1
self._finished.clear()
self._wakeup_next(self._getters)
def _wakeup_next(self, waiters):
# Wake up the next waiter (if any) that isn't cancelled.
while waiters:
waiter = waiters.popleft()
if not waiter.done():
waiter.set_result(None)
break
Does anyone know what's wrong with the demo code above? If the two coroutines are running in different threads, how could they communicate with each other by asyncio.Queue?
Short answer: no!
Because the asyncio.Queue needs to share the same event loop, but
An event loop runs in a thread (typically the main thread) and executes all callbacks and Tasks in its thread. While a Task is running in the event loop, no other Tasks can run in the same thread. When a Task executes an await expression, the running Task gets suspended, and the event loop executes the next Task.
see
https://docs.python.org/3/library/asyncio-dev.html#asyncio-multithreading
Even though you can pass the event loop to threads, it might be dangerous to mix the different concurrency concepts. Still note, that passing the loop just means that you can add tasks to the loop from different threads, but they will still be executed in the main thread. However, adding tasks from threads can lead to race conditions in the event loop, because
Almost all asyncio objects are not thread safe, which is typically not a problem unless there is code that works with them from outside of a Task or a callback. If there’s a need for such code to call a low-level asyncio API, the loop.call_soon_threadsafe() method should be used
see
https://docs.python.org/3/library/asyncio-dev.html#asyncio-multithreading
Typically, you should not need to run async functions in different threads, because they should be IO bound and therefore a single thread should be sufficient to handle the work load. If you still have some CPU bound tasks, you are able to dispatch them to different threads and make the result awaitable using asyncio.to_thread, see https://docs.python.org/3/library/asyncio-task.html#running-in-threads.
There are many questions already about this topic, see e.g. Send asyncio tasks to loop running in other thread or How to combine python asyncio with threads?
If you want to learn more about the concurrency concepts, I recommend to read https://medium.com/analytics-vidhya/asyncio-threading-and-multiprocessing-in-python-4f5ff6ca75e8
I'm writing a Python program to interact with a device based on a CAN Bus. I'm using the python-can module successfully for this purpose. I'm also using asyncio to react to asynchronous events. I have written a "CanBusManager" class that is used by the "CanBusSequencer" class. The "CanBusManager" class takes care of generating/sending/receiving messages, and the CanBusSequencer drives the sequence of messages to be sent.
At some point in the sequence I want to wait until a specific message is received to "unlock" the remaining messages to be sent in the sequence. Overview in code:
main.py
async def main():
event = asyncio.Event()
sequencer = CanBusSequencer(event)
task = asyncio.create_task(sequencer.doSequence())
await task
asyncio.run(main(), debug=True)
canBusSequencer.py
from canBusManager import CanBusManager
class CanBusSequencer:
def __init__(self, event)
self.event = event
self.canManager = CanBusManager(event)
async def doSequence(self):
for index, row in self.df_sequence.iterrows():
if:...
self.canManager.sendMsg(...)
else:
self.canManager.sendMsg(...)
await self.event.wait()
self.event.clear()
canBusManager.py
import can
class CanBusManager():
def __init__(self, event):
self.event = event
self.startListening()
**EDIT**
def startListening(self):
self.msgNotifier = can.Notifier(self.canBus, self.receivedMsgCallback)
**EDIT**
def receivedMsgCallback(self, msg):
if(msg == ...):
self.event.set()
For now my program stays by the await self.event.wait(), even though the relevant message is received and the self.event.set() is executed. Running the program with debug = True reveals an
RuntimeError: Non-thread-safe operation invoked on an event loop other than the current one
that I don't really get. It has to do with the asyncio event loop, somehow not properly defined/managed. I'm coming from the C++ world and I'm currently writing my first large program with Python. Any guidance would be really appreciated:)
Your question doesn't explain how you arrange for receivedMsgCallback to be invoked.
If it is invoked by a classic "async" API which uses threads behind the scenes, then it will be invoked from outside the thread that runs the event loop. According to the documentation, asyncio primitives are not thread-safe, so invoking event.set() from another thread doesn't properly synchronize with the running event loop, which is why your program doesn't wake up when it should.
If you want to do anything asyncio-related, such as invoke Event.set, from outside the event loop thread, you need to use call_soon_threadsafe or equivalent. For example:
def receivedMsgCallback(self, msg):
if msg == ...:
self.loop.call_soon_threadsafe(self.event.set)
The event loop object should be made available to the CanBusManager object, perhaps by passing it to its constructor and assigning it to self.loop.
On a side note, if you are creating a task only to await it immediately, you don't need a task in the first place. In other words, you can replace task = asyncio.create_task(sequencer.doSequence()); await task with the simpler await sequencer.doSequence().
i have in my app thread that append something to list and then i want to print it in other screen, but, the program run the thread after the print and it is give me error that there is no thing in my list. i am need to stop the program until the thread done, how can i do this? i tried to use .join() but it is didnt work... thanks for help
my app:
class LoginScreen(GridLayout):
def __init__(self, **kwargs):
super(LoginScreen, self).__init__(**kwargs)
self.cols = 2
self.add_widget(Label(text='username'))
self.username = TextInput(multiline=False)
self.add_widget(self.username)
self.add_widget(Label(text='Password'))
self.password = TextInput(multiline=False, password=True)
self.add_widget(self.password)
self.submit_button = Button(text='sumbit',size_hint=(.5,
.25),font_size=20)
self.submit_button.bind(on_press=self.submit_username)
self.add_widget(self.submit_button)
def submit_username(self, *args):
self.msg=threading.Thread(target=send_data(self.username.text))
self.msg.start()
self.msg.join()
sm.current = 'searchi'
sm.transition.direction = 'left'
def send_data(name):
my_socket = socket.socket()
my_socket.connect(('127.0.0.1', 8093))
my_socket.send(name.encode('utf-8'))
name,address = my_socket.recvfrom(1024)
msg = name.decode('utf-8')
alist.append(msg)
my_socket.close()
# Declare both screens
class Searchi(Screen):
def __init__(self, **kwargs):
super(Searchi, self).__init__(**kwargs)
self.add_widget(Label(text=alist[0]))
the list have to "msg" if i am delete the line"self.add_widget(Label(text=alist[0]))" then it is no problem in the recv line. i just need to wait until the thread is finish.
Why bother with a Thread if you are going to wait for it anyway? You could just call send_data(self.username.text) and be done with it.
But doing this is usually bad practice (doing blocking calls without a thread, or waiting for the thread to finish in a blocking way, which is equivalent), what you want, instead of waiting for the task to be done before proceeding, is to react to the task being done, that is, at the end of your thread, do something that will allow your app to proceed.
you could have a callback to move your user to the new screen, called at the end of the thread.
def submit_username(self, *args):
def callback():
sm.current = 'searchi'
sm.transition.direction = 'left'
threading.Thread(target=send_data(self.username.text, callback).start()
def send_data(name, callback):
my_socket = socket.socket()
my_socket.connect(('127.0.0.1', 8093))
my_socket.send(name.encode('utf-8'))
name,address = my_socket.recvfrom(1024)
msg = name.decode('utf-8')
alist.append(msg)
my_socket.close()
callback()
if you want your user to touch anything while the action happens, i would advise putting a Popup with auto_dismiss=False, and a content indicating that data is being processed, and to close it in the callback.
I have a loading widget that consists of two labels, one is the status label and the other one is the label that the animated gif will be shown in. If I call show() method before heavy stuff gets processed, the gif at the loading widget doesn't update itself at all. There's nothing wrong with the gif btw(looping problems etc.). The main code(caller) looks like this:
self.loadingwidget = LoadingWidgetForm()
self.setCentralWidget(self.loadingwidget)
self.loadingwidget.show()
...
...
heavy stuff
...
...
self.loadingwidget.hide()
The widget class:
class LoadingWidgetForm(QWidget, LoadingWidget):
def __init__(self, parent=None):
super().__init__(parent=parent)
self.setupUi(self)
self.setWindowFlags(self.windowFlags() | Qt.FramelessWindowHint)
self.setAttribute(Qt.WA_TranslucentBackground)
pince_directory = SysUtils.get_current_script_directory() # returns current working directory
self.movie = QMovie(pince_directory + "/media/loading_widget_gondola.gif", QByteArray())
self.label_Animated.setMovie(self.movie)
self.movie.setScaledSize(QSize(50, 50))
self.movie.setCacheMode(QMovie.CacheAll)
self.movie.setSpeed(100)
self.movie.start()
self.not_finished=True
self.update_thread = Thread(target=self.update_widget)
self.update_thread.daemon = True
def showEvent(self, QShowEvent):
QApplication.processEvents()
self.update_thread.start()
def hideEvent(self, QHideEvent):
self.not_finished = False
def update_widget(self):
while self.not_finished:
QApplication.processEvents()
As you see I tried to create a seperate thread to avoid workload but it didn't make any difference. Then I tried my luck with the QThread class by overriding the run() method but it also didn't work. But executing QApplication.processEvents() method inside of the heavy stuff works well. I also think I shouldn't be using seperate threads, I feel like there should be a more elegant way to do this. The widget looks like this btw:
Processing...
Full version of the gif:
Thanks in advance! Have a good day.
Edit: I can't move the heavy stuff to a different thread due to bugs in pexpect. Pexpect's spawn() method requires spawned object and any operations related with the spawned object to be in the same thread. I don't want to change the working flow of the whole program
In order to update GUI animations, the main Qt loop (located in the main GUI thread) has to be running and processing events. The Qt event loop can only process a single event at a time, however because handling these events typically takes a very short time control is returned rapidly to the loop. This allows the GUI updates (repaints, including animation etc.) to appear smooth.
A common example is having a button to initiate loading of a file. The button press creates an event which is handled, and passed off to your code (either via events directly, or via signals). Now the main thread is in your long-running code, and the event loop is stalled — and will stay stalled until the long-running job (e.g. file load) is complete.
You're correct that you can solve this with threads, but you've gone about it backwards. You want to put your long-running code in a thread (not your call to processEvents). In fact, calling (or interacting with) the GUI from another thread is a recipe for a crash.
The simplest way to work with threads is to use QRunner and QThreadPool. This allows for multiple execution threads. The following wall of code gives you a custom Worker class that makes it simple to handle this. I normally put this in a file threads.py to keep it out of the way:
import sys
from PyQt5.QtCore import QObject, QRunnable
class WorkerSignals(QObject):
'''
Defines the signals available from a running worker thread.
error
`tuple` (exctype, value, traceback.format_exc() )
result
`dict` data returned from processing
'''
finished = pyqtSignal()
error = pyqtSignal(tuple)
result = pyqtSignal(dict)
class Worker(QRunnable):
'''
Worker thread
Inherits from QRunnable to handler worker thread setup, signals and wrap-up.
:param callback: The function callback to run on this worker thread. Supplied args and
kwargs will be passed through to the runner.
:type callback: function
:param args: Arguments to pass to the callback function
:param kwargs: Keywords to pass to the callback function
'''
def __init__(self, fn, *args, **kwargs):
super(Worker, self).__init__()
# Store constructor arguments (re-used for processing)
self.fn = fn
self.args = args
self.kwargs = kwargs
self.signals = WorkerSignals()
#pyqtSlot()
def run(self):
'''
Initialise the runner function with passed args, kwargs.
'''
# Retrieve args/kwargs here; and fire processing using them
try:
result = self.fn(*self.args, **self.kwargs)
except:
traceback.print_exc()
exctype, value = sys.exc_info()[:2]
self.signals.error.emit((exctype, value, traceback.format_exc()))
else:
self.signals.result.emit(result) # Return the result of the processing
finally:
self.signals.finished.emit() # Done
To use the above, you need a QThreadPool to handle the threads. You only need to create this once, for example during application initialisation.
threadpool = QThreadPool()
Now, create a worker by passing in the Python function to execute:
from .threads import Worker # our custom worker Class
worker = Worker(fn=<Python function>) # create a Worker object
Now attach signals to get back the result, or be notified of an error:
worker.signals.error.connect(<Python function error handler>)
worker.signals.result.connect(<Python function result handler>)
Then, to execute this Worker, you can just pass it to the QThreadPool.
threadpool.start(worker)
Everything will take care of itself, with the result of the work returned to the connected signal... and the main GUI loop will be free to do it's thing!