I'm using the following template to recreate threads that I need to run into infinity.
I want to know if this template is scalable in terms of memory. Are threaded destroyed properly?
import threading
import time
class aLazyThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
def run(self):
time.sleep(10)
print "I don not want to work :("
class aWorkerThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
def run(self):
time.sleep(1)
print "I want to work!!!!!!!"
threadA = aLazyThread()
threadA.start()
threadB = aWorkerThread()
threadB.start()
while True:
if not (threadA.isAlive()):
threadA = aLazyThread()
threadA.start()
if not (threadB.isAlive()):
threadB = aWorkerThread()
threadB.start()
The thing that bother me is the following picture taking in eclipse which show debug info, and It seems that thread are stacking it.
I see nothing wrong with the image. There's the main thread and the 2 threads that you created (according to the code, 3 threads are supposed to be running at any time)
Like any other Python objects, threads are garbage collected when they're not used; e.g. in your main while cycle, when you instantiate the class (let's say aLazyThread), the old threadA value is destroyed (maybe not exactly at that point, but shortly after)
The main while cycle, could also use a sleep (e.g. time.sleep(1)), otherwise it will consume the processor, uselessly checking if the other threads are running.
Related
I am trying to display a loading gif on my PyQt5 QMainWindow while an intensive process is running. Rather than playing normally, the QMovie pauses. As far as I can tell, the event loop shouldn't be blocked as the intensive process is in its own QObject passed to its own QThread. Relevant code below:
QMainWindow:
class EclipseQa(QMainWindow):
def __init__(self):
QMainWindow.__init__(self)
self.initUI()
def initUI(self):
...
self.loadingMovie = QMovie("./loading.gif")
self.loadingMovie.setScaledSize(QSize(149, 43))
self.statusLbl = QLabel(self)
self.statusLbl.setMovie(self.loadingMovie)
self.grid.addWidget(self.statusLbl, 6, 2, 2, 2, alignment=Qt.AlignCenter)
self.statusLbl.hide()
...
def startLoadingGif(self):
self.statusLbl.show()
self.loadingMovie.start()
def stopLoadingGif(self):
self.loadingMovie.stop()
self.statusLbl.hide()
def maskDose(self):
self.startLoadingGif()
# Set up thread and associated worker object
self.thread = QThread()
self.worker = DcmReadWorker()
self.worker.moveToThread(self.thread)
self.worker.finished.connect(self.thread.quit)
self.worker.updateRd.connect(self.updateRd)
self.worker.updateRs.connect(self.updateRs)
self.worker.updateStructures.connect(self.updateStructures)
self.worker.clearRd.connect(self.clearRd)
self.worker.clearRs.connect(self.clearRs)
self.thread.started.connect(lambda: self.worker.dcmRead(caption, fname[0]))
self.thread.finished.connect(self.stopLoadingGif)
self.maskThread.start()
def showDoneDialog(self):
...
self.stopLoadingGif()
...
Worker class:
class DoseMaskWorker(QObject):
clearRd = pyqtSignal()
clearRs = pyqtSignal()
finished = pyqtSignal()
startLoadingGif = pyqtSignal()
stopLoadingGif = pyqtSignal()
updateMaskedRd = pyqtSignal(str)
def __init__(self, parent=None):
QObject.__init__(self, parent)
#pyqtSlot(name="maskDose")
def maskDose(self, rd, rdName, rdId, rs, maskingStructure_dict):
...
self.updateMaskedRd.emit(maskedRdName)
self.finished.emit()
For brevity, '...' indicates code that I figured is probably not relevant.
Your use of lambda to call the slot when the threads started signal is emitted is likely causing it to execute in the main thread. There are a couple of things you need to do to fix this.
Firstly, your use of pyqtSlot does not contain the types of the arguments to the maskDose method. You need to update it so that it does. Presumably you also need to do this for the dcmRead method which you call from the lambda but haven't included in your code. See the documentation for more details.
In order to remove the use of the lambda, you need to define a new signal and a new slot within the EclipseQa class. The new signal should be defined such that the required number of parameters for the dcmRead method are emitted, and the types correctly specified (documentation for this is also in the link above). This signal should be connected to the workers dcmRead slot (make sure to do it after the worker object has been moved to the thread or else you might run into this bug!). The slot should take no arguments, and be connected to the threads started signal. The code in the slot should simply emit your new signal with the appropriate arguments to be passed to dcmRead (e.g. like self.my_new_signal.emit(param1, param2)).
Note: You can check what thread any code is running in using the Python threading module (even when using QThreads) by printing threading.current_thread().name from the location you wish to check.
Note 2: If your thread is CPU bound rather than IO bound, you may still experience performance issues because of the Python GIL which only allows a single thread to execute at any one time (it will swap between the threads regularly though so code in both threads should run, just maybe not at the performance you expect). QThreads (which are implemented in C++ and are theoretically able to release the GIL) do not help with this because they are running your Python code and so the GIL is still held.
I have a background thread that main calls, the background thread can open a number of different scripts but occasionally it will get an infinite print loop like this.
In thing.py
import foo
def main():
thr = Thread(target=background)
thr.start()
thread_list.append(thr)
def background():
getattr(foo, 'bar')()
return
And then in foo.py
def bar():
while True:
print("stuff")
This is what it's supposed to do but I want to be able to kill it when I need to. Is there a way for me to kill the background thread and all the functions it has called? I've tried putting flags in background to return when the flag goes high, but background is never able to check the flags since its waiting for bar to return.
EDIT: foo.py is not my code so I'm hesitant to edit it, ideally I could do this without modifying foo.py but if its impossible to avoid its okay
First of all it is very difficult (if possible) to control threads from other threads, no matter what language you are using. This is due to potential security issues. So what you do is you create a shared object which both threads can freely access. You can set a flag on it.
But luckily in Python each thread has its own Thread object which we can use:
import foo
def main():
thr = Thread(target=background)
thr.exit_requested = False
thr.start()
thread_list.append(thr)
def background():
getattr(foo, 'bar')()
return
And in foo:
import threading
def bar():
th = threading.current_thread()
# What happens when bar() is called from the main thread?
# The commented code is not thread safe.
# if not hasattr(th, 'exit_requested'):
# th.exit_requested = False
while not th.exit_requested:
print("stuff")
Although this will probably be hard to maintain/debug. Treat it more like a hack. Cleaner way would be to create a shared object and pass it around to all calls.
I'm trying to utilize threading and queueing (based on a recommendation) to pause the main process.
My program basically iterates through images, opening and closing them utilizing a 3-second time-loop for each iteration.
I'm trying to use threading to interject a time.sleep(20) if a certain condition is met (x == True). The condition is being met (evident by the output of the print statement), but time.sleep(20) is not affecting the main process.
I plan to subsitute time.sleep(20) with a more complex process but for simpliclity I've used it here.
import time
import subprocess
import pickle
import keyboard
import threading
from threading import Thread
import multiprocessing
import queue
import time
with open('C:\\Users\Moondra\\Bioteck.pickle', 'rb') as file:
bio = pickle.load(file)
q = queue.LifoQueue(0)
def keyboard_press(): # This is just receiving boolean values based on key presses
while True:
q.put(keyboard.is_pressed('down'))
x = q.get()
print(x)
if x == True:
time.sleep(20)
t = Thread(target = keyboard_press, args= ())
t.start()
if __name__ == "__main__":
for i in bio[:5]:
p = subprocess.Popen(["C:\Program Files\IrfanView\i_view64.exe",'C:\\Users\Moondra\\Bioteck_charts\{}.png'.format(i)])
time.sleep(3)
p.kill()
So why isn't my thread affecting my main process?
Thank you.
Update:
So It seems I have to use flags and use flag as a global variable within my function. I would like to avoid using global but it's not working without globalizing flag within my function.
Second, I don't know how to restart the thread.
Once the thread returns the flag as false, the thread sort of just stalls.
I tried starting the thread again, with t.start, but I received the error:
RuntimeError: threads can only be started once
Here is updated code:
def keyboard_press():
while True:
global flag
q.put(keyboard.is_pressed('down'))
x = q.get()
print(x)
if x == True:
flag = False
#print('keyboard_flag is',flag)
return flag
if __name__ == "__main__":
flag = True
q = queue.LifoQueue(0)
t = Thread(target = keyboard_press, args= ())
t.start()
for i in bio[:5]:
p = subprocess.Popen(["C:\Program Files\IrfanView\i_view64.exe",'C:\\Users\Moondra\\Bioteck_charts\{}.png'.format(i)])
time.sleep(3)
print ('flag is',flag)
if flag == True:
p.kill()
else:
time.sleep(20)
p.kill()
flag = True
#t.start() #doesn't seem to work.
why isn't my thread affecting my main process?
Because you have not written any code to be executed by the keyboard_press() thread that would affect the main process.
It looks like you're trying to create a slide show that shows one image every three seconds, and you want it to pause for an extra twenty seconds when somebody presses a key. Is that right?
So, you've got one thread (the main thread) that runs the slide show, and you've got another that polls the keyboard, but your two threads don't communicate with one another.
You put a time.sleep(20) call in your keyboard thread. But that only pauses the keyboard thread. It doesn't do anything at all to the main thread.
What you need, is for the keyboard thread to set a variable that the main thread looks at after it wakes up from its three second sleep. The main thread can look at the variable, and see if a longer sleep has been requested, and if so, sleep for twenty more seconds.
Of course, after the longer sleep, you will want the main thread to re-set the variable so that it won't always sleep for twenty seconds after the first time the keyboard is touched.
P.S.: I am not a Python expert. I know that in other programming environments (e.g., Java), you also have to worry about "memory visibility." That is, when a variable is changed by one thread, there is no guarantee of when (if ever) some other thread will see the change...
...Unless, the threads use some kind of synchronization when they access the variable.
Based on what I have read (It's on the Internet! It must be true!), Python either does not have that problem now, or it did not have that problem in the recent past. I'm not sure which.
If memory consistency actually is an issue, then you will either have to use a mutex when you access the shared variable, or else you will have to make the threads communicate through some kind of a synchronized object such as a queue.
I'm trying to create a thread for a GUI that wraps a long-running function. My problem is thus phrased in terms of PyQt and QThreads, but I imagine the same concept could apply to standard python threads too, and would appreciate any suggestions generally.
Typically, to allow a thread to be exited while running, I understand that including a "wants_to_end" flag that is periodically checked within the thread is a good practice - e.g.:
Pseudocode (in my thread):
def run(self):
i = 0
while (not self.wants_to_end) and (i < 100):
function_step(i) # where this is some long-running function that includes many streps
i += 1
However, as my GUI is to wrap a pre-written long-running function, I cannot simply insert such a "wants_to_end" flag poll into the long running code.
Is there another way to forcibly terminate my worker thread from my main GUI (i.e. enabling me to include a button in the GUI to stop the processing)?
My simple example case is:
class Worker(QObject):
finished = pyqtSignal()
def __init__(self, parent=None, **kwargs):
super().__init__(parent)
self.kwargs = kwargs
#pyqtSlot()
def run(self):
result = SomeLongComplicatedProcess(**self.kwargs)
self.finished.emit(result)
with usage within my MainWindow GUI:
self.thread = QThread()
self.worker = Worker(arg_a=1, arg_b=2)
self.worker.finished.connect(self.doSomethingInGUI)
self.worker.moveToThread(self.thread)
self.thread.started.connect(self.worker.run)
self.thread.start()
If the long-running function blocks, the only way to forcibly stop the thread is via its terminate() method (it may also be necessary to call wait() as well). However, there is no guarantee that this will always work, and the docs also state the following:
Warning: This function is dangerous and its use is discouraged. The
thread can be terminated at any point in its code path. Threads can be
terminated while modifying data. There is no chance for the thread to
clean up after itself, unlock any held mutexes, etc. In short, use
this function only if absolutely necessary.
A much cleaner solution is to use a separate process, rather than a separate thread. In python, this could mean using the multiprocessing module. But if you aren't familiar with that, it might be simpler to run the function as a script via QProcess (which provides signals that should allow easier integration with your GUI). You can then simply kill() the worker process whenever necessary. However, if that solution is somehow unsatisfactory, there are many other IPC approaches that might better suit your requirements.
Python 3.4, I'm trying to make a server using the websockets module (I was previously using regular sockets but wanted to make a javascript client) when I ran into an issue (because it expects async, at least if the examples are to be trusted, which I didn't use before). Threading simply does not work. If I run the following code, bar will never be printed, whereas if I comment out the line with yield from, it works as expected. So yield is probably doing something I don't quite understand, but why is it never even executed? Should I install python 3.5?
import threading
class SampleThread(threading.Thread):
def __init__(self):
super(SampleThread, self).__init__()
print("foo")
def run(self):
print("bar")
yield from var2
thread = SampleThread()
thread.start()
This is not the correct way to handle multithreading. run is neither a generator nor a coroutine. It should be noted that the asyncio event loop is only defined for the main thread. Any call to asyncio.get_event_loop() in a new thread (without first setting it with asyncio.set_event_loop() will throw an exception.
Before looking at running the event loop in a new thread, you should first analyze to see if you really need the event loop running in its own thread. It has a built-in thread pool executor at: loop.run_in_executor(). This will take a pool from concurrent.futures (either a ThreadPoolExecutor or a ProcessPoolExecutor) and provides a non-blocking way of running processes and threads directly from the loop object. As such, these can be await-ed (with Python3.5 syntax)
That being said, if you want to run your event loop from another thread, you can do it thustly:
import asyncio
class LoopThread(threading.Thread):
def __init__(self):
self.loop = asyncio.new_event_loop()
def run():
ayncio.set_event_loop(self.loop)
self.loop.run_forever()
def stop():
self.loop.call_soon_threadsafe(self.loop.stop)
From here, you still need to device a thread-safe way of creating tasks, etc. Some of the code in this thread is usable, although I did not have a lot of success with it: python asyncio, how to create and cancel tasks from another thread