I would like to execute a certain signal handler in a multi-threading python program.
I noted that my signal handler is called for main thread and the other thread (thread_1) that I have created.
Is there any way to execute the signal handler only for thread_1?
Or is there any way to know the thread ID of the thread that calls the signal handler? Is it the main thread or thread_1?
import signal
import sys
from multiprocessing import Process
from time import sleep
x = 3;
def f(name):
print 'hello', name
while(1):
sleep (1)
print "AAAAAAAAAAAAAAAAAAAAAAAAA"
def signal_handler(signal, frame):
global x
print 'You pressed Ctrl+C!'
print x
sys.exit(0)
if __name__ == '__main__':
signal.signal(signal.SIGINT, signal_handler)
print 'Creating thread'
p = Process(target=f, args=('bob',))
p.start()
print 'Press Ctrl+C'
x += 6
signal.pause()
print 'After press CTRL+C'
Got an output:
Creating thread
Press Ctrl+C
hello bob
AAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAA
^CYou pressed Ctrl+C!
You pressed Ctrl+C!
9
3
As you can signal handler is called twice (one for main thread and another for thread_1), and as I asked above, I would like to execute handler only for thread_1 or even find a way to get thread_id of the calling thread?
The signal is handled in two separate processes rather than two distinct threads of a single process. (Hint: you import multiprocessing rather than import threading.)
The child process inherits the signal handler for SIGINT and gets its own copy of x. Now, the shell runs both your parent and child processes in a foreground process group, and sends keyboard-generated signals (like Ctrl-C => SIGINT) to the whole process group, which is why both parent and child receive the signal. Both parent and child, then, print their own value of x.
If you do switch to a threaded implementation, then only the main thread will receive signals, and your question will be moot.
Related
I have a script that uses user input and based on that it generates some data (code below). The script uses threads so the GUI does not freeze while the background tasks are running.
My question is if it is possible to pause the worker thread, ask the user for some input, send it to the worker thread and then to continue the operation?
from tkinter.constants import LEFT, RIGHT, S
import tkinter.messagebox
import tkinter as tk, time, threading, random, queue
from tkinter import NORMAL, DISABLED, filedialog
class GuiPart(object):
def __init__(self, master, client_instance):
canvas = tk.Canvas(master, height=600, width=1020, bg="#263D42")
canvas.pack()
self.button1 = tk.Button(master, text="Mueller Matrix - MMP mode", padx=10,
pady=5, bd = 10, font=12, command=client_instance.start_thread1)
self.button1.pack(side = LEFT)
def files(self):
self.filena = filedialog.askopenfilenames(initialdir="/", title="Select File",
filetypes = (("comma separated file","*.csv"), ("all files", "*.*")))
return self.filena
def processing_done(self):
tkinter.messagebox.showinfo('Processing Done', 'Process Successfully Completed!')
return
class ThreadedClient(object):
def __init__(self, master):
self.running=True
self.gui = GuiPart(master, self)
def start_thread1(self):
thread1=threading.Thread(target=self.worker_thread1)
thread1.start()
def worker_thread1(self):
if self.running:
self.gui.button1.config(state=DISABLED) # can I disable the button from the main thread?
user_input = self.gui.files() # can I pause the Worker thread here so ask the user_input from main loop?
time.sleep(5) # to simulate a long-running process
self.gui.processing_done()
self.gui.button1.config(state=NORMAL)
root = tk.Tk()
root.title('test_GUI')
client = ThreadedClient(root)
root.mainloop()
I am asking this as from time to time my script ends abruptly saying that the GUI is not part of the main loop. Thank you for your help!
In general, you should run the GUI from the main thread.
There is some confusion whether tkinter is thread-safe. That is, if one can call tkinter functions and methods from any but the main thread. The documentation for Python 3 says “yes”. Comments in the C source code for tkinter say “its complicated” depending on how tcl is built.
You should be able to call tkinter functions and methods from additional threads in Python 3 if your tkinter was built with thread support. AFAIK, the tkinter that comes with the downloads from python.org have threading enabled.
If you want to be able to pause additional threads, you should create a global variable, e.g. pause = False.
The extra threads should regularly read the value of this and pause their calculations when set to true, e.g.:
def thread():
while run:
while pause:
time.sleep(0.2)
# do stuff
Meanwhile the GUI running in the main thread should be the only one to write the value of pause, e.g. in a widget callback.
Some sample callbacks:
def start_cb():
pause = False # So it doesn't pause straight away.
run = True
def stop_cb():
pause = False
run = False
def pause_cb():
pause = True
def continue_cb():
pause = False
If you have more complicated global data structures that can be modified be several threads, you should definitely guard access to those with a Lock.
Edit1
Threaded tkinter programs without classes:
Simple threaded tkinter example program
Real-world example for unlocking ms-excel files
Edit2:
From the python source code for _tkinter:
If Tcl is threaded, this approach won't work anymore. The Tcl
interpreter is only valid in the thread that created it, and all Tk
activity must happen in this thread, also. That means that the
mainloop must be invoked in the thread that created the
interpreter. Invoking commands from other threads is possible;
_tkinter will queue an event for the interpreter thread, which will
then execute the command and pass back the result. If the main
thread is not in the mainloop, and invoking commands causes an
exception; if the main loop is running but not processing events,
the command invocation will block.
You can read the whole comment here.
I'm facing problem with the thread concept i.e I have a function which will create 10 threads to do task. If any key board interruption occurs, those created threads are still executing and i would like to stop those threads and revert back the changes.
The following code sinppet is the sample approach:
def store_to_db(self,keys_size,master_key,action_flag,key_status):
for iteration in range(10):
t = threading.Thread(target=self.store_worker, args=())
t.start()
threads.append(t)
for t in threads:
t.join()
def store_worker():
print "DOING"
The idea to make this work is:
you need a "thread pool" where threads are checking against if their do_run attribute is falsy.
you need a "sentinel thread" outside that pool which checks the thread status in the pool and adjusts the do_run attribute of the "thread pool" thread on demand.
Example code:
import threading
import random
import time
import msvcrt as ms
def main_logic():
# take 10 worker threads
threads = []
for i in range(10):
t = threading.Thread(target=lengthy_process_with_brake, args=(i,))
# start and append
t.start()
threads.append(t)
# start the thread which allows you to stop all threads defined above
s = threading.Thread(target=sentinel, args=(threads,))
s.start()
# join worker threads
for t in threads:
t.join()
def sentinel(threads):
# this one runs until threads defined in "threads" are running or keyboard is pressed
while True:
# number of threads are running
running = [x for x in threads if x.isAlive()]
# if kb is pressed
if ms.kbhit():
# tell threads to stop
for t in running:
t.do_run = False
# if all threads stopped, exit the loop
if not running:
break
# you don't want a high cpu load for nothing
time.sleep(0.05)
def lengthy_process_with_brake(worker_id):
# grab current thread
t = threading.currentThread()
# start msg
print(f"{worker_id} STARTED")
# exit condition
zzz = random.random() * 20
stop_time = time.time() + zzz
# imagine an iteration here like "for item in items:"
while time.time() < stop_time:
# the brake
if not getattr(t, "do_run", True):
print(f"{worker_id} IS ESCAPING")
return
# the task
time.sleep(0.03)
# exit msg
print(f"{worker_id} DONE")
# exit msg
print(f"{worker_id} DONE")
main_logic()
This solution does not 'kill' threads, just tell them to stop iterating or whatever they do.
EDIT:
I just noticed that "Keyboard exception" was in the title and not "any key". Keyboard Exception handling is a bit different, here is a good solution for that. The point is almost the same: you tell the thread to return if a condition is met.
I have a following code:
import time
import asyncio
from concurrent.futures import ProcessPoolExecutor
def blocking_func(x):
print("In blocking waiting")
time.sleep(x) # Pretend this is expensive calculations
print("after blocking waiting")
return x * 5
#asyncio.coroutine
def main():
executor = ProcessPoolExecutor()
out = yield from loop.run_in_executor(executor, blocking_func, 2) # This does not
print("after process pool")
print(out)
if __name__ == "__main__":
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
Output:
In blocking waiting
after blocking waiting
after process pool
10
But I was expecting the process pool will run the code in different process. So I was expecting the output to be:
Expecting output:
In blocking waiting
after process pool
after blocking waiting
10
I thought if we run the code on process pool it would not block the main loop.But in the output it came back to the main event loop after it is done with the blocking function.
What is blocking the event loop? Is it the blocking_function? If it is the blocking_function what is the use of having the process pool?
yield from here means "wait for coroutine to complete and return its result". Comparing to Python threading API, it is like calling join().
To get desired result, use something like this:
#asyncio.coroutine
def main():
executor = ProcessPoolExecutor()
task = loop.run_in_executor(executor, blocking_func, 2)
# at this point your blocking func is already running
# in the executor process
print("after process pool")
out = yield from task
print(out)
Coroutines arent' t separate processes. The difference is that coroutines need to give up control to the loop by themselves. This means if you have a blocking coroutine then it will block the whole loop.
The reason you use coroutines is mainly to handle I/O activities. If you are waiting for a message you can simply check a socket and if nothing happens you will return to the main loop. Then other coroutines can be handled before finally the control comes back to the IO function.
In your case it makes sense to use await asyncio.sleep(x) instead of time.sleep(x). This way control is suspended from blocking_func() for the sleep time. Afterwards control goes back there and the result should be as you expected it.
More infos: https://docs.python.org/3/library/asyncio.html
So I want to learn using moveToThread and see the effect of calling onTimeout() of class GenericWorker from a different thread (main thread in this case). The weird thing is that the finish_sig in GenericWorker never gets emitted (should happen at the last line of onTimeout() ). Since it connects to terminate_thread() in Sender class, it should at least print out a terminate_thread in the console, but nothing happens at all.
My original purpose for using it is to emit a signal to quit the thread after onTimeout() is done. But now I can only do t.quit() from main to quit the thread.
Thank you all for spending time taking care of my question!
from PyQt4.QtCore import *
from PyQt4.QtGui import *
import threading
from time import sleep
import sys
class GenericWorker(QObject):
finish_sig = pyqtSignal() # this one never gets emitted!
#pyqtSlot(str, str)
def onTimeout(self, cmd1, cmd2):
print 'onTimeout get called from thread ID: '
print QThread.currentThreadId()
print 'received cmd 1: ' + cmd1
print 'received cmd 2: ' + cmd2
self.finish_sig.emit() # supposed to emit here!
class Sender(QObject):
send_sig = pyqtSignal(str, str)
terminate_sig = pyqtSignal()
def emit_sig(self, cmd):
print 'emit_sig thread ID: '
print QThread.currentThreadId()
sleep(1)
self.send_sig.emit(cmd, '2nd_cmd')
def terminate_thread(self):
print 'terminate_thread'
self.terminate_sig.emit()
if __name__ == "__main__":
app = QApplication(sys.argv)
print 'Main thread ID: '
print QThread.currentThreadId()
t = QThread()
my_worker = GenericWorker()
my_worker.moveToThread(t)
t.start()
my_sender = Sender()
my_sender.send_sig.connect(my_worker.onTimeout)
my_sender.terminate_sig.connect(t.quit)
my_worker.finish_sig.connect(my_sender.terminate_thread)
# my_worker.finish_sig.connect(t.quit)
my_sender.emit_sig('hello')
sleep(1)
# my_sender.terminate_thread()
# t.quit() # this one works
# t.wait()
exit(1)
sys.exit(app.exec_())
The output:
Main thread ID:
46965006517856
emit_sig thread ID:
46965006517856
onTimeout get called from thread ID:
1111861568
received cmd 1: hello
received cmd 2: 2nd_cmd
QThread: Destroyed while thread is still running
UPDATE:
After referring to #tmoreau and #ekhumoro's answers, there are two key problems with this code:
The exit(1) is not a proper way to exit, I need to remove this line.
I don't have a way to exit the QApplication, what I need to do is to add t.finish.connect(app.quit) to exit the application. (By the way, the last line sys.exit(app.exec_()) seems not taking care of the exiting of the QApplication)
In sum, there are basically three things that I need to exit: QThread, QApplication and sys, what I missed is to exit QApplication. Let me know if my understanding is right or not...
Your issue is that you exit the program before it's complete.
my_sender.emit_sig('hello')
sleep(1)
exit(1)
sys.exit(app.exec_())
exit() ends your program, even if the thread has not finished running, hence the error:
QThread: Destroyed while thread is still running
If you remove sleep(1), you'll see the program stops even earlier:
Main thread ID:
46965006517856
emit_sig thread ID:
46965006517856
QThread: Destroyed while thread is still running
Here's more or less what's happening in parallel:
# main thread #worker thread
my_sender.emit_sig('hello') #slot onTimeout is called
sleep(1) #print "onTimeout get called..."
exit(1) #emit finish_sig
sys.exit(app.exec_())
# slot terminate_thread is called #thread ends (t.quit)
If you remove exit(1), your program will work, because you create an event loop with app.exec_(). The event loop means your program is always waiting to catch signals, and will not stop even if there's nothing left to do. So the thread has plenty of time to end :)
In Qt, you usually stop the event loop by closing your main window. Therefore, a cleaner way to implement your thread is:
class window(QWidget):
def __init__(self,parent=None):
super(window,self).__init__(parent)
t=QThread(self)
self.my_worker = GenericWorker()
self.my_worker.moveToThread(t)
t.start()
self.my_sender = Sender()
self.my_sender.send_sig.connect(self.my_worker.onTimeout)
self.my_sender.terminate_sig.connect(t.quit)
self.my_worker.finish_sig.connect(self.my_sender.terminate_thread)
self.my_sender.emit_sig('hello')
if __name__ == "__main__":
app = QApplication(sys.argv)
win=window()
win.show()
sys.exit(app.exec_())
You need self to keep a reference to the thread and classes. Otherwise they are destroyed when __init__ ends.
I Am trying to send a signal from a child thread to the main thread in a multi-threaded program (cannot use multi-processes). Unfortunately even after exhausting all the reading materials available online (which I could find), I Am unable to get a clear idea of how to do so. I Am a beginner to signals AND to python so please bear with me and explain as you would to a novice.
I cannot use the join method in the process, since I want both the threads to be running simultaneously.
Here is the code that I found related to the topic here - http://pymotw.com/2/signal/#signals-and-threads
and it doesn't really work for me.
import signal
import threading
import os
import time
def signal_handler(num, stack):
print 'Received signal %d in %s' % (num, threading.currentThread())
signal.signal(signal.SIGUSR1, signal_handler)
def wait_for_signal():
print 'Waiting for signal in', threading.currentThread()
signal.pause()
print 'Done waiting'
# Start a thread that will not receive the signal
receiver = threading.Thread(target=wait_for_signal, name='receiver')
receiver.start()
time.sleep(0.1)
def send_signal():
print 'Sending signal in', threading.currentThread()
os.kill(os.getpid(), signal.SIGUSR1)
sender = threading.Thread(target=send_signal, name='sender')
sender.start()
sender.join()
# Wait for the thread to see the signal (not going to happen!)
print 'Waiting for', receiver
signal.alarm(2)
receiver.join()
Please explain with a multi-threaded example if possible.
Thanks in advance!
Signals and threads really, really don't play nice together.
Consider use an Event or other synchronization mechanism. The following example creates an 'event' object, then passes it to two threads. One waits for two seconds, then signals the other to print out a message then exit.
source
import threading, time
def flagger_thread(event):
"""
wait for two seconds, then make 'event' fire
"""
time.sleep(2)
event.set()
def waiter_thread(event):
print("Waiting for event")
if event.wait(5):
print("event set.")
else:
print("Timed out.")
stop_event = threading.Event()
threading.Thread(target=flagger_thread, args=[stop_event]).start()
threading.Thread(target=waiter_thread, args=[stop_event]).start()
# wait for all threads to exit
for t in threading.enumerate():
if t != threading.current_thread():
t.join()
output
Waiting for event
event set.