I'm writing a UI wrapper for reading some info using esptool.py
I have two active threads: UI and procesing - SerialReader.
UI class has reference to the SerialReader and should stop SerialReader when it gets the exit command.
The problem is that I call esptool command which gets stuck in trying to read data over serial connection.
class SerialReaderProcess(threading.Thread):
def __init__(self, window):
super().__init__()
self.window = window
self.logger = window.logger
self.window.set_thread(self)
self._stop_event = threading.Event()
def run(self):
...
#read chip id
esptool.main(['chip_id'])
...
def stop(self):
self._stop_event.set()
def stopped(self):
return self._stop_event.is_set()
What I want is to kill all active process of this program. When I call close the UI and call serialReaderProcess.stop() it doesn't stop the process. I can see the output of esptool on the console.
I don't care if I interrupt anything, no data can be corrupted.
I've tried sys.exit(0) to no avail.
I've researched the problem but couldn't find a solution.
The OS is Ubuntu and I don't care about cross-platform features, but they would be nice
First import os library:
Import os
Then you can write the following code in your exit_event method:
def closeEvent(self, event):
output,errors = p1.communicate()
bashCommand = "killall python3"
sudoPassword = 'your password'
p = os.system('echo %s|sudo -S %s' % (sudoPassword, bashCommand))
As stated in comments, setting the thread as Daemon solved the problem:
super().__init__(daemon=True)
Daemon threads are automatically killed when the program quits.
More about daemons:
Daemon Threads Explanation
Related
I'm trying this simple thread with a while loop inside. When I'm inside the while loop, Ctrl+C has no effect in stopping my program. Once I go do something else after the while loop, the script stops as intended. What can I do so my script can be gracefully killed both while being in the while loop and after? (Edit: This seems to be a problem exclusive to Windows, iOS and Ubuntu seem to do what I want)
import time, threading
class MainClass(threading.Thread):
def __init__(self):
super().__init__()
def run(self):
while True:
time.sleep(1)
print("Looping")
# Script entry point
if __name__ == '__main__':
a = MainClass()
a.daemon = True
a.start()
a.join()
This is a known issue, explained in Issue 35935.
A way to solve it is to revert to the default kernel behaviour of SIGINT using signal.signal(signal.SIGINT, signal.SIG_DFL), (solution pointed out by the issue OP). As to why this has to be the case is beyond the scope of my knowledge.
This works on Windows using Python 3.8+.
Implementation:
import time, threading
import signal
class MainClass(threading.Thread):
def __init__(self):
super().__init__()
def run(self):
while True:
time.sleep(1)
print("Looping")
# Script entry point
if __name__ == '__main__':
a = MainClass()
a.daemon=True
signal.signal(signal.SIGINT, signal.SIG_DFL)
a.start()
a.join()
My tkinter app windows are not closing properly. I am using python 3.6.6 with tkinter 8.6
My code does basically this:
Open a process, where:
A test function is called via a thread that closes Gui window after 3s
Gui window is created
Wait for thread to complete (join) and guess window was closed
I tried to use:
quit -> window only closes when i hover my mouse over it
destroy -> destroy does not return
I stripped it down to the following code, please copy & execute and/or tell me whats wrong...
from time import sleep, time
import threading
from multiprocessing import Process, set_start_method
from tkinter import *
CtrlApplObj = None
def Start():
global CtrlApplObj
CtrlApplObj = None
CtrlApplObj = ControlApplication()
CtrlApplObj.run()
def End():
print("Quit now...")
#CtrlApplObj.root.destroy()
CtrlApplObj.root.quit()
class ControlApplication():
def __init__(self):
pass
def run(self):
self.root=Tk()
print("Mainloop...")
self.root.mainloop()
def test():
sleep(3)
End()
def execute():
T1 = threading.Thread(target=test)
T1.start()
Start()
T1.join()
if __name__ == "__main__":
set_start_method("spawn")
for i in range(2):
TestProcess = Process(target=execute)
TestProcess.start()
TestProcess.join()
My final solution was not using any tkinter operations in test thread. Then destroy worked.
I had another problem with my test process not closing. This was because a queue was not empty. That porevented process to close.
I wrote simple program which has pyQt interface with 2 buttons (start and cancel). Start button runs some calculations in the background (by starting update function) and thanks to threading I can still use UI.
But the application crashes after 10sec - 2 minutes. UI just dissapears, program shutdown.
when I use pythonw to run app without console thread crashes after ~25 sec but gui still works.
#!/usr/bin/python
import threading
import sys
from PyQt4 import QtGui, QtCore
import time
import os
class Class(QtGui.QWidget):
def __init__(self):
#Some init variables
self.initUI()
def initUI(self):
#some UI
self.show()
def update(self,stop_event):
while True and not stop_event.isSet():
self.updateSpeed()
self.updateDistance()
self.printLogs()
self.saveCSV()
self.guiUpdate()
time.sleep(1)
#gui button function
def initiate(self):
self.stop_event = threading.Event()
self.c_thread = threading.Thread(target = self.update, args=(self.stop_event,))
self.c_thread.start()
#Also gui button function
def cancelTracking(self):
self.stop_event.set()
self.close()
def main():
app = QtGui.QApplication(sys.argv)
ex = Class()
sys.exit(app.exec_())
ex.update()
if __name__ == '__main__':
main()
I dont know if I'm doing threading right. I found example like this on stack. I'm quite new to python and I'm using threading for the first time.
It is most likely due to calling a GUI function in your separate thread. PyQt GUI calls like setText() on a QLineEdit are not allowed from a thread. Anything that has PyQt painting outside of the main thread will not work. One way to get around this is to have your thread emit a signal to update the GUI when data is ready. The other way is to have a timer periodically checking for new data and updating the paintEvent after a certain time.
========== EDIT ==========
To Fix this issue I created a library named qt_thread_updater. https://github.com/justengel/qt_thread_updater This works by continuously running a QTimer. When you call call_latest the QTimer will run the function in the main thread.
from qt_thread_updater import get_updater
lbl = QtWidgets.QLabel('Value: 1')
counter = {'a': 1}
def run_thread():
while True:
text = 'Value: {}'.format(counter['a'])
get_updater().call_latest(lbl.setText, text)
counter['a'] += 1
time.sleep(0.1)
th = threading.Thread(target=run_thread)
th.start()
========== END EDIT ==========
#!/usr/bin/python
import threading
import sys
from PyQt4 import QtGui, QtCore
import time
import os
class Class(QtGui.QWidget):
display_update = QtCore.pyqtSignal() # ADDED
def __init__(self):
#Some init variables
self.initUI()
def initUI(self):
#some UI
self.display_update.connect(self.guiUpdate) # ADDED
self.show()
def update(self):
while True and not self.stop_event.isSet():
self.updateSpeed()
self.updateDistance()
self.printLogs()
self.saveCSV()
# self.guiUpdate()
self.display_update.emit() # ADDED
time.sleep(1)
#gui button function
def initiate(self):
self.stop_event = threading.Event()
self.c_thread = threading.Thread(target = self.update)
self.c_thread.start()
#Also gui button function
def cancelTracking(self):
self.stop_event.set()
self.close()
def main():
app = QtGui.QApplication(sys.argv)
ex = Class()
sys.exit(app.exec_())
# ex.update() # - this does nothing
if __name__ == '__main__':
main()
The other thing that could be happening is deadlock from two threads trying to access the same variable. I've read that this shouldn't be possible in python, but I have experienced it from the combination of PySide and other Python C extension libraries.
May also want to join the thread on close or use the QtGui.QApplication.aboutToQuit signal to join the thread before the program closes.
The Qt documentation for QThreads provides two popular patterns for using threading. You can either subclass QThread (the old way), or you can use the Worker Model, where you create a custom QObject with your worker functions and run them in a separate QThread.
In either case, you can't directly update the GUI from the background thread, so in your update function, the guiUpdate call will most likely crash Qt if it tries to change any of the GUI elements.
The proper way to run background processes is to use one of the two QThread patterns and communicate with the main GUI thread via Signals and Slots.
Also, in the following bit of code,
app = QtGui.QApplication(sys.argv)
ex = Class()
sys.exit(app.exec_())
ex.update()
app.exec_ starts the event loop and will block until Qt exits. Python won't run the ex.update() command until Qt has exited and the ex window has already been deleted, so you should just delete that command.
I am trying to build an .app for my pyside gui and I'm having a problem when I use signals to communicate between the main thread and another one. When I run the python code, everything works fine. The problem only occurs when I use the .app built from pyinstaller.
The project is quite big, so it's hard to include any code, so i'll try to explain what I'm doing.
My second thread is used to call some functions from a dylib which controls our devices. The main thread uses a callback to tell the dylib to stop what it's doing. The dylib calls the callback function in a loop, and stops if the value returned from the main thread is different than 0.
As I mentioned, when I run the Python code (under Windows, Ubuntu, Mac), it works perfectly. But when using the .app built with pyinstaller, it looks like the signal is not sent from the main thread. While debugging, I printed stuff and eventually saw that the dylib doesn't receive the returned value from the callback. The dylib does use the callback, and I can see the expected data in the main thread.
So why would the signal not be fired with frozen python code?
Has anybody ever encounter a similar problem?
If you could give me some debugging advices, it would be really helpful?
EDIT
I managed to write a !little code which reproduces the problem. When I run the python code, pressing "ok" followed by "Cancel" sets the message "DLL got the value" in the widget. When freezing the code with Pyinstaller, under mac, the message never gets set on the widget. On Windows 7, everything is fine.
EDIT
Actually, no need for the shared library to cause the problem. The problem occurs only with the Python code.
Here is the Python code:
import sys
from PySide import QtGui
from PySide.QtCore import QThread, Signal, QObject
def functionInDLL(callback):
return_value = 0
while (return_value == 0):
return_value = callback(2)
class MyThread(QThread):
str_signal = Signal(str)
def __init__(self):
QThread.__init__(self)
self.return_value = 0
def returnValueToDll(self, data=0):
return self.return_value
def run(self):
if functionInDLL(self.returnValueToDll) == 42:
print"DLL got the value."
self.str_signal.emit("DLL got the value.")
else:
print"DLL did not get the value."
self.str_signal.emit("DLL did not get the value.")
self.exec_()
def setValueSentToDll(self, data):
self.return_value = data
class MainThreadClass(QObject):
int_signal = Signal(int)
def __init__(self):
super(MainThreadClass, self).__init__()
self.test_thread = MyThread()
self.int_signal.connect(self.test_thread.setValueSentToDll)
def startMyThread(self):
self.test_thread.start()
def sendStopValueToDLL(self):
self.int_signal.emit(42)
class MyGUI(QtGui.QWidget):
def __init__(self):
super(MyGUI, self).__init__()
self.text_information = QtGui.QLabel(text="No started yet...")
self.dll_information = QtGui.QLabel(text="")
self.ok_button = QtGui.QPushButton("OK")
self.cancel_button = QtGui.QPushButton("Cancel")
self.close_button = QtGui.QPushButton("Close")
label_hbox = QtGui.QHBoxLayout()
hbox = QtGui.QHBoxLayout()
hbox.addWidget(self.ok_button)
hbox.addWidget(self.cancel_button)
hbox.addWidget(self.close_button)
vbox = QtGui.QVBoxLayout()
label_hbox.addWidget(self.text_information)
label_hbox.addWidget(self.dll_information)
vbox.addLayout(label_hbox)
vbox.addLayout(hbox)
self.setLayout(vbox)
self.ok_button.clicked.connect(self.onOk)
self.cancel_button.clicked.connect(self.onCancel)
self.close_button.clicked.connect(self.onClose)
self.show()
self.main_thread = MainThreadClass()
self.main_thread.test_thread.str_signal.connect(lambda data: self.setDllStatusOnWidget(data))
def onOk(self):
self.main_thread.startMyThread()
self.text_information.setText("Started")
self.dll_information.setText("")
def onCancel(self):
self.main_thread.sendStopValueToDLL()
self.text_information.setText("Canceled")
def onClose(self):
self.main_thread.test_thread.exit()
self.close()
def setDllStatusOnWidget(self, text=""):
self.dll_information.setText(text)
if __name__ == '__main__':
app = QtGui.QApplication(sys.argv)
ex = MyGUI()
sys.exit(app.exec_())
Thanks
Using Python2.7, Mac Pro Yosemite
Thanks to codewarrior, my bug was solved.
When I replaced the line:
self.main_thread.test_thread.str_signal.connect(lambda data: self.setDllStatusOnWidget(data))
by this line, removing the lambda:
self.main_thread.test_thread.str_signal.connect(self.setDllStatusOnWidget)
See the link for more details. I hope this can help other people.
pyinstaller_issue
I'm curious, why the code below freezes. When I kill python3 interpreter, "cat" process remains as a zombie. I expect the subprocess will be terminated before main process finished.
When I send manually SIGTERM to cat /dev/zero, the process is correctly finished (almost immediately)
#!/usr/bin/env python3
import subprocess
import re
import os
import sys
import time
from PyQt4 import QtCore
class Command(QtCore.QThread):
# stateChanged = QtCore.pyqtSignal([bool])
def __init__(self):
QtCore.QThread.__init__(self)
self.__runned = False
self.__cmd = None
print("initialize")
def run(self):
self.__runned = True
self.__cmd = subprocess.Popen(["cat /dev/zero"], shell=True, stdout=subprocess.PIPE)
try:
while self.__runned:
print("reading via pipe")
buf = self.__cmd.stdout.readline()
print("Buffer:{}".format(buf))
except:
logging.warning("Can't read from subprocess (cat /dev/zero) via pipe")
finally:
print("terminating")
self.__cmd.terminate()
self.__cmd.kill()
def stop(self):
print("Command::stop stopping")
self.__runned = False
if self.__cmd:
self.__cmd.terminate()
self.__cmd.kill()
print("Command::stop stopped")
def exitApp():
command.stop()
time.sleep(1)
sys.exit(0)
if __name__ == "__main__":
app = QtCore.QCoreApplication(sys.argv)
command = Command()
# command.daemon = True
command.start()
timer = QtCore.QTimer()
QtCore.QObject.connect(timer, QtCore.SIGNAL("timeout()"), exitApp)
timer.start(2 * 1000)
sys.exit(app.exec_())
As you noted yourself, the reason for zombie is that signal is caught by shell and doesn't effect process created by it. However there is a way to kill shell and all processes created by it; you have to use process group feature. See How to terminate a python subprocess launched with shell=True Having said that if you can manage without shell=True that's always preferable - see my answer here.
I solved this problem in a different way, so here's the result:
I have to call subprocess.Popen with shell=False, because otherwise it creates 2 processes (shell and the process) and __cmd.kill() send signal to shell while "process" remains as a zombie