I have the following code in PyQT, it is doing its job properly. There is a readfile function in a QThread. I want to start a "loading" animation when it is starting to find the hostnames in the file. And I want to stop the "animation" when it went through of all of it. Hostnames comes by user input and they append in a queue ( the queue can contain multiple hostnames). What it is doing now opening and closing the file constantly and like by that the loading animation is poping up and closing rapdily. Sure there is a way to open the file once and cycle through all the hostnames instead to open it for every hostname in the queue.
Here is the code:
class Worker2(QThread):
found = Signal(str)
notFound = Signal(str)
animation = Signal()
def __init__(self):
QThread.__init__(self)
self.queue = Queue()
def run(self):
self.keepRunning = True
while self.keepRunning:
hostlist = self.queue.get()
if not self.keepRunning:
break
for hostname in hostlist.splitlines():
if not self.keepRunning:
break
if hostlist is None:
continue
self.animation.emit()
f = open("allhosts.txt", "r")
if hostname in f.read():
self.found.emit(hostname)
else:
self.notFound.emit(hostname)
self.finished.emit()
def stop(self):
self.keepRunning = False
self.queue.put(None)
def lookUp(self, hostname):
self.queue.put(hostname)
Related
I have below code, where I am using OpenCV to start webcam video. Along with that I also have a thread running that pings www.google.com to check network connectivity.
import time
import cv2
import os
from threading import Thread
stopThread = False
def CheckNetwork():
global stopThread
while True:
time.sleep(60)
host = "www.google.com"
response = os.system("ping " + host)
if response == 0:
print("Internet host reachable")
else:
print("Internet host not reachable")
if stopThread:
break
def main():
global stopThread
Thread(target=CheckNetwork).start()
cam = cv2.VideoCapture(0)
while True:
ret_val, img = cam.read()
cv2.imshow('Camera', img)
key = cv2.waitKey(1)
if key == ord('q'):
stopThread = True
break
cv2.destroyAllWindows()
main()
This code is running fine. If I have to close the application by pressing q, OpenCV window closes but application keeps running for 60sec because of the thread and only after 60sec whole application terminates safely.
I wanted to know if this is a good way to close the threads. Is there any better way available which can immediately terminate threads in Python?
There's no native way of stopping a thread in Python. Instead of using a stop flag, you can also use ctypes that calls the Python API to raise an exception in the thread.
import ctypes
# Other imports...
class ThreadWithException(threading.Thread):
def __init__(self, name):
threading.Thread.__init__(self)
self.name = name
def run(self):
# code here...
def get_id(self):
# returns id of the respective thread
if hasattr(self, '_thread_id'):
return self._thread_id
for id, thread in threading._active.items():
if thread is self:
return id
def raise_exception(self):
thread_id = self.get_id()
res = ctypes.pythonapi.PyThreadState_SetAsyncExc(thread_id,
ctypes.py_object(SystemExit))
if res > 1:
ctypes.pythonapi.PyThreadState_SetAsyncExc(thread_id, 0)
print('Exception raise failure')
main method takes no arguments. main method should read all files and folders on the hard disk. However, it seems that there is no way to execute the main method in a thread. In the beginning, it reads well, then slows down from the middle, and the QListWidget, ScanInfo, stops and prints out at once, which is strange. I want it to print smoothly. What should I do?
InfoAV.py
def AScanButtonClicked(self):
set1 = Setting.fileSet
set2 = Setting.disinfectSet
self.iv = InfonetV('C:/', set1.checkState(), set2.checkState())
self.window = ScanForm()
self.iv.started.connect(self.window.show)
self.iv.logSignal1.connect(self.window.ScanInfo.addItem)
self.iv.logSignal1.connect(self.window.ScanInfo.scrollToBottom)
self.iv.logSignal2.connect(self.window.ResultInfo.addItem)
self.iv.scan()
IV.py
class InfonetV(QObject):
started = pyqtSignal()
finished = pyqtSignal()
logSignal1 = pyqtSignal(str)
logSignal2 = pyqtSignal(str)
def __init__(self, path, set1, set2, parent = None):
...
...
def scan(self):
threading.Thread(target=self.main).start()
...
def main(self):
self.started.emit()
logSinal1 is a signal to add the Item of ScanList which is QListWidget. Write time.sleep(0.05) before logSignal1 is executed. Then it runs fine.
time.sleep(0.05)
self.logSignal1.emit(str(fname+ " " +message))
I am trying to use Flask-Socketio to connect to 3 separate RabbitMQ queues:
- 1 that listens for a config update message
- and 2 that are defined in a database
On the server start, I am connecting to the database, getting the configs there, and starting the consumers. Then, in a web frontend, if one of the configuration settings are changed, these changes are being written to the database, and a config update message is being added to the first RabbitMQ queue. Ideally what this would trigger a shutdown of the Pika consumer that I currently have running, a joining of that thread, and a relaunch of another thread with the new configuration information.
Everything that I just laid out is working, but on the first attempt to shut down the consumer, I'm always getting the error:
There was an error stopping the consumer: Second simultaneous read on fileno x detected.
Unless you really know what you're doing, make sure that only one greenthread can read any particular socket.
Consider using a pools.Pool. If you do know what you're doing and want to disable this error, call eventlet.debug.hub_prevent_multiple_readers(False)...
The consumers are eventually closed, and restarted, however, I would like to understand why this is occurring, and how I could change my code to stop it. The place where the error always happens is in the hand off between these 2 function, the first is in my Consumer class and the second is in my Queue class:
def run(self):
while True:
self.go = True
self.message_queue = Queue(self.configs, self.go, self.mongo_config)
self.message_queue.start()
# here I wait for an event which I set when the config is updated
self.event.wait()
self.go = False
setattr(self.message_queue, 'go', False)
new_config = self.refresh_configs()
setattr(self, 'configs', new_config)
# when this is called, it should close the existing connection and join the thread
self.message_queue.refresh_connection()
self.message_queue.join()
def refresh_connection(self):
while True:
if not self.go:
break
self.rmq_connection = rabbitmq_consumer(...)
self.rmq_connection.start_consuming()
self._lock.acquire()
try:
# right here is where the second read error occurs
self.rmq_connection.stop_consuming()
self.rmq_connection.close()
except Exception as e:
print('There was an error stopping the consumer: {0}'.format(e))
self._lock.release()
Below is a much more complete example of the code, in case it helps shed some light on the issue.
thread = None
thread_lock = Lock()
event = Event()
class Queue(Thread):
def __init__(self, configs, go, outbound):
Thread.__init__(self)
self._lock = eventlet.semaphore.Semaphore(1)
self.go = go
self.configs = configs
self.outbound = outbound
...
self.rmq_connection = None
def on_rmq_message(self, ...):
...
self._lock.acquire()
socketio.emit('eventEmit', {'data': results}, namespace='/')
result = rabbitmq_producer(...)
self._lock.release()
def refresh_connection(self):
while True:
if not self.go:
break
self.rmq_connection = rabbitmq_consumer(...)
self.rmq_connection.start_consuming()
self._lock.acquire()
try:
self.rmq_connection.stop_consuming()
self.rmq_connection.close()
except Exception as e:
print('There was an error stopping the consumer: {0}'.format(e))
self._lock.release()
def run(self):
self.refresh_connection()
class Consumer(Thread):
def __init__(self, configs, event, channel, mongo_config):
Thread.__init__(self)
self.configs = configs
self.mongo_config = mongo_config
self.event = event
self.message_queue = None
self.go = None
...
def refresh_configs(self):
r = None
mconnection = connect_mongodb(...)
results = retrieve(...)
for result in results.data:
if result.get('channel') == self.channel:
r = result
return r
def run(self):
while True:
self.go = True
self.message_queue = Queue(self.configs, self.go, self.mongo_config)
self.message_queue.start()
self.event.wait()
self.go = False
setattr(self.message_queue, 'go', False)
new_config = self.refresh_configs()
setattr(self, 'configs', new_config)
self.message_queue.refresh_connection()
self.message_queue.join()
class Producer(Thread):
def __init__(self, configs, event):
Thread.__init__(self)
self._lock = eventlet.semaphore.Semaphore(1)
self.configs = configs
self.event = event
...
self.channel = self.configs.get('channel', None)
self.config_consumer = None
def on_config_message(self, ...):
...
self._lock.acquire()
socketio.emit('configEmit', {'data': results}, namespace='/')
self._lock.release()
self.event.set()
self.event.clear()
def run(self):
self.config_consumer = rabbitmq_consumer(...)
self.config_consumer.start_consuming()
def init_connections():
with app.test_request_context('/'):
mconnection = connect_mongodb(...)
results = retrieve(mconnection.engine, mongo_collection, mongo_db, cursor=False)
...
t1 = Producer(configs, event)
for result in results.data:
config = {
...
}
t2 = Consumer(result, event, result['channel'], config)
t2.start()
t1.start()
t1.join()
t2.join()
#socketio.on('connect')
def handle_connection():
global thread
socketio.emit('connectEmit', {'data': 'Connected!'}, namespace='/')
with thread_lock:
if thread is None:
thread = socketio.start_background_task(target=init_connections)
Thank you for any help you can offer.
I'm learning about sockets in python and as practice I made a python gui app with a text box, two entry boxes and a button. It works like a chat app, I can run many modules at once, and in each, when a user enters text and click send, it shows the message entered, on the text box of the user's module and all other modules open. I've gotten most of it working but the issue is that it only updates the text box when the send button is pressed but I want to update constantly so as soon as a new message is sent, it shows it on the screen.
Here's my code so far:
#!/usr/bin/python
class Application(tk.Frame):
def __init__(self, master=None):
super().__init__(master)
self.pack()
self.root = tk.Tk()
self.create_widgets()
def create_widgets(self):
... #code that creates widgets shortended for readability
#text box widget named self.txt
self.send_btn["command"] = self.send_msg //this handles sending messages to the server when the button is pressed
... #code that creates widgets shortended for readability
def send_msg(self):
s = socket.socket()
host = socket.gethostname()
port = 9999
address = (host, port)
msg=self.name_bar.get() + ": " +self.input_bar.get()
#name bar is another widget so I can enter an identity for each module
#input bar is the entry box where text is entered in
self.input_bar.delete(0, len(msg))
s.connect(address)
s.send(msg.encode())
#Wrote this code that updates textbox when button pushed
#self.txt.insert(tk.END, s.recv(1024))
#self.txt.insert(tk.END, "\n")
#Method I created to call constantly to update textbox
def rcv_msg(self):
s = socket.socket()
host = socket.gethostname()
port = 9999
address = (host, port)
s.connect(address)
if s.recv(1024).decode() != "":
#self.txt.insert(tk.END, s.recv(1024))
#self.txt.insert(tk.END, "\n")
Also I been doing Java lately so I apologise if my terminology is mixed up.
I've already made the method to update the text box, I just don't know how to call it, I tried a while loop but it just stops the app from running. Also, the server is really simple and just sends the message from the client above to all the all modules of the client open. I din't know if the code was necessary, I was told previously to try to keep questions short but if it's needed please tell me. Thank you!
You could have something listening for new information. A separate thread that doesn't interfere with the GUI. (Pseudo code ahead!)
import socket
from threading import Thread, Lock
import time
class ListenThread(Thread):
def __init__(self, app, lock):
Thread.__init__(self)
self._app = app
self._lock = lock
self._terminating = False
def destroy(self):
self._terminating = True
def run(self):
s = socket.socket()
host = socket.gethostname()
port = 9999
address = (host, port)
while True:
if self._terminating:
break;
if s.recv(1024).decode() != "":
self._lock.acquire()
self._app.post_new_data(s.recv(1024))
self._lock.release()
time.sleep(0.1)
class App(object):
# ...
def __init__(self):
self._lock = Lock()
self._listener = ListenThread(self, self._lock)
self._listener.start()
def post_new_data(self, data):
# Post information to the GUI
def all_data(self):
self._lock.acquire()
# ...
self._lock.release()
def destroy(self):
self._listener.destroy()
# ... Tk mainloop ...
Breakdown
class ListenThread(Thread):
This is the item that will listen for new data coming from other connections and post them to the GUI via self._app.post_new_data(...) call in the run(self) operation.
def run(self):
s = socket.socket()
When we first start the execution on a thread (via start()) we create our socket and get a connection going. This way, all incoming transmissions will route through this in order to keep our GUI free for things it likes to do (paint the interface, take on user input, etc.)
The while loop on the ListenThread.run will keep looking for new data until we've killed the thread. As soon as it receives data, it will push that information to our App. In the post_new_data function, you can add our new data onto the GUI, store it, whatever you would like.
def post_new_data(self, data):
self.txt.insert(tk.END, data)
self.txt.insert(tk.END, "\n")
I have created a program to generate data points of functions that I later plot. The program takes a class which defines the function, creates a data outputting object which when called generates the data to a text file. To make the whole process faster I put the jobs in threads, however when I do, the data generated is not always correct. I have attached a picture to show what I mean:
Here are some of the relevant bits of code:
from queue import Queue
import threading
import time
queueLock = threading.Lock()
workQueue = Queue(10)
def process_data(threadName, q, queue_window, done):
while not done.get():
queueLock.acquire() # check whether or not the queue is locked
if not workQueue.empty():
data = q.get()
# data is the Plot object to be run
queueLock.release()
data.parent_window = queue_window
data.process()
else:
queueLock.release()
time.sleep(1)
class WorkThread(threading.Thread):
def __init__(self, threadID, q, done):
threading.Thread.__init__(self)
self.ID = threadID
self.q = q
self.done = done
def get_qw(self, queue_window):
# gets the queue_window object
self.queue_window = queue_window
def run(self):
# this is called when thread.start() is called
print("Thread {0} started.".format(self.ID))
process_data(self.ID, self.q, self.queue_window, self.done)
print("Thread {0} finished.".format(self.ID))
class Application(Frame):
def __init__(self, etc):
self.threads = []
# does some things
def makeThreads(self):
for i in range(1, int(self.threadNum.get()) +1):
thread = WorkThread(i, workQueue, self.calcsDone)
self.threads.append(thread)
# more code which just processes the function etc, sorts out the gui stuff.
And in a separate class (as I'm using tkinter, so the actual code to get the threads to run is called in a different window) (self.parent is the Application class):
def run_jobs(self):
if self.running == False:
# threads are only initiated when jobs are to be run
self.running = True
self.parent.calcsDone.set(False)
self.parent.threads = [] # just to make sure that it is initially empty, we want new threads each time
self.parent.makeThreads()
self.threads = self.parent.threads
for thread in self.threads:
thread.get_qw(self)
thread.start()
# put the jobs in the workQueue
queueLock.acquire()
for job in self.job_queue:
workQueue.put(job)
queueLock.release()
else:
messagebox.showerror("Error", "Jobs already running")
This is all the code which relates to the threads.
I don't know why when I run the program with multiple threads some data points are incorrect, whilst running it with just 1 single thread the data is all perfect. I tried looking up "threadsafe" processes, but couldn't find anything.
Thanks in advance!