I'm creating 5 threads for handling various tasks such as reading from sensors (Raspberry Pi), TCP connections and recently recording audio (pyAudio).
I am instantiating all threads in main() identically e.g.:
if __name__ == '__main__':
main()
def main():
global network_thread
network_thread = threading.Thread(name="NET-CONN", target=network_thread_run, args=())
network_thread.start()
I keep a global reference so I can kill the threads at shutdown with join().
Now, I have added thread #5:
global audio_thread
audio_thread = threading.Thread(name="AUDIO", target=audio_thread_run(), args=())
audio_thread.start()
...but my logging indicates it's running on the MainThread. I also double-checked inside the audio_thread_run() function and it is indeed running on MainThread:
if threading.current_thread() is threading.main_thread():
logger.warning("Audio thread is the same as MainThread!")
Why is this thread running on the MainThread? Have I hit a limit on the Pi?
Let's have a look at the two places where you create threads, modified slightly so they'll fit on one line, and with white-space inserted so they line up:
net_thread = threading.Thread(name="NET", target=net_run , args=())
aud_thread = threading.Thread(name="AUD", target=aud_run(), args=())
# Hmmm, what's this suspicious-looking thing here? ---->^^
Enough fun :-) The problem is that you are actually calling the audio_thread_run() function directly from your main thread and presumably, if it ever returned, you would then try to use the result as a callable to start a thread.
If you actually got rid of the thread start stuff altogether, it would boil down to the much simpler:
audio_thread_run()
which will very much run that function from the context of the main thread.
What you need to do is to remove the parentheses so it matches what you've down with the network threads:
audio_thread = threading.Thread(name="AUDIO", target=audio_thread_run, args=())
Related
i have multiple threads running an infinite while true without them knowing of each other's existence.
Inside their respective loops i need them to check the time and do something based on it before the next iteration, something like this:
Thread:
while True:
now = timedate.now()
# do something
time.sleep(0.2)
these threads are started in my main program in such a way:
Main:
t1.start()
t2.start()
t3.start()
...
...
while True:
#main program does something
Onto the problem, i need all the threads running to receive the same time when they check for it.
I was thinking maybe about creating a class with a lock on it and a variable to store the time, the first thread that acquires the lock saves the time in it so that the following threads can read it but to me this seems quinda a hacky way of doing things (plus i wouldn't know how to check when all the threads have read the time to update it).
What would be the best way, if possible, to implement this?
I have a simple watchdog in python 3 that reboots my server if something goes wrong:
import time, os
from multiprocessing import Pool
def watchdog(x):
time.sleep(x)
os.system('reboot')
return
def main():
while True:
p = Pool(processes=1)
p.apply_async(watchdog, (60, )) # start watchdog with 60s interval
# here some code thas has a little chance to block permanently...
# reboot is ok because of many other files running independently
# that will get problems too if this one blocks too long and
# this will reset all together and autostart everything back
# block is happening 1-2 time a month, mostly within a http-request
p.terminate()
p.join()
return
if __name__ == '__main__':
main()
p = Pool(processes=1) is declared every time the while loop starts.
Now here the question: Is there any smarter way?
If I p.terminate() to prevent the process from reboot, the Pool becomes closed for any other work. Or is there even nothing wrong with declaring a new Pool every time because of garbage collection.
Use a process. Processes support all of the features you are using, so you don't need to make a pool with size one. While processes do have a warning about using the terminate() method (since it can corrupt pipes, sockets, and locking primitives), you are not using any of those items and don't need to care. (In any event, Pool.terminate() probably has the same issues with pipes etc. even though it lacks a similar warning.)
I'm trying to utilize threading and queueing (based on a recommendation) to pause the main process.
My program basically iterates through images, opening and closing them utilizing a 3-second time-loop for each iteration.
I'm trying to use threading to interject a time.sleep(20) if a certain condition is met (x == True). The condition is being met (evident by the output of the print statement), but time.sleep(20) is not affecting the main process.
I plan to subsitute time.sleep(20) with a more complex process but for simpliclity I've used it here.
import time
import subprocess
import pickle
import keyboard
import threading
from threading import Thread
import multiprocessing
import queue
import time
with open('C:\\Users\Moondra\\Bioteck.pickle', 'rb') as file:
bio = pickle.load(file)
q = queue.LifoQueue(0)
def keyboard_press(): # This is just receiving boolean values based on key presses
while True:
q.put(keyboard.is_pressed('down'))
x = q.get()
print(x)
if x == True:
time.sleep(20)
t = Thread(target = keyboard_press, args= ())
t.start()
if __name__ == "__main__":
for i in bio[:5]:
p = subprocess.Popen(["C:\Program Files\IrfanView\i_view64.exe",'C:\\Users\Moondra\\Bioteck_charts\{}.png'.format(i)])
time.sleep(3)
p.kill()
So why isn't my thread affecting my main process?
Thank you.
Update:
So It seems I have to use flags and use flag as a global variable within my function. I would like to avoid using global but it's not working without globalizing flag within my function.
Second, I don't know how to restart the thread.
Once the thread returns the flag as false, the thread sort of just stalls.
I tried starting the thread again, with t.start, but I received the error:
RuntimeError: threads can only be started once
Here is updated code:
def keyboard_press():
while True:
global flag
q.put(keyboard.is_pressed('down'))
x = q.get()
print(x)
if x == True:
flag = False
#print('keyboard_flag is',flag)
return flag
if __name__ == "__main__":
flag = True
q = queue.LifoQueue(0)
t = Thread(target = keyboard_press, args= ())
t.start()
for i in bio[:5]:
p = subprocess.Popen(["C:\Program Files\IrfanView\i_view64.exe",'C:\\Users\Moondra\\Bioteck_charts\{}.png'.format(i)])
time.sleep(3)
print ('flag is',flag)
if flag == True:
p.kill()
else:
time.sleep(20)
p.kill()
flag = True
#t.start() #doesn't seem to work.
why isn't my thread affecting my main process?
Because you have not written any code to be executed by the keyboard_press() thread that would affect the main process.
It looks like you're trying to create a slide show that shows one image every three seconds, and you want it to pause for an extra twenty seconds when somebody presses a key. Is that right?
So, you've got one thread (the main thread) that runs the slide show, and you've got another that polls the keyboard, but your two threads don't communicate with one another.
You put a time.sleep(20) call in your keyboard thread. But that only pauses the keyboard thread. It doesn't do anything at all to the main thread.
What you need, is for the keyboard thread to set a variable that the main thread looks at after it wakes up from its three second sleep. The main thread can look at the variable, and see if a longer sleep has been requested, and if so, sleep for twenty more seconds.
Of course, after the longer sleep, you will want the main thread to re-set the variable so that it won't always sleep for twenty seconds after the first time the keyboard is touched.
P.S.: I am not a Python expert. I know that in other programming environments (e.g., Java), you also have to worry about "memory visibility." That is, when a variable is changed by one thread, there is no guarantee of when (if ever) some other thread will see the change...
...Unless, the threads use some kind of synchronization when they access the variable.
Based on what I have read (It's on the Internet! It must be true!), Python either does not have that problem now, or it did not have that problem in the recent past. I'm not sure which.
If memory consistency actually is an issue, then you will either have to use a mutex when you access the shared variable, or else you will have to make the threads communicate through some kind of a synchronized object such as a queue.
I used raspberry pi3, python 2.7 and OpenCV3
I have two functions:
Func1() to run Picamera
Func2() to move motor forward
My problem is : when I run Func2() the Func1() stop temporary until Func2() finishes, although I used separate threads for each function.
My code is:
def picam():
t1=threading.thread(target=Func1())
t1.start()
t1.join()
def move():
t2=threading.thread(target=Func2())
t2.start()
t2.join()
in my GUI I have 2 button . one to call picam() and other to call move()
What is the problem? I don't need Func1() stop temporarily because the camera will stop temporarily also.
I am providing this answer because Threading is genuinely a difficult topic. Part of the problem, especially for beginners, is the Catch 22 of opaque terminology, which makes it very difficult to understand what the different components do if you don't already understand.
Your problem is with how you use join. join is a way to wait for a given thread to finish. So calling t1.start() followed immediately by t1.join() makes the main thread wait for t1 to finish.
Aside from the fact that you started a couple of separate thread, your code is pretty much exactly equivalent to doing
Func1()
Func2()
in the main thread.
You should put the joins after both threads are started if you want them to run simultaneously:
t1=threading.thread(target=Func1())
t2=threading.thread(target=Func2())
t1.start()
t2.start()
t1.join()
t2.join()
This will create two threads, start them, and then wait until both are done.
I have a code that calls to threads over a loop, something like this:
def SubmitData(data):
# creating the relevant command to execute
command = CreateCommand(data)
subprocess.call(command)
def Main():
while(True):
# generating some data
data = GetData()
MyThread = threading.Thread(target=SubmitData,args=(data,))
MyThread.start()
obviously, I don't use join() on the threads.
My question is how to join() those threads without making the main thread wait for them?
Do I even need to join() them? what will happend if I won't join() them?
some important points:
the while loop is suppose to for a very long time (couple of days)
the command itself is not very long (few seconds)
I'm using threading for Performance so if someone have a better idea instead, I would like to try it out.
Popen() doesn't block. Unless CreateCommand() blocks, you could call SubmitData() in the main thread:
from subprocess import Popen
processes = []
while True:
processes = [p for p in processes if p.poll() is None] # leave only running
processes.append(Popen(CreateCommand(GetData()))) # start a new one
Do I even need to join() them? what will happend if I won't join() them?
No. You don't need to join them. All non-daemonic threads are joined automatically when the main thread exits.