What is proper practice for stopping a continuously running Python process? Consider the following code:
from multiprocessing import Process
run_process = Process(target=self.run)
def start_running():
run_process.start()
def run():
while True:
# Do stuff
def stop_running():
# ???
run_process.join()
I would expect that the ideal situation would be to have the run() process end on its own when stop_running() is called. One idea is to signal a semaphore in stop_running(), which is checked in the run() loop, so it knows to break. But I would like to know what common practice is.
There is no "proper" way of doing much of anything in Python. As you are running a process instead of a thread, you have more options, which is good.
If your process does not have a risk of being stuck completely and it does not have the risk of being stuck on IO waiting for input potentially indefinitely (for example from a queue), I would use a semaphore or a variable to signal the process it should exit now.
If there is a risk of the process being stuck in wait, you can get rid of it by run_process.kill() or run_process.terminate(). Kill equals kill -9 in shell and is guaranteed to get the job done.
The drawback in killing/terminating a process is that if the process holds any shared objects (queues, shared variables etc), those become corrupted also in the other processes that share them. It is safe to discard them but if you keep reading from them, you may encounter occasionally obscure exceptions that are hard to debug.
So as always it depends. The variable/semaphore method has its strengths but if there is a risk of the subprocess being stuck in sleep or wait and not checking the condition, you do not achieve anything. If your subprocess does not share any resources with other processes, kill may be simpler and a guaranteed way of getting rid of your process.
Related
i have multiple threads running an infinite while true without them knowing of each other's existence.
Inside their respective loops i need them to check the time and do something based on it before the next iteration, something like this:
Thread:
while True:
now = timedate.now()
# do something
time.sleep(0.2)
these threads are started in my main program in such a way:
Main:
t1.start()
t2.start()
t3.start()
...
...
while True:
#main program does something
Onto the problem, i need all the threads running to receive the same time when they check for it.
I was thinking maybe about creating a class with a lock on it and a variable to store the time, the first thread that acquires the lock saves the time in it so that the following threads can read it but to me this seems quinda a hacky way of doing things (plus i wouldn't know how to check when all the threads have read the time to update it).
What would be the best way, if possible, to implement this?
I have read that a forever process like daemon should run with a sleep() in their while(1) or for(;;) loop. They say, it is required because otherwise this process will always be in a run queue and the kernel will always run it. This will block the other process. I don't agree that it will block the other process completely. If there is a time slicing, then it will execute other process. But, certainly it will steal a time from others. Making a delay for other process since this process is always in the run state. By default, the Linux runs as a round-robin. The first task is swapd, then other tasks . This is a circular link list with first task as swapd(process-id is 0) and then other tasks. I believe this is still based as time sliced. A particular time for each process. These tasks are nothing but the process-descriptor. I believe this link list is maintained by the init process. Please do correct me here If I am wrong. Other question is if we need to give a sleep() then what should be its value? How can we determine the sleep value to get the best results?
If your program has useful things to do, don't throttle it. A program can move out of the run queue by doing blocking stuff like IO and waiting.
If you are writing a polling loop that can spin an arbitrary number of times you probably want to throttle it a bit with sleep because spinning too often has little value.
That said, polling loops are a means of last resort. Normally, programs perform useful work with every instruction, so they don't sleep at all.
Sleep is almost certainly the wrong solution.
Usually what you do it call a blocking function which wakes you up when there's something for you to do.
For example, if you're a network service you'd want to remain inactive until a request arrives.
In other words, the core of your daemon should not look like this:
while(1)
{
if (checkIfSomethingToDo())
doSomething();
else
sleep(1);
}
but rather a little like this:
while(1)
{
int ret = poll(fds, nfds, -1);
if (ret > 0)
doSomething();
}
Have the kernel put you to sleep until there's actual work to do. It's not hard to implement, you'd be a lot more efficient (not stealing CPU time from others, only to waste it doing no actual work) and your response latency will go down too.
A sleep forces the os to pass execution to another thread and therefore is helpfull, or at least fair. Start with sleep one. Should be ok.
I use timer in my program:
timer = new Qtimer(); connect(timer, SIGNAL(timeout()), this, SLOT(readData())); timer.start(1000);
And there is also other slots which may be triggered by UI interation:
/*SLOT FUNCTION*/ on_pushbutton_triggered(){..../*write data*/...}.
(the code is written in qt, but I think it's a common question)
So I worry about the potential problem: may readData() reads wrong data while on_pushbutton_triggereed() is writting data?
I am not so familiar with how the timer really work behind the screen: is it in the same thread with my program?
Will readData() and on_pushbutton_triggereed() be called, executed, finished serially and have no mutex problem(that is: I have to use lock() and unlock())? Thank you for reading! I really hope for your hints!
Qt is using an event loop to implement concurrent activity in general and QTimer in particular within a single thread.
The event providers (QTimer in this case) are producing events and publish them to the event loop. Then they are processed according to their priority and order of publishing. This approach doesn't require any synchronization as there is only one section of code executed at the time, so it's safe to access data.
On Unix-like systems ps -eLf command will show information about all processes (PID column in the output) and their threads (LWP column). NLWP column shows how many threads particular process has.
I'm using Thread::Pool::Simple for multi-threading.
I have a couple of questions which are quite general to multi-threading, I guess:
Each of my threads might die if something unexpected hapens. This is totally accepted by me, since it means some of my assertion are wrong and I need to redesign the code. Currently, when any thread dies the main program (calling thread) also dies, yielding something like:
Perl exited with active threads:
0 running and unjoined
0 finished and unjoined
4 running and detached
Are these "running and detached"
zombies? Are they "dangerous" in any
way? Is there a way to kill all of
them if any of the threads dies?
What is the common solution for such
scenarios?
Generally, my jobs are independent.
However, I pass each of them as an
argument a unique hash which is
taken form one big hash oh hashes.
the thread might change this
personal hash (but it can't get to
the large hash - it doesn't even
know about it). Hence, I guess I
don't need any locks etc. Am I
missing anything?
When your main program exits, all threads are terminated.
Perl threads work in one of two ways.
1) You can use join:
my $thr = threads->create(...);
# do something else while thread works
my $return = $thr->join(); # wait for thread to terminate and fetch return value
2) You can use detach:
my $thr = threads->create(...);
$thr->detatch(); # thread will discard return value and auto-cleanup when done
That message lists the threads that hadn't been cleaned up before the program terminated.
"Running and unjoined" is case 1, still running. "Finished and unjoined" is case 1, finished but the return value hasn't been fetched yet. "Running and detached" is case 2, still running.
So it's saying you have 4 threads that had been detached but hadn't finished before the program died. You can't tell from that whether they would have finished if the program had run longer, or they were stuck in an infinite loop, or deadlocked, or what.
You shouldn't need any locks for the situation you describe.
I'm looking for some exit code that will be run from a thread but will be able to kill the main script. It's in Jython but I can't use java.lang.System.exit() because I still want the Java app I'm in to run, and sys.exit() isn't working. Ideally I would like to output a message then exit.
My code uses the threading.Timer function to run a function after a certain period of time. Here I'm using it to end a for loop that is executing for longer than 1 sec. Here is my code:
import threading
def exitFunct():
#exit code here
t = threading.Timer(1.0, exitFunct)
t.start()
for i in range(1, 2000):
print i
Well, if you had to, you could call mainThread.stop(). But you shouldn't.
This article explains why what you're trying to do is considered a bad idea.
If you want to kill the current process and you don't care about flushing IO buffers or reseting the terminal, you can use os._exit().
I don't know why they made this so hard.