Hi all~ I have a problem boring me so much.
Sometimes when I exit my program, there are some thread still running, in Linux system, it will cause crash after I quit the main loop. Is there any method that can kill all threads when I quit main loop?
It would help a lot if you specified your programming language and threading library of choice.
The usual way to control this type of situation (that is for a parent thread to wait until children complete before terminating) is to call a function supplied by the library, usually named join or wait.
pthread supplies you with pthread_join, for example.
If you're spawning processes via fork, you should use wait or waitpid in the parent to halt until the child completes - try man waitpid or take a look at this.
This way you can inform your children that you are about to exit via the usual means, wait until they wrap up and terminate, then cleanly exit the main loop.
Does this help? This is the least brutal way of synchronizing termination, if you want to actively kill the children threads there are alternatives, of course (like pthread_kill for pthreads, for example).
If you are using java try using the jconsole (Java Monitoring & Management Console) shipped with jdk6u23 in my case. You can get the thread name that is not killed. You can use join for that thread to complete.
But there can be program issue like, in my case i had a timer thread hanging [Timer-0] java.util.Timer to make an a timer.cancel() which closed that timer.
Related
I am taking an OS class and trying to wrap my head around this question, any help would be appreciated:
Consider a multi-threaded process with 10 threads. Thread 3 invokes an execlp() system call. Describe what will happen to each of the threads.
My understanding of exec() is that is replaces the current process with a new one, and it's main difference from fork() is that fork() creates a clone and you end up with duplicates.
So if exec() replaces the current process, would it kill the threads of the old process and replace them with the new one? Any help will be appreciated.
exec()...replaces the current process with a new one.
Actually, it's still the same process after calling exec, and that's important because the parent process may still need to communicate with it, signal it, etc. What exec does is, it guts the process—wipes out all of it's virtual memory, resets all of its signal handlers, unlocks locks, closes some open files, etc. (See here for more)—and then it loads a new program into the existing process and starts executing it.
would it kill the threads of the old process...?
man 2 execve says, "All threads other than the calling thread are destroyed during
an execve(). Mutexes, condition variables, and other pthreads
objects are not preserved."
In Linux & C, will not waiting (waitpid) for a fork-execve launched process create zombies?
What is the correct way to launch a new program (many times) without waiting and without resource leaks?
It would also be launched from a 2nd worker thread.
Can the first program terminate first cleanly if launched programs have not completed?
Additional: In my case I have several threads that can fork-execve processes at ANY TIME and THE SAME TIME -
1) Some I need to wait for completion and want to report any errors codes with waitpid
2) Some I do not want to block the thread and but would like to report errors
3) Some I don't want to wait and don't care about the outcome and could run after the program terminates
For #2, should I have to create an additional thread to do waitpid ?
For #3, should I do a fork-fork-execve and would ending the 1st fork cause the 2nd process to get cleaned up (no zombie) separately via init ?
Additional: I've read briefly (not sure I understand all) about using nohup, double fork, setgpid(0,0), signal(SIGCHLD, SIG_IGN).
Doesn't global signal(SIGCHLD, SIG_IGN) have too many side effects like getting inherited (or maybe not) and preventing monitoring other processes you do want to wait for ?
Wouldn't relying on init to cleanup resources leak while the program continues to run (weeks in my case)?
In Linux & C, will not waiting (waitpid) for a fork-execve launched process create zombies?
Yes, they become zombies after death.
What is the correct way to launch a new program (many times) without waiting and without resource leaks? It would also be launched from a 2nd worker thread.
Set SIGCHLd to SIG_IGN.
Can the first program terminate first cleanly if launched programs have not completed?
Yes, orphaned processes will be adopted by init.
I ended up keeping an array of just the fork-exec'd pids I did not wait for (other fork-exec'd pids do get waited on) and periodically scanned the list using
waitpid( pids[xx], &status, WNOHANG ) != 0
which gives me a chance report outcome and avoid zombies.
I avoided using global things like signal handlers that might affect other code elsewhere.
It seemed a bit messy.
I suppose that fork-fork-exec would be an alternative to asynchronously monitor the other program's completion by the first fork, but then the first fork needs cleanup.
In Windows, you just keep a handle to the process open if you want to check status without worry of pid reuse, or close the handle if you don't care what the other process does.
(In Linux, there seems no way for multiple threads or processes to monitor the status of the same process safely, only the parent process-thread can, but not my issue here.)
Given the following code:
from threading import Thread
def dosomething():
for _ in range(100):
print("background thread running")
t = Thread(target=dosomething)
t.start()
The program runs as long as the background thread needs to completely go through the loop. My question is, why does the main thread wait for the background thread to finish and doesn't exit immediately after starting the background thread.
Because that's the way it is. From the threading docs:
A thread can be flagged as a “daemon thread”. The significance of this flag is that the entire Python program exits when only daemon threads are left. The initial value is inherited from the creating thread. The flag can be set through the daemon property or the daemon constructor argument.
I assume that you ran something like python my_script.py on the command line and wondered why it doesn't return you to a prompt until the worker thread is done?
Well if you change your code to:
from threading import Thread
def dosomething():
for _ in range(100):
print("background thread running")
t = Thread(target=dosomething, daemon=True)
t.start()
You will find that you are returned to the terminal. However your daemon thread will also die (again from the docs):
Note Daemon threads are abruptly stopped at shutdown. Their resources (such as open files, database transactions, etc.) may not be released properly. If you want your threads to stop gracefully, make them non-daemonic and use a suitable signalling mechanism such as an Event.
This is because killing the program means killing the process and since that worker thread is handled by that process it dies - try this article for a fuller explanation on that.
Because that's the way it ought to be!
Back in the days before multi-threading, a program would end when its main routine ended which also, coincidentally, was when its last (and only) thread ended. When multi-threading became a thing, some languages (e.g., Python, Java) generalized the classic behavior to "program ends when last thread ends," while others (e.g., C/C++) generaized it to "program ends when its main thread ends."
Having worked in both worlds, It's my strong opinion that Python and Java got it right. The main thread is special to the extent that it's the first: It's the one that calls the program's designated entry point. But after things get going, there is no good reason why the main thread should be treated any differently from any other thread. That only complicates things---makes the behavior that much more difficult to explain.
Processes should only terminate themselves, when all their threads are
terminated!
It's a question in our mock exam, and we aren't sure whether the statement is true or false.
Thanks a lot
First, I need to point out that this exam question contains an incorrect presumption. A running process always has at least one thread. The initial thread, the thread that first calls main or equivalent, isn't special; it's just like every other thread created by pthread_create or equivalent. Once all of the threads within a process have exited, the process can't do anything anymore — there's no way for it to execute even a single additional CPU instruction. In practice, the operating system will terminate the process at that point.
Second, as was pointed out in the comments on the question, the use of "should" makes your exam question ambiguous. It could be read as either "Processes only terminate when all of their threads are terminated" — as a description of how the system works. Or it could be read as "You, the programmer, should write code that ensures that your processes only terminate when all of their threads are terminated" — as a prescription for writing correct code.
If you are specifically talking about POSIX threads ("pthreads"), the answer to the descriptive question is that it depends on how each thread terminates. If all threads terminate by calling pthread_exit or by being cancelled, the process will survive until the last thread terminates, no matter which order they exit in. On the other hand, if any thread calls exit or _exit, or receives a fatal signal, that will immediately terminate the entire process, no matter how many threads are still active. (I am not 100% sure about this, but I think it doesn't matter whether any threads have been detached.)
There's an additional complication, which is that returning from a function passed to pthread_create is equivalent to calling pthread_exit for that thread, but returning from main is equivalent to calling exit. That makes the initial thread a little bit special: unless you specifically end main by calling pthread_exit, the entire process will be terminated when the initial thread exits. But technically this is not a property of the thread itself, but of the code running in that thread.
I do not know the answer to the descriptive question for threads libraries other than POSIX; in particular I don't know the answer for either Windows native threads, or for the threads library added to ISO C in its 2011 revision.
The answer to the prescriptive question is yes with exceptions. You, a programmer, should write programs that, under normal conditions, take care to end their process only when all of their threads have finished their work. (With POSIX threads, this translates to making sure that main does not return until all the other threads have been joined.) However, sometimes you have a few threads that run an infinite loop, without holding any locks or anything, and there's no good way to tell them to exit when everything else is done; as long as exiting the process out from under them won't damage any persistent state, go ahead and exit the process out from under them. (This is the intended use case for detached threads.) Also, it's OK, and often the best choice, to terminate the entire process abruptly if you encounter some kind of unrecoverable error. Those are the only exceptions I can think of off the top of my head.
I've read that a lot can happen when mixing threads and forking and it should better be avoided. I'm finding myself in a situation where I don't have a choice and I receive a kernel-crash of my kernel-module.
My reduced test-case has 2 threads. One of it is doing ioctls to an open device-node in a loop. The other one is doing one fork, waits for the child to exit, which it does immediately. If I use pthread_atfork to synchronized my thread with the fork-call it is working.
Where can I look at to find out more on what happens during a fork on open file-descriptors which are currently executing an ioctl? What kind of corruption can happen?
EDIT: Andreas made me change my test case. Instead of having the child exiting immediatly I'm not waiting 10 seconds before exiting. I'm collecting all PID in the parent-process to later do a waitpid. I'm forking 100 times. If makes it crash after 2 or 3 forks.
There should be no problem caused by threading in that regard. There should especially be no kernel crashes.
From your description it sounds like you are writing your own kernel module that handles the file descriptor in question. Note that a forked process gets copies of all open file descriptors. When it exits it closes those.
It doesn't matter that it apparently does nothing but exit immediately, the close (and flush in file_operations) happens always. That is where you should be looking in your kernel code.