nvidia-smi screenshot
The process with PID 14420 is a zombie process and its parent id is 1(init). I want to clear 4436MiB memory occupied by this zombie process without rebooting.
How should I proceed?
Use this command to list all the processes on the GPU (on Linux).
sudo fuser -v /dev/nvidia*
and find PID from the listed processes and simply use
kill <PID> .
This will kill process with specified ID.
Related
I wanna stop Node server on linux but when I try to stop it, It automatically starts with new PID how can I stop this completely. Here you can see I tried to stop Nginx and Node.
How can I check if it is a linux zombie process? If yes then How can I kill it? otherwise
I have tried tried these commands.
kill pid
Kill -9 pid
killall node <<command not working
This was spawn by it's parent process, I search out parent process after killing that this stop as well.
I'm running my execuatable through mpirun command. I want to get the pid of processes being created so that I could kill them later if required. I'm using MPICH. In openmpi there is an option -report-pid which gives pids. Is there anything similar in MPICH?
Tried with some examples like ps and ps -ef,after killing a process by using kill-9 PID in Linux,how to verify weather the process is killed or not?
Just run ps aux stat,pid again, and you will see the process with this pid is either a zombie ('Z' in the first column) or dead.
Edit:
Thanks, Mark B, for pointing me about zombies.
Ater the kill, check the PID of the process:
$ pidof PROCESS
You should see no output if the process is gone.
Another similar way:
$ ps aux | grep PROCESS
Notes:
You can kill your own process, but only root user can kill system process or another user process.
After you KILL the process you can get a zombie, you can still see it in the process list but with process STATE Z (wich means Zombie). Zombies cant be killed, they are already dead, so to "kill" zombies you need to kill the zombie's parents. That said, in general you don’t need to get rid of zombie processes unless you have a large amount of them.
I execute the following strace command with the intention of getting data about PID 13221
strace -fF -tT -all -o abc.txt -p 13221
However when the command executes and finishes I get output like below :
Process 13221 attached with 12 threads - interrupt to quit
Process 13252 attached
Process 13253 attached (waiting for parent)
Process 13253 resumed (parent 13252 ready)
Process 13252 suspended
Process 13252 resumed
Process 13253 detached
Process 13252 detached
Process 13232 detached
Process 13228 detached
Process 13225 detached
Process 13222 detached
Process 13221 detached
What are these extra PID's ? Are these the children of 13221 ? Who is creating them ?
Thanks.
What are these extra PID's ? Are these the children of 13221 ?
It must have been threads of your program. You have used "-f" in strace and this is why threads are also monitored.
How to know which of these are threads
If you run ls /proc/<PID>/task for your process you will get PIDs of all threads in your process (including a PID of the main thread).
It is simpler to do when you need to get thread PIDs comparing with running pstack for the same process. pstack is actually a gdb script, it stops a process when attaches. So it is simpler just to run ls /proc/<PID>/task
If I want to strace a multi-threaded process (of all of its threads), how should I do it?
I know that one can do strace -f to follow forked process? But how about attaching to a process which is already multi-threaded when I start stracing? Is a way to tell strace to trace all of system calls of all the threads which belong to this process?
2021 update
strace -fp PID just does the right thing on my system (Ubuntu 20.04.1 LTS). The strace manual page points this out:
-f Trace child processes as they are created by currently traced processes as a result of the fork(2), vfork(2) and clone(2) system
calls. Note that -p PID -f will attach all threads of process PID if it is multi-threaded, not only thread with thread_id = PID.
Looks like this text was added back in 2013. If -f had this behavior on my system at the time, I didn't realize it. It does now, though!
Original 2013 answer
I just did this in a kludgy way, by listing each tid to be traced.
You can find them through ps:
$ ps auxw -T | fgrep program_to_trace
me pid tid1 ...
me pid tid2 ...
me pid tid3 ...
me pid tid4 ...
and then, according to man strace, you can attach to multiple pids at once:
-p pid Attach to the process with the process ID pid and begin tracing. The trace may be terminated at any time by a keyboard interrupt
signal (CTRL-C). strace will respond by detaching itself from the traced process(es) leaving it (them) to continue running. Mul‐
tiple -p options can be used to attach to up to 32 processes in addition to command (which is optional if at least one -p option is
given).
It says pid, but iirc on Linux the pid and tid share the same namespace, and this appeared to work:
$ strace -f -p tid1 -p tid2 -p tid3 -p tid4
I think that might be the best you can do for now. But I suppose someone could extend strace with a flag for expanding tids. There would probably still be a race between finding the processes and attaching to them in which a freshly started one would be missed. It'd fit in with the existing caveat about strace -f:
-f Trace child processes as they are created by currently traced processes as a result of the fork(2) system call.
On non-Linux platforms the new process is attached to as soon as its pid is known (through the return value of fork(2) in the par‐
ent process). This means that such children may run uncontrolled for a while (especially in the case of a vfork(2)), until the par‐
ent is scheduled again to complete its (v)fork(2) call. On Linux the child is traced from its first instruction with no delay. If
the parent process decides to wait(2) for a child that is currently being traced, it is suspended until an appropriate child
process either terminates or incurs a signal that would cause it to terminate (as determined from the child's current signal dispo‐
sition).
On SunOS 4.x the tracing of vforks is accomplished with some dynamic linking trickery.
As answered in multiple comments, strace -fp <pid> will show the trace of all threads owned by that process - even ones that process already has spawned before strace begins.