Is it possible to find out all the programs getting executed in Linux. There will be many scripts and other executables getting launched and killed during the life time of a system and I would like to get a list of these (or a print while execuction start). I am looking for this to understand program flow in an embedded board.
Type ps aux in the terminal will give information about the processes(Started time, runtime), more information about keep track of processes
There's a kernel interface which will notify a client program of every fork, exec, and exit. See this answer: https://stackoverflow.com/a/8255487/5844347
Take a look at ps -e and eventually crontab if you want to take that information periodically.
Related
I'm looking for a way to record child processes and their arguments spawned by a script or other command. A good example would be recording the build commands spawned by "make" and any children they, in turn, spawn.
In my specific case working with a build script from an embedded systems manufacturer. It's a Python script that calls cmake, ninja, and heaven knows what else. I'd really like to know what's being called so I can maybe pare down the noise.
This seems like an obvious utility for someone to have written decades ago but I'll be darned if I can find it.
Much thanks for any advice.
I have multiple cron jobs written in Perl and they appear to be causing a high server load.
When I run top through SSH it shows that Perl is using the most CPU and memory. However, since there's multiple cron jobs, I need to know specifically which one is using the most resources.
Is there any way to check which of the Perl files is using the most resources?
Note the PID of the process top shows is using cpu. Then do a
ps -ef | grep perl
Match the PID to one listed and you'll see the full commandline of the perl process for the high cpu job.
Well, if you look at ps -ef you should see which script maps to that process id. You could also use strace -fTt -p <pid> to attach a debugger to a specific process id to see what it's doing.
Or you could modify the script to change $0 to something meaningful that tells you which script is which.
But it's hard to say without a bit more detail. Is there any chance the script is taking longer to run than you cron job takes to 'fire'? Because if you start backlogging a cron job you'll slowly get worse as more and more start piling up behind it.
I am developing a sandbox on linux. And now i am confused terminating all process in the sandbox.
My sandbox works as follows:
At first only one process run in the sandbox.
Then it can create several child process.
And child process will create their subprocess also.
And parent process may exit at some time before its children exited.
At last sandbox will terminate all the process.
I used to do this by using killall or pkill -u with a unique user attached to the sandbox.But it seems doesn't work on the program which uses fork() fastly.
Then I search for the source code of pkill and realized that pkill is lose of atomicity.
So how could i achieve my goal ?
You could use process groups setpgid(2) and sessions setsid(2), but I don't qualify what you do as a sandbox (in particular because if one of the processes is setuid or change its process group or session itself, you'll lose it; read execve(2) carefully and several times!). Notice that kill(2) with a negative pid kills an entire process group.
Read a good book like Advanced Linux Programming. Consider also using chroot(2).
And explain what and why you really want to do. sandboxing is harder that what you think. See also capabilities(7), credentials(7) and SElinux.
I have a simple Shell script in which multiple executables will run sequentially. Every time a new executable starts to run, a new process with a new PID will start. Is it possible to run them with the same PID? I know for a shell script, we can use "source". But I do not know how to handle executables.
In principle, I believe it should be possible, but in practice it would be very complicated and brittle.
The exec family of system calls in Linux allows a process to replace itself with an entirely new process, which holds on to the same PID. The tricky part would be to somehow "return" from the second process back to the first. When exec is called, the OS loads everything it needs to start running the new process, and wipes out every piece of state related to the current process (the one being replaced). And when the new process terminates, the OS releases all resources (including the PID) associated with that process.
So if you really wanted to do this, you would have to hijack how processes terminate in order to restart your original process rather than let the OS clean everything up. How can you do this? Well, execle and execvpe functions allow a program to specify the environment of the new process before starting the process. Since every process depends on libc (or equivalent) to bring up/tear down a process, you should be able to provide a custom libc which would re-start execution of your script, or exec another process. The great difficulty would be hacking such a libc. Additionally, you would have to figure out a good way for your master program to keep state even though the OS wipes away any memory it might have been using when it called exec. You can probably accomplish this with temporary files.
So long story short, don't do it. While it's kind of fun for me to sit here and think about the massive hacks that would be required to pull this off, it would be a huge pain and I'm sure there is a much more elegant solution to whatever problem you are actually trying to solve.
The PID is assigned by OS when shell creates a new process. There is no way to tell the OS to use some specific PID. So it's not possible.
Fixed:
Well this seems a bit silly. Turns out top was not displaying correctly and programs actually continue to run. Perhaps the CPU time became too large to display? Either way, the program seems to be working fine and this whole question was moot.
Thanks (and sorry for the silly question).
Original Q:
I am running a simulation on a computer running Ubuntu server 10.04.3. Short runs (<24 hours) run fine, but long runs eventually stall. By stall, I mean that the program no longer gets any CPU time, but it still holds all information in memory. In order to run these simulations, I SSH and nohup the program and pipe any output to a file.
Miscellaneous information:
The system is definitely not running out of RAM. The program does not need to read or write to the hard drive until completion; the computation is done completely in memory. The program is not killed, as it still has a PID after it stalls. I am using openmp, but have increased the max number of processes and the max time is unlimited. I am finding the largest eigenvalues of a matrix using the ARPACK fortran library.
Any thoughts on what is causing this behavior or how to resume my currently stalled program?
Thanks
I assume this is an OpenMP program from your tags, though you never actually state this. Is ARPACK threadsafe?
It sounds like you are hitting a deadlock (more common in MPI programs than OpenMP, but it's definitely possible). The first thing to do is to compile with debugging flags on, then the next time you find this problem, attach with a debugger and find out what the various threads are doing. For gdb, for instance, some instructions for switching between threads are shown here.
Next time your program "stalls", attach GDB to it and do thread apply all where.
If all your threads are blocked waiting for some mutex, you have a
deadlock.
If they are waiting for something else (e.g. read), then you need to figure out what prevents the operation from completing.
Generally on UNIX you don't need to rebuild with debug flags on to get a meaningful stack trace. You wouldn't get file/line numbers, but they may not be necessary to diagnose the problem.
A possible way of understanding what a running program (that is, a process) is doing is to attach a debugger to it with gdb program *pid* (which works well only when the program has been compiled with debugging enabled with -g), or to use strace on it, using strace -p *pid*. the strace command is an utility (technically, a specialized debugger built above the ptrace system call interface) which shows you all the system calls done by a program or a process.
There is also a variant, called ltrace that intercepts the call to functions in dynamic libraries.
To get a feeling of it, try for instance strace ls
Of course, strace won't help you much if the running program is not doing any system calls.
Regards.
Basile Starynkevitch