Suspicious processes when running ps -Af - linux

Can anyone explain for me what to do if i see this on my PS list?!
apache 23687 2388 5 12:40 ? 00:00:02 /usr/sbin/httpd -DFOREGROUND
postfix 23830 1971 0 12:40 ? 00:00:00 cleanup -z -t unix -u
apache 24167 1 0 Jun13 ? 00:00:17 /tmp/wwrr10/wwrr10
apache 24235 1 0 Jun13 ? 00:00:17 /tmp/wwrr/wwrr
apache 24276 1 0 Jun13 ? 00:00:17 /tmp/wwrr20/wwrr20
apache 25509 1 0 Jun11 ? 00:00:30 /tmp/sdvvrtw/sdvvrtw
apache 25546 1 0 Jun11 ? 00:00:30 /tmp/sdvvrtw10/sdvvrtw10
apache 25576 1 0 Jun11 ? 00:00:31 /tmp/sdvvrtw20/sdvvrtw20
apache 29100 1 0 Jun15 ? 00:00:03 /tmp/bgbh/bgbh
apache 29120 1 0 Jun15 ? 00:00:03 /tmp/bgbh10/bgbh10
apache 29140 1 0 Jun15 ? 00:00:03 /tmp/bgbh20/bgbh20

Your Apache installation may be using prefork. This mpm spawns a process per request. So if you are under a bit of load and nowadays knowing how many concurrent requests some browsers send, that list of process is quite usual. if this is the case it can be a misconfiguration in your php files making them behave incorrectly.
You can also try to reducing the number of of clients in the httpd.conf file.

Related

Multiple httpd processes running in Docker Container

This is the Dockerfile I created for installing httpd on centos:
#Installing HTTPD
FROM centos:latest
MAINTAINER xxx#gmail.com
RUN yum install -y httpd
EXPOSE 80
#ENTRYPOINT ["systemctl"]
ENTRYPOINT ["/usr/sbin/httpd"]
After building, when I run the container I can see too many httpd process running inside this container:
docker run -d -p 80:80 httpd:4.0 -DFOREGROUND
Output of Docker top command:
UID PID PPID C STIME TTY TIME CMD
root 2457 2443 0 04:26 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND
apache 2474 2457 0 04:26 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND
apache 2475 2457 0 04:26 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND
apache 2476 2457 0 04:26 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND
apache 2477 2457 0 04:26 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND
apache 2478 2457 0 04:26 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND
apache 2491 2457 0 04:26 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND
apache 2492 2457 0 04:26 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND
apache 2493 2457 0 04:26 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND
root 2512 2500 0 04:27 pts/0 00:00:00 /bin/bash
apache 2532 2457 0 04:27 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND
Please let me know why so many httpd processes running and how to have only one process with PID 1 ?
Apache runs multiple processes to be ready catch up a client request fast, because spawning a server process is slow, so it is better to have one ready when request comes in.
You can configure their number in httpd.conf through StartServers, MinSpareServers, MaxSpareServers and ServerLimit directives.

apache2 multiple instances under www-data

I am running apache2 on my RaspberryPi, mainly to interface with an mpd php client for streaming audio. After a month or so, I see the following:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1496 www-data 20 0 55900 17m 2112 S 0.0 9.3 0:30.32 apache2
7198 www-data 20 0 54868 15m 2188 S 0.0 8.4 0:10.57 apache2
7182 www-data 20 0 54868 15m 2168 S 0.0 8.3 0:11.67 apache2
1497 www-data 20 0 53844 15m 2132 S 0.0 8.2 0:07.58 apache2
2609 mysql 20 0 314m 15m 280 S 0.7 8.1 71:58.52 mysqld
7185 www-data 20 0 54868 14m 2180 S 0.0 8.1 0:08.71 apache2
7183 www-data 20 0 54868 14m 2120 S 0.0 8.1 0:14.36 apache2
1499 www-data 20 0 53844 14m 2144 S 0.0 8.0 0:07.73 apache2
1932 mpd 20 0 81204 8152 584 S 0.0 4.3 145:46.25 mpd
7211 www-data 20 0 45652 8004 2204 S 0.0 4.2 0:01.65 apache2
3318 www-data 20 0 45652 7944 2140 S 0.0 4.2 0:03.43 apache2
7210 www-data 20 0 45652 7784 2176 S 0.0 4.1 0:01.28 apache2
1965 root 20 0 44532 5268 216 S 0.0 2.8 1:53.06 apache2
7168 www-data 20 0 45652 7956 2140 S 0.0 4.2 0:02.42 apache2
Along with mpd and mysql, and the root apache2 process, 11 apache2 process running as www-data. On reboot, I see 5 apache2 processes under www-data.
Why are more processes spawnded, and not closed down? I continue to see this grow until there are 20+ processes, which slows down the something this small with limited resources.
Why are new processes spawned, then persist? Can I control this in conf.d (I have tried, but the feedback process takes a few days or week, so it's hard to tell).
Apache - when running in pre-fork mode - spawns a pool of worker processes in order to keep the response time low. Every worker will be responsible for a certain request. So, if there are 11 workers running, apache could serve 11 requests in "parallel" without spawning a new process (what would take a significant amount of time)
Apache spawns that workers / and keeps them alive intelligently, but you can set the maximum and minimum amount of workers in the apache2.conf

Memory usage with high precision

How can I get high precision memory usage per proccess with "ps aux"?
$ ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.0 3672 1984 ? Ss Dec11 0:07 /sbin/init
root 2 0.0 0.0 0 0 ? S Dec11 0:00 [kthreadd]
root 3 0.0 0.0 0 0 ? S Dec11 0:23 [ksoftirqd/0]
root 6 0.0 0.0 0 0 ? S Dec11 0:00 [migration/0]
...
I need more than 1 digit after point.
Maybe I can format column with %MEM?
Look into the proc filesystem /proc/[pid]/status, /proc/[pid]/statm, /proc/[pid]/smaps.
To get fully detailed memory map /proc/[pid]/maps
Read the proc(5) manual page for all the details.
the ps command have that. you can type man ps for detail.
when the termianl show info after you typed it, you can type /memory , then heighlight contain 'memory''s string. you can type n show next palace with memory.

How to detect system power events such as suspend, hibernate, etc

I want to make a Linux application that send a message to a remote host when a local system (where the application runs) is going to suspend, hibernate or shutdown. I googled how to do this and found acpi_listen but it doesn't work.
Here is the result of ps aux | grep acpi on the system I tested (Ubuntu 10.04 LTS):
root 35 0.0 0.0 0 0 ? S Dec03 0:00 [kacpid]
root 36 0.0 0.0 0 0 ? S Dec03 0:00 [kacpi_notify]
root 37 0.0 0.0 0 0 ? S Dec03 0:00 [kacpi_hotplug]
root 934 0.0 0.0 2048 872 ? Ss Dec03 0:00 acpid -c /etc/acpi/events -s /var/run/acpid.socket
Communicating with acpid directly via a UNIX domain socket wouldn't work since that's the way acpi_listen works, although I didn't try it. Is there any other way?
Listen for them over D-Bus.

How is it possible that kill -9 for a process on Linux has no effect?

I'm writing a plugin to highlight text strings automatically as you visit a web site. It's like the highlight search results but automatic and for many words; it could be used for people with allergies to make words really stand out, for example, when they browse a food site.
But I have problem. When I try to close an empty, fresh FF window, it somehow blocks the whole process. When I kill the process, all the windows vanish, but the Firefox process stays alive (parent PID is 1, doesn't listen to any signals, has lots of resources open, still eats CPU, but won't budge).
So two questions:
How is it even possible for a process not to listen to kill -9 (neither as user nor as root)?
Is there anything I can do but a reboot?
[EDIT] This is the offending process:
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
digulla 16688 4.3 4.2 784476 345464 pts/14 D Mar28 75:02 /opt/firefox-3.0/firefox-bin
Same with ps -ef | grep firefox
UID PID PPID C STIME TTY TIME CMD
digulla 16688 1 4 Mar28 pts/14 01:15:02 /opt/firefox-3.0/firefox-bin
It's the only process left. As you can see, it's not a zombie, it's running! It doesn't listen to kill -9, no matter if I kill by PID or name! If I try to connect with strace, then the strace also hangs and can't be killed. There is no output, either. My guess is that FF hangs in some kernel routine but which?
[EDIT2] Based on feedback by sigjuice:
ps axopid,comm,wchan
can show you in which kernel routine a process hangs. In my case, the offending plugin was the Beagle Indexer (openSUSE 11.1). After disabling the plugin, FF was a quick and happy fox again.
As noted in comments to the OP, a process status (STAT) of D indicates that the process is in an "uninterruptible sleep" state. In real-world terms, this generally means that it's waiting on I/O and can't/won't do anything - including dying - until that I/O operation completes.
Processes in a D state will normally only be there for a fraction of a second before the operation completes and they return to R/S. In my experience, if a process gets stuck in D, it's most often trying to communicate with an unreachable NFS or other remote filesystem, trying to access a failing hard drive, or making use of some piece of hardware by way of a flaky device driver. In such cases, the only way to recover and allow the process to die is to either get the fs/drive/hardware back up and running so the I/O can complete or to give up and reboot the system. In the specific case of NFS, the mount may also eventually time out and return from the I/O operation (with a failure code), but this is dependent on the mount options and it's very common for NFS mounts to be set to wait forever.
This is distinct from a zombie process, which will have a status of Z.
Double-check that the parent-id is really 1. If not, and this is firefox, first try sudo killall -9 firefox-bin. After that, try killing the specific process IDs individually with sudo killall -9 [process-id].
How is it even possible for a process not to listen to kill -9 (neiter as user nor as root)?
If a process has gone <defunct> and then becomes a zombie with a parent of 1, you can't kill it manually; only init can. Zombie processes are already dead and gone - they've lost the ability to be killed as they are no longer processes, only a process table entry and its associated exit code, waiting to be collected. You need to kill the parent, and you can't kill init for obvious reasons.
But see here for more general information. A reboot will kill everything, naturally.
Is it possible, that this process is restarted (for example by init) just at the time you kill it?
You can check this easily. If the PID is the same after kill -9 PID then the process wasn't killed, but if it has changed the process has been restarted.
I lately get trapped into a pitfall of Double Fork and had landed to this page before finally finding my answer. The symptoms are identical even if the problem is not the same:
WYKINWYT :What You Kill Is Not What You Thought
The minimal test code is shown below based on an example for an SNMP Daemon
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <signal.h>
int main(int argc, char* argv[])
{
//We omit the -f option (do not Fork) to reproduce the problem
char * options[]={"/usr/local/sbin/snmpd",/*"-f","*/-d","--master=agentx", "-Dagentx","--agentXSocket=tcp:localhost:1706", "udp:10161", (char*) NULL};
pid_t pid = fork();
if ( 0 > pid ) return -1;
switch(pid)
{
case 0:
{ //Child launches SNMP daemon
execv(options[0],options);
exit(-2);
break;
}
default:
{
sleep(10); //Simulate "long" activity
kill(pid,SIGTERM);//kill what should be child,
//i.e the SNMP daemon I assume
printf("Signal sent to %d\n",pid);
sleep(10); //Simulate "long" operation before closing
waitpid(pid);
printf("SNMP should be now down\n");
getchar();//Blocking (for observation only)
break;
}
}
printf("Bye!\n");
}
During the first phase the main process (7699) launches the SNMP daemon (7700) but we can see that this one is now Defunct/Zombie. Beside we can see another process (7702) with the options we specified
[nils#localhost ~]$ ps -ef | tail
root 7439 2 0 23:00 ? 00:00:00 [kworker/1:0]
root 7494 2 0 23:03 ? 00:00:00 [kworker/0:1]
root 7544 2 0 23:08 ? 00:00:00 [kworker/0:2]
root 7605 2 0 23:10 ? 00:00:00 [kworker/1:2]
root 7698 729 0 23:11 ? 00:00:00 sleep 60
nils 7699 2832 0 23:11 pts/0 00:00:00 ./main
nils 7700 7699 0 23:11 pts/0 00:00:00 [snmpd] <defunct>
nils 7702 1 0 23:11 ? 00:00:00 /usr/local/sbin/snmpd -Lo -d --master=agentx -Dagentx --agentXSocket=tcp:localhost:1706 udp:10161
nils 7727 3706 0 23:11 pts/1 00:00:00 ps -ef
nils 7728 3706 0 23:11 pts/1 00:00:00 tail
After the 10 sec simulated we will try to kill the only process we know (7700). What we succeed at last with waitpid(). But Process 7702 is still here
[nils#localhost ~]$ ps -ef | tail
root 7431 2 0 23:00 ? 00:00:00 [kworker/u256:1]
root 7439 2 0 23:00 ? 00:00:00 [kworker/1:0]
root 7494 2 0 23:03 ? 00:00:00 [kworker/0:1]
root 7544 2 0 23:08 ? 00:00:00 [kworker/0:2]
root 7605 2 0 23:10 ? 00:00:00 [kworker/1:2]
root 7698 729 0 23:11 ? 00:00:00 sleep 60
nils 7699 2832 0 23:11 pts/0 00:00:00 ./main
nils 7702 1 0 23:11 ? 00:00:00 /usr/local/sbin/snmpd -Lo -d --master=agentx -Dagentx --agentXSocket=tcp:localhost:1706 udp:10161
nils 7751 3706 0 23:12 pts/1 00:00:00 ps -ef
nils 7752 3706 0 23:12 pts/1 00:00:00 tail
After giving a character to the getchar() function our main process terminates but the SNMP daemon with the pid 7002 is still here
[nils#localhost ~]$ ps -ef | tail
postfix 7399 1511 0 22:58 ? 00:00:00 pickup -l -t unix -u
root 7431 2 0 23:00 ? 00:00:00 [kworker/u256:1]
root 7439 2 0 23:00 ? 00:00:00 [kworker/1:0]
root 7494 2 0 23:03 ? 00:00:00 [kworker/0:1]
root 7544 2 0 23:08 ? 00:00:00 [kworker/0:2]
root 7605 2 0 23:10 ? 00:00:00 [kworker/1:2]
root 7698 729 0 23:11 ? 00:00:00 sleep 60
nils 7702 1 0 23:11 ? 00:00:00 /usr/local/sbin/snmpd -Lo -d --master=agentx -Dagentx --agentXSocket=tcp:localhost:1706 udp:10161
nils 7765 3706 0 23:12 pts/1 00:00:00 ps -ef
nils 7766 3706 0 23:12 pts/1 00:00:00 tail
Conclusion
The fact that we ignored the double fork mechanism made us think that the kill action did not succeed. But in fact we simply killed the wrong process !!
By adding the -f option ( Do Not (Double) Fork ) all go as expected
ps -ef | grep firefox;
and you can see 3 process, kill them all.
sudo killall -9 firefox
Should work
EDIT: [PID] changed to firefox
You can also do a pstree and kill the parent. This makes sure that you get the entire offending process tree and not just the leaf.

Resources