I think my server has been compromised and it has many perl processes running. However, I don't know what file they are being launched from so I can delete it. How can I find this information?
If your system has been hacked, you cannot trust any of the software, not even the kernel. Format the disk and re-install everything. There is just no way to be sure you've cleaned out the infection, because you can't trust the very tools you would use to clean things. You can't copy new tools onto the box, because you can't trust the SSH daemon or the /bin/cp command. Anything -- ls, vi, ps, cat, dd, etc. -- could have been replaced with a trojan that works to hide the infected files.
You could check the symbolic link /proc/pid/cwd, also check the ppid from ps(1).
The first thing I would do is look at the parent process id (PPID). That said, if the PPID is 1, that doesn't tell you anything.
Auditing the filesystem could help see here
pstree could also help
If you run the command "ps -ef" you should get a list of all processes running on your machine. Each process will have a process id number (PID), and also a parent PID. Find the offending process(es) and check their parent PIDs. Then find the process with a matching PID, and it should be your culprit.
Try ls -l /proc/<pid>/exe, or ls -l /proc/<pid>/fd. I don't remember if perl keeps the script file open after the program starts, but if it does, it will be one of the process's file descriptors.
But if your system is pwned, don't expect anything to make sense.
Related
I'm using CentOS 7.5.1804.
Right after booting-up, the operating system delays.
For example, when I try to write "python" in a terminal,
first I write "pyt" and press .
I have to wait a few seconds for the OS to interpolate to "python".
This phenomenon occurs just after booting-up.
After a few days later, this phenomenon goes away.
Does anyone know a clue to solve this problem?
The bit when you press pyt-"tab" is part of bash-completion package as the command completion happens after you typed the full command. So the cause has to be investigated starting with bash. My educated guess is that some process or I/O is keeping the system busy.
You can start with some generic system information tools as soon as the system start:
uptime to see the system load
vmstat -n 1 to check the status of the CPU
ps aux to check running processes
iotop to check for I/O
systemctl list-jobs to show running jobs in systemd
and based on the result of them perform deeper analysis.
Another thing might be the access to the disk slowing down the systemt at startup. Where is the machine running?
I don't know about fixing — there are all kinds of things that could go cause delays. But I can offer a few tips to investigate.
The first step to investigate is to run set -x to get a trace of the commands that the shell executes to generate the completions. Watch where it pauses.
do you have the issue with different auto-completion? if its only python you can time the execution of your command
time python
you can watch if you have some problems at launch with redirect standar output and error to a file.
strace python 2>&1 launch.log
take a strace at boot and one later then you can check if there is difference between:
diff -u delays.log delays2.log | grep ^+
hope it can help.
RH6. Is it possible to issue, for example, a cd command in a running xterm process FROM a different process? I know the pid of the existing xterm proc. I actually want to "echo" a message first, and then cd. Something like...
echo "Your time in this workarea has expired. You are being sent home"
cd ~
It would be great if I could do this as a different user somehow (not the owner of the target proc) (I am not and cannot be root). But if that is not possible, perhaps having the target xterm owner create an executable which wraps these commands inside, and then setting the sticky bit on the executable might work when the 2nd proc goes to run it. Not sure if lint checking will catch this as some sort of foul.
I would just make this a comment, but I don't have enough reputation. But I think this might be on the right track:
https://serverfault.com/questions/178457/can-i-send-some-text-to-the-stdin-of-an-active-process-running-in-a-screen-sessi
I'm writing a Bash script to monitor a process and detect when it has crashed. To do this, I am monitoring the /proc directory;
start_my_process;
my_process_id=$!;
until [[ ! -d "/proc/$my_process_pid" ]]; do
# alert the process is dead and restart it...
done
Can I be guaranteed that the process's entry in /proc/ will be created BEFORE Bash finishes executing the command to start the process? Or is it possible that by time my check above is executed, the entry for start_my_process might not yet be created?
EDIT:
In the end I actually went against a custom solution and chose monit which is an excellent watchdog tool.
/proc/<pid> is never created. It is not a real directory.
/proc is a virtual filesystem. When you open one of its "files" and read from its output stream, the data are being provided by the kernel. Since the kernel is also responsible for managing process <pid>, the kernel will tell you that /proc/<pid> directory exists as soon as and for as long as the kernel is keeping track of it.
Since bash won't be able to set $! until the process exists, you are definitely safe checking for the process's virtual directory under /proc after that time.
I'm a PHP developer, and know very little about shell scripting... So I appreciate any help here.
I have four php scripts that I need running in the background on my server. I can launch them just fine - they work just fine - and I can kill them by looking up their PID.
The problem is I need my script to, from time to time, kill the processes and restart them, as they maintain long standing HTTP requests that sometimes are ended by the other side.
But I don't know how to write a command that'll find these processes and kill them without looking up the PID manually.
We'll start with one launch command :
/usr/local/php5/bin/php -f /home/path/to/php_script.php > /dev/null &
Is there a way to "assign" a PID so it's always the same? or give the process a name? and how would I go about writing that new command?
Thank you!
Nope, you can't "assign" the process PID; instead, you should do as "real" daemons do: make your script save its own PID in some file, and then read it from that file when you need to kill.
Alternative would be to use something like supervisor, that handles all that for you in a quite nice way.
Update - supervisor configuration
Since I mentioned supervisor, I'm also posting here a short supervisor configuration file that should do the job.
[program:yourscriptname]
command=/usr/local/php5/bin/php -f /home/path/to/php_script.php
Have a look here for more configuration options.
Then you can use it like this:
# supervisorctl status
to show the process(es) status.
# supervisorctl start yourscriptname
to start your script
# supervisorctl stop yourscriptname
to stop your script
Update - real world supervisor configuration example
First of all, make sure you have this in your /etc/supervisor/supervisord.conf.
[include]
files = /etc/supervisor/conf.d/*.conf
if not, just add those two lines and
mkdir /etc/supervisor/conf.d/
Then, create a configurtion file for each process you want to launch:
/etc/supervisor/conf.d/script1.conf
[program:script1]
command=/usr/local/php5/bin/php -f /home/path/to/php_script.php
stdout_logfile=/var/log/script1.log
stderr_logfile=/var/log/script1-error.log
/etc/supervisor/conf.d/script2.conf
[program:script2]
command=/usr/local/php5/bin/php -f /home/path/to/php_script2.php
stdout_logfile=/var/log/script2.log
stderr_logfile=/var/log/script2-error.log
...etc, etc.. for all your scripts.
(note that you don't need the trailing & as supervisor will handle all the daemonization thing for you; in fact you shouldn't execute programs that are self-daemonizing inside supervisor).
Then you can start 'em all with:
supervisorctl start all
or just one with something like:
supervisorctl start script1
Starting supervisor from php
Of course, you can start/stop the supervisor-controlled processes using the two commands above, even from inside a script.
Remember however that you'll need root privileges, and it's quite risky to allow eg. a web page to execute commands as root on the server..
If that's the case, I recommend you have a look at the instructions on how to run supervisor as a normal user (I never did that, but you should be able to run it as the www-data user too..).
The canonical way to solve this is to have the process write its PID into a file in a known location, and then any utility scripts can look up the file, read the PID, and manipulate that process. Add a command line argument to the script that gives the name of the PID file to write to.
A work around to this would be to use ps aux, this will show all of the processes with the command that called them. This presumes of course that the 4 scripts are different files, or can be uniquely identified by the command that called them. Pipe that through a grep and you're all set ps aux | grep runningscript.php
OK! so this has been a headache and a half for my who knows NOTHING about shell/bash whatever scripting...
#redShadow 's response would had been perfect, except my hosting provider will not give me access to the /etc/supervisor/ directory. as he said, you must be root - and even using sudo was an admin wouldn't let me make any chances there...
Here's what I came up with:
kill -9 `ps -ef | grep php | grep -v grep | awk '{print $2}'`
because the only types of commands I was executing showed up in the top command as php this command loops thru running processes, finds the php commands and their corresponding PIDs and KILLS them! woot!!
What I do is have my script check for a file that I name "run.txt". If it does not
exist, they exit. Then just br renaming that (empty) file, I can stop all my scripts.
I am trying to start a process and although ps -ef|grep myprocessname does not show it running, when I invoke the script to start it it says process already running, exiting.
I have searched internet for about one hour and I can not find any answers. Can anyone help? Thank you.
#TILO: There is no file under /var/run that has a name even close to my process. Any other suggestions?
#VKRAM: This is a third party software. Any suggestions?
check under /var/run if there is a .pid file for the process you're trying to start.
e.g. /var/run/mysqld/mysqld.pid would be such a file.
That file contains the PID of the process...
run a
ps -edaf | grep PID # with the pid you find in the file
if the process is not found, you can delete the pid-file -- then try starting your process again
Try using strace on the program in question:
strace yourprogram
Shortly before it terminates, you should see the system calls it used to determine that another instance was running, and can from there reverse engineer the method it is using.
you said you can't find the PID-file...
If you can't find the PID file (maybe because some of the directories under /var/run are deeply nested),
try this to see a list of all PID-files in there:
find /var/run -type f -name '*.pid'
find /var -type f -name '*.pid'
maybe you'll see a filename that looks similar to the process name you're trying to start.
Or you can also put a grep at the end of that line and try to grep for the process name in the list.
Some programs put use also lock files -- these can be usually found under /var/lock/ or /var/lock/subsystem
If that doesn't help, try to look at the start-script that you're using , e.g. under /etc/init.d/
Look at it in detail and look for something like LOCK_FILE or PID_FILE