I tried to search this in many places and also documents/man pages of openlog(), syslog(0, rsyslogd(8) but couldn't find answer for this.
My question is, if rsyslogd is stopped or not yet started, then do the new syslog messages get lost? Or rsyslogd fetches them from /dev/log later when it's enabled?
My test is:
On a running system, rsyslog is running. Now, do the following:
logger -p local7.notice "my custom message1"
grep message1 | /var/log/messages ----> Success
Stop rsyslogd process
logger -p local7.notice "My other custom message2"
now, start the rsyslogd daemon
grep message2 | /var/log/messages ----> FAIL
I understand from openlog(3) and syslog(3) man pages that a socket is opened for /dev/log file and if there is an error while sending the message to syslog (as rsyslogd is not running) then the connection is closed (and message is printed on console/stderror if you have used LOG_CONS/LOG_PERROR).
Could anybody please tell me:
Is there any way rsyslogd to get all those messages came in absence of it in syslog file when it comes up?
If not by default, is there any syscall, command,etc.etc.way to do that??
Thank you in advance.
-Neo
It won't happen by default. You can use the 'cat' command and pipe it to logger to get them in, though. Something like the following should work.
cat your.log | logger -n yourserver
You can also use the 'tail' command similarly to 'cat'.
Related
I'm trying to setup a robust AF_UNIX data transfer.
So I make this test (on a Ubuntu 20.04):
I generate logs :
for a in `seq 1 1000000`; do echo $a | logger; echo $a ; done
I check my log file :
tail -f /var/log/syslog | less
I can see my logs logged by rsyslog in this file.
I stop rsyslog :
systemctl stop syslog.socket
No logs arrived anymore in /var/log/syslog as expected.
But when I restart rsyslog :
systemctl start syslog.socket
I can see that the logs arrvie again but most of the one created during the rsyslog outage are missing.
I tired to increase the max_dgram_qlen
echo 5000000 > /proc/sys/net/unix/max_dgram_qlen
but it doesn't change anything.
I read on this post What is the max size of AF_UNIX datagram message in Linux? that I can monitor the socket using:
ss -ax | grep log
but the counters looked to be at zero all the time whether rsyslog is started or not.
What I trying to do :
increase the buffer to be able to have at least 500MB or 500,000 messages in queue
be able to monitor the size of the queue : size or queue length
if possible logging in dmesg when messages are dropped.
How to send a message to rsyslog deamon programmatically (from a custom program)?
In the syslog there are openlog...syslog...closelog functions available. But how can i do it in case of rsyslog?
rsyslog is a central log program.
You could cat /etc/rsyslog.conf to watch how your rsyslog is configured.
Default configuration use imuxsock and imjournal.
imuxsock module actually listen like /dev/log or /run/systemd/journal/syslog. This means you could use syslog(3) man 3 syslog or cmd logger 123 to write log into /var/log/messages.
imjournal means rsyslog read log from systemd-journald(/var/log/journal/$(uuid)/*.journal. You could use sd-journal(3) api or journal cmd like echo 123 | systemd-cat to write to journal, and then rsyslog read log from it. you can see journal with journalctl -e to see the newest journal.
I want to run a game server inside my Ubuntu machine. I want to run it in the background and write the live output of that process inside a log file. I tried using nohup and running the game server using "&" at the end but I couldn't make it work the way I wanted.
Then I started reading about named pipes and actually gave it a go. I made a simple script that in theory should work. But, of course I am missing something.
First, I made a pipe using the mkfifo command.
mkfifo testpipe
Then I created a small script:
#!/bin/bash
./mta-server64 > pipe &
pid=$!
echo $pid // so I know the pid of the process
cat < pipe > log.txt &
(Note: I wrote this code from memory.)
The code works only when there is an error and the process stops. It actually records the game console error. But when the game server is running I get no output in the log file.
I want to read the output (stdout and stderr if I am not mistaken) of a process running in background and record it those inside a log file.
I also thought about using screen as it logs everything inside a file but I would prefer not using it if there is a better solution.
EDIT:
First of all: thank you for the interest you had in helping me. In the same way, I have to apologize for only giving scarce details about what I intend to do with this small project and for my limited understanding of stdout and stderr.
Let's go to the first base.
I want to run a game server named Multi Theft Auto (https://multitheftauto.com/). This is GTA San Andreas but multiplayer.
I can easily run this game server in my Ubuntu server by calling the executable ./mta-server-64. After calling it the game server console appears:
[|] MTA: San Andreas :: 0/32 players :: 196 resources :: 125 fps (25)
MTA:BLUE Server for MTA:SA
==================================================================
= Multi Theft Auto: San Andreas v1.5.6 [64 bit]
==================================================================
= Server name : Default MTA Server
= Server IP address: auto
= Server port : 22884
=
= Log file : /root/mta/mods/deathmatch/logs/server.log
= Maximum players : 32
= HTTP port : 22564
= Voice Chat : Disabled
= Bandwidth saving : Medium
==================================================================
[09:49:07] Resource 'mapmanager' requests some acl rights. Use the command 'aclrequest list mapmanager'
[09:49:07] Resources: 196 loaded, 0 failed
[09:49:07] Starting resources...
[09:49:07] Server minclientversion is now 1.5.6-9.16588.0
[09:49:07] INFO: MAPMANAGER: Some important ACL permissions are missing. To ensure the correct functioning of Mapmanager, please write: aclrequest allow mapmanager all
[09:49:07] Gamemode 'play' started.
[09:49:07] Authorized serial account protection is enabled for the ACL group(s): `Admin` See http://mtasa.com/authserial
[09:49:07] WARNING: <owner_email_address> not set
[09:49:07] Server started and is ready to accept connections!
[09:49:07] To stop the server, type 'shutdown' or press Ctrl-C
[09:49:07] Type 'help' for a list of commands.
[09:49:07] Querying MTA master server... success! (Auto detected IP:xxx.xxx.xxx.xxx)
I am using the following script to run the process in the background and (try to) get the live output from:
#!/bin/bash
newport=$(shuf -i 22003-22900 -n 1)
newip=$(shuf -i 22003-22900 -n 1)
rm -rf ~/server/*
cp -r /home/user*/ftp/server/mtaserver/serverfiles/* ~/server
sed -i "s/<httpport>[0-9][0-9][0-9][0-9][0-9]<\/httpport>/<httpport>$newport<\/httpport>/g" ~/server/mods/deathmatch/mtaserver.conf
sed -i "s/<serverport>[0-9][0-9][0-9][0-9][0-9]<\/serverport>/<serverport>$newip<\/serverport>/g" ~/server/mods/deathmatch/mtaserver.conf
~/server/mta-server64 2>&1 | tee -a outfile &
mta_pid=$!
echo $mta_pid
sleep 6
pkill $mta_pid
(Note: Because of some technical problems I had to add the first few lines of script which automatically replace the game files with new ones and also replace the existing ports with random ones.)
This script starts the server and tries to log the output of the process. The process is automatically killed after few seconds so there is only one instance of the game server at any given time.
THE ISSUE:
This script only logs the output if there is an error. I still cannot get the live output of the process when it is still running. Maybe this is an issue with the game server but truly believe there should be a way to make it work the way I intend.
I believe you want to use tee command to split the pipe output to log file.
I suggest you read this article and these answers 1 2.
Usually this is enough nohup somecommand > somecommand.log 2>&1 & then, tail -F somecommand.log to follow the logs.
After 2 days I finally figured out a way to make it work (the way I intended to work, without taking in consideration any major security/performance risks).
Reading the comments made me realize I was attacking the wrong point. The stdout of the game server is buffered, thus making it impossible to log it into a log file using the methods I tried when I posted my question At least this is what I came to understand).
I did some research on how to run the application without having the stdout buffered: https://serverfault.com/questions/294218/is-there-a-way-to-redirect-output-to-a-file-without-buffering-on-unix-linux
My code now:
stdbuf -o0 ~/server/mta-server64 >> pipe &
cat < pipe | tee -a outfile &
After creating the named pipe it executes the game server inside that pipe and then appends the stdout into the log file.
The stdbug -o0 command disables the stdout buffering (as noted in the link above).
This works for me and I cannot guarantee it will work for anybody else. I am still not aware if disabling the buffering is a safe approach to my issue but for now it is what I need.
I am outputting content of a log via netcat to an application over the network. I don't know if what I'm doing is the most efficient, especially when I notice the netcat session becomes non-responsive. I have to stop netcat and start it again for the application to work again.
The command I run is:
/bin/tail -n1 -f /var/log/custom_output.log | /bin/nc -l -p 5020 --keep-open
This needs to run like this 24/7. Is this the most efficient way of doing it? How can I improve on it so I don't have to restart the process daily?
EDIT
So I realised that when the log is being rotated, netcat is locked onto a file that's not longer being written to. I can deal with this easily enough.
The question still stands. Is this the best way to do something like this?
It's been 6 years, but maybe someone will come in handy.
To account for log rotation, use tail with the -F flag.
nc (aka netcat) variant
LOG_FILE="/var/log/custom_output.log"
PORT=5020
tail -n0 -F "$LOG_FILE" | nc -k -l -p $PORT
Notes:
Flag -k in nc is analog to --keep-open in "the OpenBSD rewrite of netcat";
Multiple clients can connect to nc at the same time, but only the first one will be receive appended log lines;
tail will run immediately, so it will collect appended log lines even if no client is connected. Thus, the first client can receive some buffered data - all log lines that have been appended since tail was run.
socat variant
LOG_FILE="/var/log/custom_output.log"
PORT=5020
socat TCP-LISTEN:$PORT,fork,reuseaddr SYSTEM:"tail -n0 -F \"$LOG_FILE\" </dev/null"
Note: here socat will fork (clone itself) on each client connection and start a separate tail process. Thus:
Each connected client will receive appended log lines at the same time;
Clients will not receive any previously buffered by tail strings.
additional
You can redirect stderr to stdout in the tail process by adding 2>&1 (in both variants). In this case, clients will receive auxiliary message lines, e.g.:
tail: /var/log/custom_output.log: file truncated;
tail: '/var/log/custom_output.log' has become inaccessible: No such file or directory - printed when the log file has been removed or renamed, only if -F is used;
tail: '/var/log/custom_output.log' has appeared; following new file - printed when a new log file is created, only if -F is used.
I'm trying to write a script that informs the user when someone has logged in on the machine via ssh.
My current idea is to parse the output of "w" using grep in intervals.
But that's neither elegant nor performant. Has anyone got a better idea how to implement such a program?
Any help would really be appreciated!
Paul Tomblin has the right suggestion.
Set up logging in your sshd_config to point to a syslog facility that you can log separately:
=> see man 3 syslog for more facilities. Choose one like e.g.
# Logging
SyslogFacility local5
LogLevel INFO
Then set up your syslog.conf like this:
local5.info |/var/run/mysshwatcher.pipe
Add the script you're going to write to /etc/inittab so it keeps running:
sw0:2345:respawn:/usr/local/bin/mysshwatcher.sh
then write your script:
#!/bin/sh
P=/var/run/mysshwatcher.pipe
test -p $P || mkfifo $P
while read x <$P; do
# ... whatever, e.g.:
echo "ssh info: $x" | wall
done;
Finally, restart your syslogd and get your inittab reloaded (init q) and it should work. If other variantes of these services are used, you need to configure things accordingly (e.g. newsyslogd => /etc/newsyslog.conf; Ubuntu: /etc/event.d isntead of inittab)
This is very rudimentary and lacking, but should be enough to get you started ...
more info: man sshd_config for more logging options/verbosity.
On Ubuntu (and I'd guess all other Debian distros, if not all Linuces), the file /var/log/auth.log records successful (and unsuccessful) login attempts:
sshd[XXX]: pam_unix(sshd:session): session opened for user XXX
You could set up a very simple monitor using this command (note that you have to be root to see the auth log):
sudo tail -F /var/log/auth.log | grep sshd
If you do not care how they logged in (telnet/ssh), the 'last' Unix command line utility shows you the last few logins in the machine. Remote users will show the IP address
[root#ex02 www]# last
foo pts/1 81.31.x.y Sun Jan 18 07:25 still logged in
foo pts/0 81.31.x.y Sun Jan 18 01:51 still logged in
foo pts/0 81.31.x.y Sat Jan 17 03:51 - 07:52 (04:00)
bar pts/5 199.146.x.y Fri Jan 16 08:57 - 13:29 (04:32
Set up a named pipe, and set up a log file parser to listen to it, and send the ssh messages to it. The log file parser can do what you want, or signal to a daemon to do it.
Redirecting the log file is done in a config file in /etc/ whose name escapes me right now. /etc/syslog.conf, I think.
I have made a program (which i call Authentication Monitor) that solves the task described in the question.
If you wanted to, you are more than welcome to download it to investigate how I solve this problem (using log-files).
You can find Authentication Monitor freely available here: http://bwyan.dk/?p=1744
We had the same problem, so we wrote our own script.
It can be downloaded from the github.
Hope it helps :)
cheers!
Ivan