So the syslog log is made up of a header (timestamp + hostname) and a message (tag + content). Is any of this customizable? I mean how can i decide the format of timestamp or whether I want hostname to be logged.
Is there someway to do this from the syslog system call or syslog.conf file?
UPDATE:
I'm using syslogd and FREEBSD8
It's pretty much hardcoded for security reasons (you don't want faked messages which look like they came from host X at time T but came from x at t).
If you want to mess with it, you'd have to hack usr.sbin/syslogd/syslogd.c, functions printline() and logmsg().
Syslog's date format, limited as it is, is defined in RFC 5424, which FreeBSD's built-in Syslog follows accurately.
If you want more flexibility or detail, you'll either need to post-process your logs, or switch to a different syslog daemon. Note that by doing so, you may make it impossible for standard log analysis tools to interpret your logs.
One favourite alternative is syslog-ng, which is available in the ports tree.
cd /usr/ports/sysutils/syslog-ng && make install clean
Related
I need to build a small size tcpdump for the embedded project that I am working on. Since the memory size of my embedded device is limited, I need to strip all the unwanted functionality in the TCPDUMP while building it. My target is make the tcpdump executable size less that 300KB. After using "strip tcpdump option" and disabling package options in the configure, I have reached 750KB. To achieve this, I want to remove all the protocol decoding capability of tcpdump. I want the tcpdump to have no more that hex dump capability. I have a below initial list of unwanted protocols that has to be removed.
print-802_11.c
print-802_15_4.c
print-ah.c
print-ahcp.c
print-aodv.c
print-aoe.c
print-ap1394.c
print-atalk.c
print-atm.c
print-babel.c
print-bootp.C
print-bt.c
print-calm-fast.c
print-carp.c
print-cdp.c
print-cfm.c
print-chdlc.c
print-cip.c
print-cnfp.c
print-dccp.c
print-decnet.c
print-dtp.c
print-dvmrp.c
print-eap.c
print-egp.c
print-eigrp.c
print-enc.c
print-esp.c
print-fddi.c
print-forces.c
print-ipx.c
print-isakmp.c
print-isoclns.c
print-juniper.c
print-krb.c
print-lane.c
print-m3ua.c
print-sip.c
print-sl.c
print-sll.c
print-sunatm.c
print-zephyr.c
print-usb.c
print-vjc.c
print-vqp.c
print-timed.c
print-tipc.c
print-token.c
I started to remove these from Makefile.in and removing the function calls manually in the source code. But then I realized this approach is not scalable.
Is there a better way to do this ? Someway by using configure options?
I am new to this. So please explain.
Is there a better way to do this ? Someway by using configure options?
No, there are no such configure options. You'll have to do it the non-scalable way.
"I want to remove all the protocol decoding capability of tcpdump. I
want the tcpdump to have no more that hex dump capability. [...] Is
there a better way to do this ?"
I think there is, but with a very different approach.
If all you want from tcpdump is:
the capability of specifying an interface,
put this interface on promiscuous mode or not, or monitor mode if it's a Wi-Fi interface,
apply a capture filter,
and then spit the output in a file or as hex to stdout,
...you'd be better write your own from scratch, using libpcap (which is what tcpdump uses BTW).
This should be no more than 100-400 lines of C code depending on the options you want to have, you'll have a very, very small executable, and no more dependencies than tcpdump which require libpcap anyway. All the complexity is in the dissection, once you remove all that, what you have is basically... a pcap loop.
It's not that hard, and looks to me as far less work than your approach - and also more interesting work.
There's a tutorial to start with (30-60 minutes read):
http://www.tcpdump.org/pcap.html
...at the end of this tutorial, you'll already have the core of your program.
And you can find loads of info (and ask questions) in the related SO tags:
https://stackoverflow.com/tags/libpcap/info
https://stackoverflow.com/tags/pcap/info
...and have about 70 well-written man pages documenting the full pcap API (you'll end up using maybe 10-20 of these).
i still want to check my Bootloader + Linux Startupcode for an embedded device. Therefore i want to catch the time for every command printed to the serial port.
I know there are programs like putty (which i can dearly recommend), getty, cutecom, picocom, screen etc. But none of these add timestamps to the incomming messages on the host screen (I'm not really talking about the date, more like how many ms have gone since the first output). It actually sounds not like a big deal.
I found out there is one script doing what i wanted to have, called grabserial but it's not working properly, since it's to slow to process the whole output. I discussed this problem in a different forum (if you want to know: grabserial problem but it's not part of the topic). So i can't use that script.
Now again: can you tell me a terminal for Linux which adds timestamps to every line, which was received from a Serial Port?
Thank you
[Edit:] I've found a pretty rough workaround with cereal, which wants to have some settings, since it locks the port everytime you use it. In the end, it adds the actual date and time, not the startup time and difftime between each step, so as you can see I'm still looking for an adequate solution.
This might come 3 years too late, but minicom (https://en.wikipedia.org/wiki/Minicom) supports timestamps for every line printed on the terminal. In Ubuntu it's directly available in the default repos.
You might want to look in to using strace to monitor the serial port. See How can I monitor data on a serial port in Linux?
If you are willing to build the binary by yourself, you can try a branched picocom (https://github.com/codepox/picocom). This is based on picocom 1.7 which is a little old.
I have forked and enhanced this picocom and made it be able to show either delta-time or wall-clock timestamp. You can find it here (https://github.com/tdwong/picocom-with-timestamp). You still have to build the binary by yourself.
Here is how I use it. Note, N is the command to enable/toggle timestamp.
$ picocom -b 115200 /dev/ttyUSB0
...
<ctrl-a> N # enable delta-time timestamp
<ctrl-a> N # toggle wall-clock timestamp
<ctrl-a> N # disable timestamp
tio found at https://tio.github.io provides various timestamp options:
-t, --timestamp
Enable line timestamp.
--timestamp-format <format>
Set timestamp format to any of the following timestamp formats:
24hour 24-hour format ("hh:mm:ss.sss")
24hour-start 24-hour format relative to start time
24hour-delta 24-hour format relative to previous timestamp
iso8601 ISO8601 format ("YYYY-MM-DDThh:mm:ss.sss")
Default format is 24hour
In your case, showing how much time has passed since start, that would be something like this:
tio -t --timestamp-format 24hour-start /dev/ttyUSB0
Settings can also be enabled in tio configuration file ~/.tioconfig
I believe that ExtraPuTTY is the solution you are looking for.
However, I wasn't clear if you wanted something to run ON Linux or just to be able to monitor it (SSH to Linux). If you didn't want a Windows solution, then I apologize.
I have two user processes A and B. Both use syslog using facility LOG_USER.
I want to have different threshold levels for them:
For A, only messages of priority ERR-and-above must be logged
For B, only messages of priority CRIT-and-above must be logged
I found that if I setup /etc/syslog.conf as
user.err /var/log/messages
then messages of ERR-and-above are logged, but, from both A and B.
How can I have different minimum threshold levels for different processes?
Note: I am exploring if there is a config file based solution. Otherwise, there is another approach that works. In each process, we can use setlogmask() to install process specific priority mask.
EDIT (Nov 18): I want to use syslog and some portable solution.
A config file based solution is available. I think CentOS by default ships with rsyslog and even if it does not, you can always install rsyslog with yum. This solution works only with rsyslog and nothing else.
The is a catch, though. You can not separate log messages with rsyslog (or pretty much any syslog daemon implementation) between processes with same name ie. the same executable path. However, rsyslog does allow you to filter messages based on program name. Here lies a possible solution: most programs call openlog(3) using argv[0], ie. the executable name, as the first argument. Now since you don't reveal the actual program you're running, there is no way to determine this for you, but you can always read the sources of your own program, I guess.
In most cases the executable path is the program name, though some daemons do fiddle with argv[0] (notable examples are postfix and sendmail). Rsyslog on the other hand provides a filtering mechanism which allows one to filter messages based on the name of the sending program (you can now probably see how this is all connected to how openlog(3) is called). So, instead of trying to filter directly processes, we can do filtering on program names. And that we can affect by creating symbolic links.
So, this solution only works given following conditions: a) the process you're running does not fiddle with argv[0] after beginning execution; b) you can create symlinks to the binary, thus creating two different names for the same program; c) your program is calling openlog(3) using argv[0] as the first parameter to the call.
Given those two conditions, you can simply filter messages on /etc/rsyslog.conf like this (example directly from rsyslog documentation):
if $programname == 'prog1' then {
action(type="omfile" file="/var/log/prog1.log")
}
if $programname == 'prog2' then {
action(type="omfile" file="/var/log/prog2.log")
}
E.g. if your program is called /usr/bin/foobar and you've created symbolic links /usr/bin/prog1 and /usr/bin/prog2 both pointing at /usr/bin/foobar, the above configuration file example will then direct messages from processes started as "prog1" and "prog2" to different log files respectively. This example will not fiddle with anything else, so all those messages are still going to general log files, unless you filter them out explicitly.
This tutorial http://www.freebsd.org/cgi/man.cgi?query=syslog.conf&sektion=5 helped me. The following seem to work:
# process A: log only error and above
!A
*.err /var/log/messages
# process B: log only critical and above
!B
*.critical /var/log/messages
# all processes other than A and B: log only info and above
!-A,B
*.info /var/log/messages
Let's say I have 10 programs (in terminals) working in tandem: {p1,p2,p3,...,p10}.
It's hard to keep track of all STDOUT debug statements in their respective terminal. I plan to create a GUI to keep track of each STDOUT such that, if I do:
-- Click on p1 would "tail" program 1's output.
-- Click on p3 would "tail" program 4's output.
It's a decent approach but there may be better ideas out there? It's just overwhelming to have 10 terminals; I'd rather have 1 super terminal that keeps track of this.
And unfortunately, linux "screen" is not an option. RESTRICTIONS: I only have the ability to either: redirect STDOUT to a file. (or read directly from STDOUT).
If you are looking for a creative alternative, I would suggest that you look at sockets.
If each program writes to the socket (rather than STDOUT), then your master terminal can act as a server and organize the output.
Now from what you described, it seems as though you are relatively constrained to STDOUT, however it could be possible to do something like this:
# (use netcat (or nc on some systems) to write to a socket on the provided port)
./prog1 | netcat localhost 12312
I'm not sure if this fits in the requirements of what you are doing (and it might be more effort than it is worth!), but it could provide a very stable solution.
EDIT: As was pointed out in the comments, netcat does exactly what you would need to make this work.
We have an application on Linux that used the syslog mechanism. After a week spent trying to figure out why this application was running slower than expected, we discovered that if we eliminated syslog, and just wrote directly to a log file, performance improved dramatically.
I understand why syslog is slower than direct file writes. But I was wondering: Are there ways to configure syslog to optimize its performance?
You can configure syslogd (and rsyslog at least) not to sync the log files after a log message by prepending a "-" to the log file path in the configuration file. This speeds up performance at the expense of the danger that log messages could be lost in a crash.
There are several options to improve syslog performance:
Optimizing out calls with a macro
int LogMask = LOG_UPTO(LOG_WARNING);
#define syslog(a, ...) if ((a) & LogMask ) syslog((a), __VA_ARGS__)
int main(int argc, char **argv)
{
LogMask = setlogmask(LOG_UPTO(LOG_WARNING));
...
}
An advantage of using a macro to filter syslog calls is that the entire call is
reduced to a conditional jump on a global variable, very helpful if you happen to
have DEBUG calls which are translating large datasets through other functions.
setlogmask()
setlogmask(LOG_UPTO(LOG_LEVEL))
setlogmask() will optimize the call by not logging to /dev/log, but the program will
still call the functions used as arguments.
filtering with syslog.conf
*.err /var/log/messages
"check out the man page for syslog.conf for details."
configure syslog to do asynchronous or buffered logging
metalog used to buffer log output and flushed it in blocks. stock syslog and syslog-ng
do not do this as far as I know.
Before embarking in new daemon writing you can check if syslog-ng is faster (or can be configured to be faster) than plain old syslog.
One trick you can use if you control the source to the logging application is to mask out the log level you want in the app itself, instead of in syslog.conf. I did this years ago with an app that generated a huge, huge, huge amount of debug logs. Rather than remove the calls from the production code, we just masked so that debug level calls never got sent to the daemon. I actually found the code, it's Perl but it's just a front to the setlogmask(3) call.
use Sys::Syslog;
# Start system logging
# setlogmask controls what levels we're going to let get through. If we mask
# them off here, then the syslog daemon doesn't need to be concerned by them
# 1 = emerg
# 2 = alert
# 4 = crit
# 8 = err
# 16 = warning
# 32 = notice
# 64 = info
# 128 = debug
Sys::Syslog::setlogsock('unix');
openlog($myname,'pid,cons,nowait','mail');
setlogmask(127); # allow everything but debug
#setlogmask(255); # everything
syslog('debug',"syslog opened");
Not sure why I used decimal instead of a bitmask... shrug
Write your own syslog implementation. :-P
This can be accomplished in two ways.
Write your own LD_PRELOAD hook to override the syslog functions, and make them output to stderr instead. I actually wrote a post about this many years ago: http://marc.info/?m=97175526803720 :-P
Write your own syslog daemon. It's just a simple matter of grabbing datagrams out of /dev/log! :-P
Okay, okay, so these are both facetious answers. Have you profiled syslogd to see where it's choking up most?
You may configure syslogd's level (or facility) to log asynchronously, by putting a minus before path to logfile (ie.: user.* [tab] -/var/log/user.log).
Cheers.
The syslog-async() implementation may help, at the risk of lost log lines / bounded delays at other times.
http://thekelleys.org.uk/syslog-async/
Note: 'asynchronous' here refers to queueing log events within your application, and not the asynchronous syslogd output file configuration option that other answers refer to.