I understand that a daemon should not write to stdout (and stderr) because that wouldn't be available once detached from the controlling terminal. But can I reopen stdout to a regular file, so that all my original logging will still work? This would be very nice and useful for me.
I tried something like this after forking,
freopen("/dev/null/", "r", stdin);
freopen("log", "w", stdout);
freopen("log", "w", stderr);
BOOST_LOG_TRIVIAL(info) << "daemonized!";
The daemon can be launched (to be precise, it doesn't fail and exit) and the log file can be created. But the log is empty (no "daemonized!"). Is this not the right way to daemonize? Could someone shed some light?
There is a library function, daemon(int nochdir, int noclose), that is available to help code appropriately daemonize and additionally reopen the standard I/O streams connected to /dev/null. Using it and a system log facility (like syslog) would be the way I'd go insofar as a "right" way to daemonize.
Having the standard I/O streams open and associated with /dev/null would provide the benefit of avoiding any hiccups due to any left-over I/O with these (that might for example block the process or cause an unexpected signal). It'd additionally prevent any new descriptors from unsuspectingly acquiring them and unwittingly getting output from say left over printf statements.
As far as associating the standard I/O streams with a regular file, the following warning in the online daemonize program man page seems useful to recognize:
Be careful where you redirect the output! The file system containing the open file cannot be unmounted as long as the file is open. For best results, make sure that this output file is on the same file system as the daemon's working directory.
Related
I was wondering if there are problems with not closing the file os.devnull in python. I am aware that normally we need to close files that we open. Nevertheless, I am wondering if it is possible to tread os.devnull as we treat sys.stdout or sys.stderr where we don't close them.
With normal files, you run the risk of losing data when you don't close them due to buffering. This is obviously not a concern for /dev/null.
However, while /dev/null is technically not a regular file but a device file, it uses file descriptors the same way. You can even inspect the file descriptor in Python using the fileno() method:
import os
with open(os.devnull) as devnull:
print(devnull.fileno())
Operating systems limit the number of open file descriptors any single process can have. This is very unlikely to be a problem, but it's still good practice to treat /dev/null like any other file and close it for this reason alone.
Actually, I am using libev; but under the hood this is using epoll (I'm only on linux). When I add a watcher to read a file and all data has been read then I do get a call back that there is data to read, but read(2) returns 0 (EOF). At that point I have to stop the watcher or else it will continue to tell me that there is something to read. However, if I stop the watcher and then some other process appends data to that file then I'll never see it.
What is the correct way to get notified that there is additional/appended data in a file that can be read when before I already read till the end?
I'd prefer the answer in terms of libev, but lower level will do too (I can then probably translate that to how to do that with libev).
It is very common, for some reason, for people to think that making an fd nonblocking, or calling poll/select/.. has different behaviour for files compared to other types of file descriptions, but nonblocking behaviour and I/O readyness behaviour is essentially the same for all of types of file descriptions: the kernel will immediately return from read/write etc. if the outcome is known, and will signal I/O readyness when this is the case. When a socket has an EOF condition, select will signal that the socket is ready to read, and you will get 0 (for EOF). The same happens for files - if you are at the end of a file, the kernel will return immediately from read and return 0 to signal EOF.
The important difference is that files can change contents at random places, and can be extended. Pipes and sockets are not random access and cannot be appended to once closed. Thus, while the behaviour is consistent, it is often not what is wanted, namely waiting for a file to change in some way.
The conflict in many people's minds is simply that they want to be told "when there is new data", but if you think about it a bit, you will realise that simply waking you up would not be an adequate interface for this, as you have no way of knowing why you woke up, and what changed.
POSIX doesn't have an interface to do that, other than regularly polling the fd or file (and in case of random changes, regularly reading the whole file!). Some operating systems have an interface to do something similar to that (kqueue on BSDs, inotify on GNU/Linux) , but they are usually not a perfect match, either (for example, inotify cannot watch an fd for changes, it will watch a path for changes).
The closest you can get with libev is to use an ev_stat watcher. It behaves as if you would stat() a path regularly, and invoke the watcher callback whenever the stat data changes. Portably, it does just that: it regularly calls stat, but on some operating systems (currently only inotify on GNU/Linux, as kqueue doesn't have correct semantics for this) it can use other mechanisms to speed this up in some cases, although it will fall back to regular stat polling everywhere, for example for when the file is on a network file system, where inotify can't see remote changes.
To answer your question: If you have a path, you can use an ev_stat watcher to watch for stat data changes, such as size/mtime etc. changes. Doing this right can be a bit tricky (see the libev documentation, especially the part about stat time resolution: http://pod.tst.eu/http://cvs.schmorp.de/libev/ev.pod#code_ev_stat_code_did_the_file_attri), and you have to keep in mind that this watches a path, not a file descriptor, so you might want to compare the device/inode of your file descriptor and the watched path regularly to see if you still have the correct file open.
This still doesn't tell you what part of the file has changed.
Alternatively, since you apparently only want to read appended data, you could opt to just read() the file regularly (in an ev_timer callback) and do away with all the complexity and hassles of an ev_stat watcher setup (while not forgetting to also compare the path stat data with your fd stat data to see if you still hasve the right file open, depending on whether the file your are reading might get renamed or replaced. Sometimes programs also truncate files, something you can also detect by seeing the size decrease between stat calls).
This is essentially what older tail -f implementations do, while newer ones might, for example, take hints (only) from inotify, just like ev_stat watchers do.
None of that is easy, and details depend on your knowledge of how exactly the file changes, but it's the best you can do.
I would like to monitor the STDERR channel of all the processes running on my Linux. Monitoring should preferably be done at real-time (i.e. while the process is running), but post-processing will also do. It should be done without requiring root permissions, and without breaking any security features.
I have done a good bit of searching, and found some utilities such as reptyr and screenify, and a few explanations on how to do this with gdb (for example here). However, all of these seem to be doing both too much and too little. Too much in the sense that they take full control of the process's stream handles (i.e. closing original one and opening a new one). Too little in the sense that they have serious limitations, such as the fact that require disabling security features, such as ptrace_scope.
Any advice would be highly appreciated!
Maybe this question would get more answers on SU. The only thing I could think of would be to monitor the files and devices already opened as STDERR. Of course, this would not work if STDERR is redirected to /dev/null.
You can get all the file descriptors for STDERR with:
ls -l /dev/proc/[0-9]*/fd/2
If you own the process, accessing its STDERR file descriptor or output file should be possible in the language of your choice without being root.
I would like to be able to get a list of all of the file descriptors (now considering this question to pertain to actual files) that a process ever opened during the runtime of the process. The problem with polling /proc/(PID)/fd/ is that you only get a snapshot in time of what is currently open. Is there a way to force linux to keep this information around long enough to log it for the entire run of the process?
First, notice that a file descriptor which is open-ed then close-d by the application is recycled by the kernel (a future open could give the same file descriptor). See open(2) and close(2) and read Advanced Linux Programming.
Then, consider using strace(1); you'll be able to log all the syscalls (or perhaps just open, socket, close, accept, ... that is the syscalls changing the file descriptor table). Of course strace is using the ptrace(2) syscall (which you probably don't want to bother using directly).
The simplest way would be to run strace -o /tmp/mytrace.tr yourprog argments... and to look, e.g. with some pager like less, into the quite big /tmp/mytrace.tr file.
As Gearoid Murphy commented you could restrict the output of strace using e.g. -e trace=file.
BTW, to debug Makefile-s this is the wrong approach. Learn more about remake.
Why is it necessary to close standard input/output/error when writing a (unix) daemon?
Not only stdin, stdout and stderr should be closed, but all open files.
From "Advanced Programming in the UNIX Environment", W. Richard Stevens, Addison-Weseley, 18th Printing, 1999, page 417.
Unneeded file descriptors should be closed. This prevents the daemon from holding open any descriptors that is may have inherited from its parent (which could be a shell or some other process).
Mr. Stevens proposal is to get the maximum file descriptor and close all files up to that value.
The cited chapter is about 'Daemon Processes'. Please note that closing file descriptors is only one point from five when writing daemons.
You don't have to definitely close standard output/error, but you have to be sure where your daemon (and child) are going to writing to. Redirecting the output to a log file is a common practice.
Closing standard input is required so that the daemon (and/or child) doesn't get stuck trying to read some input from the user.
But if you're going to detach you daemon from a controlling TTY using setsid(), the standard input/output/error are going to be invalid.