I have an embedded application that I want a simple-minded logger for.
The system starts from a script file, which in turn runs the application. There could be various reasons that the script fails to run the application, or the application itself could fail to start. To diagnose this remotely, I need to view the stdout from the script and the application.
I tried writing a tee-like logger that would repeat its stdin to stdout, and save the text in a FIFO for later retrieval via the network. Then I naively tried
./script | ./logger
I ended up with only the script stdout going to the logger, and the application stdout disappearing. I had similar results trying tee.
The system is running kernel 2.4.26, and busybox.
What is going on, and how can I accomplish my desired ends?
It turns out it was working exactly as I thought it should work, with one minor gotcha. stdout was being buffered, and without any fflush(stdout) commands, I never saw it. Had I been really patient, I would have suddenly seen a big gush of output when the stdout buffer filled up. A call to setlinebuf(3) fixed my problem.
Apparently, the application output doesn't end up on stdout...
The output is actually on stderr (which is usually also connected to the terminal)
./script.sh 2>&1 | ./logger
should then work
The application actively disconnects from stdin/stdout (e.g. by closing/reopening file descriptors 0,1(,2) or, using nohup, exec or similar utilities)
the script daemonizes (which also detaches from all standard streams)
Related
I'm writing a terminal logging program - think the script command but a bit more featureful. One of the differences is that, whereas script captures stdout, stdin and stderr as one big character stream, I would like to keep them separate and record them as such.
In order to do this, I use the standard approach of running a child shell connected to a pty, but instead of using a single pty with stdin, stdout and stderr all connected to it, I use two ptys - with stdin and stderr connected to one pty, and stdout on the other. This way, the master process can tell what is coming from stdout and what from stderr.
This has, so far, worked fine. However, I'm starting to run into a few issues. For example, when trying to set the number of columns, I get the following:
$stty cols 169
stty: stdout appears redirected, but stdin is the control descriptor
This seems to be a result of this piece of code, which seems to check whether stdout and stderr are both ttys, but complains if they are not the same.
My question, therefore, is this: am I violating any fundamental assumptions about how Posix processes behave by acting in this way? If not, any idea why I'm seeing errors such as this? If so, is there any way I can get around this and still manage to separate stdout and stderr nicely?
One idea I had about this is to use a process directly on the pty which then runs the target program, e.g.
(wrapper) -> pty -> (controller) -> script
The controller would be responsible for running the script and capturing the stdout and stderr separately, feeding them back to the wrapper, perhaps by some non-std fd, or alternatively, serialising the data before shipping it back, e.g. prefixing output from stderr with stderr: and stdout with stdout: - then in the wrapper deserialize this and feed it back upstream or whatever you want to do with it.
I have come across strange scenario where when I am trying to redirect stdout logs of perl script into a log file, all the logs are getting written at the end of execution when script completes instead of during execution of the script.
While running script when I do tail -f "filename", I could able to see log only when script has completed its execution not during execution of script.
My script details are given below:
/root/Application/download_mornings.pl >> "/var/log/file_manage/file_manage-$(date +\%Y-\%m-\%d).txt"
But when I run without redirecting log file, I can see logs on command prompt as when script progresses.
Let me know if you need any other details.
Thanks in advance for any light that you all might be able shed whats going on.
Santosh
Perl would buffer the output by default. You can say:
$| = 1;
(at the beginning of the script) to disable buffering. Quoting perldoc perlvar:
$|
If set to nonzero, forces a flush right away and after every write or
print on the currently selected output channel. Default is 0
(regardless of whether the channel is really buffered by the system or
not; $| tells you only whether you've asked Perl explicitly to flush
after each write). STDOUT will typically be line buffered if output is
to the terminal and block buffered otherwise. Setting this variable is
useful primarily when you are outputting to a pipe or socket, such as
when you are running a Perl program under rsh and want to see the
output as it's happening. This has no effect on input buffering. See
getc for that. See select on how to select the output channel. See
also IO::Handle.
You might also want to refer to Suffering from Buffering?.
I'm using expect to execute a bunch of commands in a remote machine. Then, i'm calling the expect script from a shell script.
I don't want the expect script to log to stdout the sent commands but i want it to log the output of the commands, so my shell script can do other things depending on that results.
log_user 0
Hides both the commands and the results, so it doesn't fit my needs. How can i tell expect to log the results?
Hmm... I'm not sure you can do that, since the reason for seeing the commands you send is because the remote device echoes them back to you. This is standard procedure, and is done so that a user sees what he or she types when interacting with the device.
What I'm trying to say is that both the device output to issued commands, and the echoed-back commands, are part of the spawned process's stdout, therefore I don't believe you can separate one from the other.
Now that I think of it, I think you can configure a terminal to not display echoed commands... but not sure how you would go about doing that with a spawned process that is not using an interactive terminal.
Let us know if you find a way, I'd be interested of knowing if there is one.
I am using an embedded system, which runs linux. When i run a compiled C program in the forground, it works correctly. However, when i add the '&' after the program call, to make it run as a job in the background, certain features do not work correctly. The main feature which stops working is the use of the 'read' function (unistd.h), used to read from a socket.
Does running a process in the backround reduce its permissions?
What else could cause this behaviour?
Edit:
The function uses the 'select' and 'read' function to read from a socket used for the reception of CANbus message frames. When the data is received, we analyse it and 'echo' a string into a .txt file, to act as a datalogger. When run in the foreground, the file is created and added to successfully, but when in the background, the file is not created/appended.
The only difference between running a process in foreground of background is the interaction with your terminal.
Typically when you background a process it's stdin gets disconnected (it no longer reads input from your keyboard) and you can no longer send keyboard-shortcut signals like Ctrl-C/Ctrl-D to the process.
Other then that nothing changes, no permissions or priorities are changed.
No, a process doesn't have its persmissons changed when going into background.
Internally whats happening is before the process's code starts getting executed, the file descriptors 0,1,2 (stdin,out,err) will be pointed to /dev/null instead of usual files.
Similarly if you use >/file/path the stdout descriptor will point to that particular file
You can verify this with
ls -l /proc/process_number/fd
I have some legacy scientific code running on a Rocks cluster, with SGE. I have an application-specific job submission script that generates qsub scripts (i.e. the script which Sun Grid Engine takes and runs).
Within the qsub script, my legacy app is called. This app sends it's output to STDOUT. SGE intercepts STDOUT and spools it into a file in the users home directory, so the user can see results build up in real-time. I want this behavior to be maintained, but at the same time, I want to transparently log all output in the background. I figured tee would be perfect to achieve this.
So I modified the job submission script to run the app and pipe STDOUT to tee, which saves STDOUT to a file that is copied to a central store once the job completes. The app is run and piped to tee as follows:
\$GMSCOMMAND | tee \$SCRATCHDIR/gamess_output.log
The problem is, ever since I've started piping the code to tee, the app has been dying with SIGTERMs, especially when I request several nodes. I tried using the -i (ignore interrupts) parameter with tee: it makes no difference.
Things work fine if I redirect the app output to a file then cat the file once the app is done, but then I can't allow users to view results buildup in real-time (which is an important requirement).
Any ideas about why this use of tee might be failing? Or alternatively, any ideas about how else I might achieve the desired functionality?
I don't know anything about why your particular case is failing, but one option might be to make $GMSCOMMAND do it's own logging. (Effectively put the tee inside the app). I guess this option depends on cost of changing the legacy app.
Failing that you could wrap the 'legacy app' with your own script/application to do the redirection/duplication.
If pipes are your problem perhaps you can get around this by using a 'while/read' loop with process substitution. Does this work for you?
while read line; do
echo "$line"
echo "$line" >> ${SCRATCHDIR}/gamess_output.log
done <(${GMSCOMMAND})