Ncurses shell escape drops parent process output - ncurses

Question

I'm using Ubuntu Linux 12.04 and writing a program that uses ncurses. My program has an option to execute subordinate processes (a "shell escape"). Before creating the subordinate process I do
reset_shell_mode( );
putp( exit_ca_mode ); // From <term.h>
Then when the subordinate process exits I restore my curses display with
putp( enter_ca_mode ); // From <term.h>
reset_prog_mode( );
refresh( );
This works fine. However, my program wants to also output some information just before launching the the subordinate process. It also wants to output some additional information when the subordinate process exits but before returning to a full curses display. Thus I have (abbreviated):
reset_shell_mode( );
putp( exit_ca_mode );
printf( "Don't forget... blah, blah\n" );
system( external_command );
printf( "Updating, etc\n" );
putp( enter_ca_mode );
reset_prog_mode( );
refresh( );
The problem is that the text produced by my program immediately before and after the call to system( ) does not appear. I guess maybe it's still going into some curses related buffer. I don't know.
How can I get the parent process to also output on the terminal as well as the child process?

Answer 1

Curses keeps its own buffer with an idea of what the screen should look like. When you call refresh(), it adjusts the screen to match that buffer, which means everything that curses doesn't know about will be overwritten (*).
printf, and the output of any external command, bypass that buffer, going directly to the screen (more exactly, to standard output, which happens to be connected to the screen, because they inherit their standard output from your shell).
So, to get your printf output into curses, you need to replace printf with printw. To get the output of the other program into curses, you have to capture its output into your program, then feed it to curses.
The easy way to do this is redirect the output to a file, then read the file:
system("ls > tempfile");
if ((fp=fopen("tempfile", "r"))!=NULL) {
while (fgets(buf, sizeof buf, fp))
printw("%s", buf);
fclose(fp);
}
WARNING: this example is stripped down a lot, to give you an idea. It doesn't catch errors well, it uses fgets which is prone to all sorts of buffer overflows, and it uses a constant name for the temporary file which causes a lot of concurrency problems.
A better way is to create a pipe between your process and the program you're trying to run:
int p[2];
pipe(p);
if (fork()==0) { // child process
close(1);
dup(p[1]);
close(p[1]);
close(p[0]);
execlp("ls", "ls", NULL);
} else { // parent process
close(p[1]);
if ((fp=fdopen(p[0], "r"))!=NULL) {
while (fgets(buf, sizeof buf, fp))
printw("%s", buf);
fclose(fp);
}
}
Again, this example is stripped down a lot (and i typed it directly into the browser, never compiled or ran it). To really understand it, and add all the missing error checking, learn about the linux/unix process model, pipes, file descriptors vs. C file pointers - there's lots of tutorials out there, and this is far beyond your original question.
But, to sum it up: if you want curses to put anything on the screen, you have to use the appropriate curses functions. Everything that bypasses curses might get overwritten as soon as curses refreshes the screen.
(*) If curses thinks there's only difference between the screen and the internal buffer, it will update only the different charactes, not the whole screen. So, if your external program writes to parts of the screen that curses thinks don't have to be updated, it will leave those parts alone, which means part of your program's output will remain.

Answer 2

In the example
reset_shell_mode( );
putp( exit_ca_mode );
printf( "Don't forget... blah, blah\n" );
system( external_command );
printf( "Updating, etc\n" );
putp( enter_ca_mode );
reset_prog_mode( );
refresh( );
The reset_shell_mode() call tries to restore the terminal settings. There is one problem. curses (generally speaking, not just ncurses) sets the terminal modes to "raw" (to allow your input characters to be read without interference by I/O buffering), but it also sets output-buffering (for performance).
It does this with some variant of setvbuf, which according to the standard cannot reliably be turned off/on:
The setvbuf() function may be used after the stream pointed to by stream is associated with an open file but before any other operation (other than an unsuccessful call to setvbuf()) is performed on the stream.
That's not just a fine detail; some implementations dump core if you try to discard buffering. So ncurses stays in line. But again there's something to note:
until late 2012 (to fix a problem with signals), ncurses used the same buffer for its output as the standard output (or whatever stream was fed into its initialization).
since then, ncurses uses a separate buffer. There are special cases such as putp which use the same output buffering, but printw uses a separate buffer which ncurses flushes during its repainting operations such as refresh.
In either case, the fix for this example would be to use fflush when using a different output stream than the preceding call. This should work:
reset_shell_mode( );
putp( exit_ca_mode );
printf( "Don't forget... blah, blah\n" );
fflush(stdout); // added
system( external_command );
printf( "Updating, etc\n" );
putp( enter_ca_mode );
fflush(stdout); // added
reset_prog_mode( );
refresh( );