Why does piping into BASH command groups sometimes work? - linux

Question

I have used the following command for a while to keep headers on ps output.
ps aux | { head -1; grep root; }
The output will look something like the following.
USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND
root 142 0.0 0.0 1234567 2520 ?? Ss 3:14AM 0:08.03 /usr/sbin/notifyd
root 55 0.0 0.0 7890123 2460 ?? Ss 3:14AM 0:01.94 /usr/sbin/syslogd
...
However, when used with other command line programs the output is not as expected.
Take the following df example.
df -h
Outputs the following.
Filesystem Size Used Avail Use% Mounted on
/dev/disk1s1 466G 103G 362G 22% /
/dev/disk1s4 466G 1.1G 362G 1% /blah/blah/blah
Using df in a similar syntax as the above example with ps.
df -h | { head -1; grep disk1; }
Outputs the following.
Filesystem Size Used Avail Use% Mounted on
The expectation is that the output would look essentially the same as the straight df -h command.
Why does this differ from ps?
I feel that knowing these differences will help me understand BASH processing more completely.
Thank you!

Answer 1

It's because head is buffering its input. It reads into a large buffer from the pipe, then starts extracting lines from that buffer. After it has read and printed the first N lines, it exits. Then grep starts reading from the pipe. But anything that head already read into its buffer is not available.
The reason it seems to work with ps is because it produces lots of output, which doesn't fit into this buffer. grep is then able to process the rest of the output. But I think if you check carefully you'll see that the result is incomplete.
The output of df is much smaller, it all fits into the buffer that head uses, so there's nothing left for grep to process.
The buffer size is probably something like 4K characters.
You can do what you want with awk:
df -h | awk 'NR == 1 || /disk1/'
ps aux | awk 'NR == 1 || /root/'
NR is the line number, so this prints the line if it's the first line or it matches the regexp.

Answer 2

Sed(1) can also be used to filter output in this case:
ps aux | sed -n '1p; /root/p'
-n: Don't echo line of input to standard output after all commands have been applied to it.
1p; "address" of line1, with 'p' to print pattern space
/root/p; "address" of /regexp/ matching "root", with 'p' to print pattern space
Alternative:
ps aux | sed '1p; /root/p; d;'
Some systems may require ps -aux i.t. dash (-) to prefix options. Linux and *BSD systems do not (cannot be sure how macOS behaves, I don't have such a system to check out).