stdio buffering considered harmful

[dan at bbncd]

From:  Dan Franklin <dan at bbncd>

UNIX developers were once justifiably proud of the fact that pipes were an
INTERACTIVE way of coupling programs; if the standard input and output of a
pipeline were the terminal, you could type a line at it and immediately see the
result.  But now that stdio is used everywhere, this feature has been "fixed":
a pipeline is usually NOT interactive.  I found this very annoying when I tried
to use m4 with adb, and ended up adding a flag to m4 to force non-buffering.
Bell must have encountered this problem too, since cat -u forces non-buffering.
But cat -u is clearly wrong (even though Rob Pike blesses it); the problem is
with stdio, not with individual commands, and we should not change all the UNIX
commands that might be used in an interactive pipeline to implement -u.  The
question is, what should be done?  I have just encountered this problem again
with a different program, and I would like to solve it once and for all.  One
solution would be to have stdio test an fd to see if it is a pipe (somehow),
and line-buffer stdout if it is, treating it like a terminal.  This could be
pretty expensive compared to writing in 1kbyte blocks, so it might be better to
leave the default the way it is and provide a way to tell stdio to line-buffer
on demand.  The obvious way to communicate with stdio, without changing every
command, is to use the environment.  Before writing, stdio would look in the
environment to see if there were any special buffering instructions.  Example:
        STDOUT=LINE_BUFFERED m4 | adb
Is this reasonable?  Is there a better way?

	Dan Franklin