Are 3B1 "pipes" really slower than molasses?

[Thad P Floryan]

Hmmm, this topic is generating a lot of interest; I have received numerous
'phone calls today about it confirming my suspicions, and a few "Hey, Thad,
doncha know about read(0,..) and write(1,..)."

As I stated, I did research this quite carefully.  I have 3 versions of EACH
of those 3 programs (send, pass, recv) and the ones I posted ARE the fastest.

Yep.  Look at the macros for getchar() and putchar() in /usr/include/stdio.h
on YOUR system ... not bad code at all.  The 2nd version of my test suite used
{ read(0, &buf,1); and (write(1, &buf, 1) }, and the 3rd version of the suite
used buffered read(0,,) & write(1,,) along the lines of the getchar() example
in K&R.  The version I posted produced the best times.

I've also read the relevant chapters in both Bach's "The Design of the UNIX
Operating System" and Leffler's, et al "The Design and Implementation of the
4.3BSD Operating System" with no new insight on the matter.

What's interesting to me is that the 156KBytes/S on the 25MHz 68030 is approx.
5 times the 35Kbytes/S on the 10MHz 68010, leading me to speculate pipe
performance is perhaps limited by the CPU (considering the FIFO ring-buffer
implemention via its inode by the kernel).  Dunno; without source I'm guessing.

The 3B1's 35Kbytes/S pipe throughput and tape-change/-retension time also
coincides perfectly with the observed time required to back up the system to
tape, leading me to again conclude that pipes (and NOT disk or tape drive
speeds) are the bottleneck.

And I'm not using the word "pipe" loosely.  If you examine the tape scripts,
you'll note that filenames are piped to tapecpio which then pipes to dbuf
which then "standard outputs" to the actual device.

And, to add more fuel to the fire, I just 45 minutes ago received yet another
call from a person who wishes to remain anonymous (to the net) stating that
his company's booth at a trade show was visited by a passerby who "played"
for awhile with their system in the booth and uttered: "Oh, your system has
the AT&T pipe bug, too!"

NOW: AT&T has never (to my knowledge) acknowledged any pipe bug(s), but that
person's company DID make some kernel changes after the show which (allegedly)
boosted pipe performance 10 times.  Further inquiry on my part revealed the
SCCS modules have long since been archived (to tape, natch! :-), so finding
the specific fix(es) is simply not feasible since that company is presently
doing an SVR4 port and has no interest in looking at 5-year old code for
their "ancient" SVR2 and early SVR3 releases (for which the alleged fix was
implemented by them).

Sigh; without source, I have no hope of resolving this matter concerning pipes.
Everyone else is confirming my posted stats.  And various ktune changes aren't
changing the stats one way or another.  And I'm weary of rebooting to test.

And the fact there's no actual "bug" (i.e. the code works, albeit slowly)
obviates any hope of a third fix-disk from AT&T.

So, on to checking out tar's performance, and checking out the stuff I snarfed
from simtel20.  A quick glance indicates Fred Fish's BRU program may be the
best hope for speedy (and "maybe" streaming) tape backups.

Any other ideas and suggestions are still welcome.

Thad Floryan [ thad at cup.portal.com (OR) ..!sun!portal!cup.portal.com!thad ]