Ross M. Greenberg
greenber at utoday.UUCP
Thu Sep 7 10:52:01 AEST 1989
In article <357 at simon.UUCP> steve at simon.UUCP (Steven E. Piette) writes:
> We all make mistakes from time to time, but too many
>hide from the heat when that time comes.
>Thanks Ross for being honest with us.
I figure it's easier to be honest than to try to coverup a mistake.
I must admit that the problems the MPE review caused have already changed
things here. The review coming out in the next issue is on the ATT 6386E
unit. The benchmarks we *were* going to run were going to be be
Whetstone/Dhrystone/IOstone on it, with 1 to five processes running.
Without a comparison, they'd be almost useless, right? So, we went to
do a comparative benchmark. The only machine we had available was
a Compaq/386/16MHz running XENIX. It would have been obvious to anyone
reading the review and seeing the benchmarks that we were obviously
coparing apples and oranges (the 6386E is a 20Megger), but due to the
postings in *this* group, I've opted to not include the benchmarks as
stated. Instead, when we can more reliably cross-check the hardware and
software (6386E & SVR3.2 v 6386E & XENIX v COMPAQ 386/20 & SVR3.2 v
COMPAQ 386/20 & XENIX), then I'll include the benchmark data.
But, that brings up some interesting points.
As I've indicated, we're going to be doing some massive work in the
benchmarking area shortly. Part of my responsibility is to figure out
what those benchmarks really need, and how to compare apples and oranges.
In the DOS world, it's easy. In the UNIX world, not so easy. So. I
figure I'll ask the same guys that gave me such a hard time (thanks for
volunteering! :-) ): what do *you* think is important in a UNIX
benchmark? How do I compare Machine A (with hardware and software
configuration of 'A') to Machine B with its H/W configuration of 'B'?
Doe Whetstone/Dhrystones/IOStones cover enough? Application testing
is important, obviously, but should it be *real* applications (limiting
the number of machines we can test) or psuedo-apps (take a 100,000 x 1024
byte file and sort it a coupla hundred times while adding up a given
field)? Is it important to test with one or more processes doing the
same thing? How about with one or more *users* doing the same thing?
Should the results be output in tabular format, graphical format, both?
Should we list all tested machines in each issue (doubtful: each editorial
page is precious)? Should a given machine's configured price be used
to figure out which other machines to compare its benchmark to? And,
which price? The list price or the quantity 100 price?
Setting up a benchmarking suite is going to be an ongoing process for
us. I'd like to ask your help in helping us to figure oput what to do.
No promises, though: if we can't afford it in money or in time, we
can't implement it. But, I *know* that I'm not a benchmarking expert! :-)
I'd be happy to collect mailgrams with people's ideas, but an open discussion
might prove a bit more interesting to all.
One thing: flames regarding our *current* testing (or even compliments)
should be sent to me at the below address. Until we get a biz.group set
up, let me not use the net to tell people what a great magazine we are? :-)
Anyway...just to lay one thing to rest. Space permitting, we'll
be printing the MPE benchmarks versus a *current* version of Interactive
as soon as possible. And the cartoon on our /etc page will probably show me
wiping a bunch of egg off my face.
>Now if I only could figure out what happened to the subscription I had...
>(only two copies ever showed up here from the beginning of Unix Today!)
Sigh. Considering that our circ department is just now geting on-line,
bear with us. We're getting there. Chances are you lost out when we
requalified. Send mail to the below address and qualify again?
Ross M. Greenberg, Review Editor, UNIX Today! greenber at utoday.UUCP
594 Third Avenue, New York, New York, 10016
Voice:(212)-889-6431 BIX: greenber MCI: greenber CIS: 72461,3212
To subscribe, send mail to circ at utoday.UUCP with "Subject: Request"
More information about the Comp.unix.i386