Tuning information for ISC 386/ix

[Andrew Tannenbaum]

In article <14663 at haddock.ima.isc.com> I responded to madd at std.com about
reconfiguring his kernel to clean up his dblock failure problems.  One
of my co-workers made some helpful comments on my posting that I'd
like to share.

I think my advice on which programs to run to change the params was
OK.  My own system was probably configured a little on the liberal
side, though.  I just kept doubling stuff when the max values for
dblocks got close to the alloc values, or when I got failures.  Since
non-zero numbers in the fail column (of a netstat -m listing) seemed to
me to coincide with strange hangs and crashes, I figured that as
superstitious practices go, this was a pretty good one.

I still don't know exactly which parts of the kernel use which size of
dblocks.  I am told by reliable sources, however, that failures on the
4096 byte (NBLK4096 - dblock class 8) blocks is no big deal, apparently
386/ix will happily use 2048 byte blocks instead or something, so 4096
failures are not a problem.  Perhaps the rest of my estimates are also
a bit liberal, but when I try to cut back on them, I get failures in
netstat -m, so these are things you can play with (by recompiling and
reinstalling new kernels, these guys are not dynamic, else we wouldn't
have to go through this stuff).

In the tuning arena, my colleague also mentioned kernel buffers and
RAMdisk.  Note that in the following paragraphs the defaults I talk
about are in 2.0.1, so they might be different on your system.

I see that there are files called /etc/conf/kconfig.d/param.*MB which
have something to do with configuration.  (I don't know how they get
used.)  These param files have stune kind of param numbers, but not in
the stune format.  Apparently, there is one file for 2MB, and second
for 4, 8, and 16MB.  Since configuration has to do with more than how
much memory you have, this is excusable.  The point is that you can
play with these params in stune.  Many of the params are nicely
described in the ops/sysadm guide.

Anyway, I have a 9meg machine where I jacked up the NBUF/NHBUFs from the
default 250/64 to 1000/512, and I ran my pathalias build under each.
(I had to fix the mtune file to allow NHBUF to get that big.  Note
that NHBUF should be a power of 2 according to the docs.  386/ix uses
NHBUF to find a prime for a hash table size.)

The timings for pathalias went from over 14 minutes to under 5 minutes
real time.  This was on a multi-user system both times, but the
difference is still quite significant:

(I just ran it again, and the real time was less than 4 minutes, but I
don't feel like changing all the numbers in the note.)

	250 bufs
	real    14m17.79s
	user    1m0.37s
	sys     1m17.48s

	1000 bufs
	real    4m20.88s
	user    0m59.98s
	sys     0m36.41s

If you want to check out your buffer stats, you can run a command like
sar -b 10 50 (system activity report on buffers, every 10 seconds, 50
iterations).  Check the sar(1) man page to figure out what the stats
mean.  Anyway, I found when I jacked up the buffers, I got a whole lot
more buffers processed per second, and it seems the system time on my
pathalias went down from 77 secs to 36 secs, which must been related to
buffer munging.  (Note that this was a pathalias, sort, and makedb on
16600 entries, I don't want you all thinking that the machine is slow.
The actual pathalias run time went from 112 secs real to 60 secs real,
and makedb went from 12 minutes real to less than 3.)

As for RAMdisk for /tmp, I have no experience with it, but if you have
memory locations that have never been referenced since you bought your
system, you might want to try sticking /tmp in a meg or two.  I am not
saying this is bad, I'm saying I have no experience with it.  Some
people swear by it, I don't want to argue the virtues of /tmp on
RAMdisk, but it's something for you tuning heads to play with.
Frankly I don't have any complaints with the speed of the 386/ix file
system, perhaps someone could post some comparative stats on RAMdisk
stuff.

In summary, you can goof with your kernel by playing with the tune
parameters, (and maybe by running with files like /tmp on RAMdisk) and
you can check them with sar and with the special purpose stat programs
like netstat, and by timing your favorite applications.  I hope this
proved helpful if not inspirational.  If anyone knows about any
386/ix bottlenecks that can be fixed by diddling a parameter, by all
means speak up!

	Andrew Tannenbaum   Interactive   Cambridge, MA   +1 617 661 7474