How many administrators needed per site?

[Steve Romig]

>   Enough griping about the situation here.  How do *real* CS departments
>   stack up?  We've heard from UMd Eng (but not CS), Yale, Ohio State Physics
>   (but not CS).

We've got ~230 diskless SLCs, served by 21 Sun 3/180 file servers (and
a 4/280 and a 4/330); ~10 diskless HP somethings served by 1 HP file
server; 4 Pyramids; 1 Multimax; 1 Butterfly; 300+ Macs; and some odds
and ends.  There are roughly 9 different hardware/software platforms
that we currently support (sun3 running sunos 4.1, sun4 running sunos
4.1, etc).  This is all connected through 1 main ethernet (our
backbone), 24+ Ethernet subnets and a bunch of Appletalk stuff that I
don't want to know anything about.  

Our users: roughly 1700-1800.  45 faculty, 200 grad students, rest are
undergrads or guest acounts.  These facilities are for instruction and
research in the Computer and Info Sciences Department at OSU.  That
count doesn't include the students using the Macs in the low level
courses, which is probably another 1500 folks or so.

Our staff: We're split into 3 parts: software, hardware and
operations.

    Software staff deals with software development and systems
    installation, maintenance and bug tracking/fixing.  Consists of 8
    full time folks (6 Unix, 1 Mac, 1 Unix/parallel research support)
    and 8 part time folks (grads and undergrads).  We're all (but 1)
    general Unix folks, though we each tend to specialize in different
    areas (X, postscript, printers and text processing stuff,
    networks, mail, news, strange languages, ntp, nameservers, etc).

    Hardware staff deals with hardware install, maintenance, advice on
    upgrades and etc.  We do almost all of our Sun (and I think most
    of our Mac support) in house, at the board component level.  We
    also do most of our own peripheral integration (select, buy and
    install disks, tapes, etc).  The rest is through support contracts
    with the vendors.  Hardware consists of 2 full time folks and
    something like 6 part time folk.  Oh, they take care of the nets
    too.

    Operations staff deals with keeping things running: acount
    installation, maintenance, file system stuff (creating and
    maintaining user and project directories, backups, restores,
    handling common problems and emergencies) annnnd they are
    stationed in the labs when they are open to handle problems,
    answer questions, and keep people from walking away with or
    destroying machines.  1 full time person, 35 part time.

We don't do much in the way of course-ware development, but do do alot
of consulting type things with our various users.  The software staff
(especially) is expected to and is trying to do more development type
work (make new/better sysadmin tools, better user interface type
things, etc), though our main "purpose" is to keep things running and
reasonably up to date.

Lessons we've learned:

    Keep everything as much the same as possible.  All of our Sun
    clients are clones of a master copy, all of the servers are clones
    of a master server, all of the Pyramids look alike, etc.

    Reduce the number of platforms as much as possible.  We used to
    have something like 13 platforms, we're down to 9, and may soon be
    down to 7 if we lose the Pyramids...In my mind, though reducing
    the number of platforms is nice, you have to balance that against
    having a rich environment, which is also nice.

    Localize local changes to /usr/local (or some scheme like that) as
    much as possible, which makes upgrades easier.  Try to refrain
    from hacking on and reinstalling local versions of things in /bin,
    /usr/ucb, and so on.

    Diskless workstations are your friend, as long as you have enough
    memory on them.  It takes me about 2 hours to install a new copy
    of / on all 220+ diskless SLCs, including shutting them down,
    copying the stuff and bringing everything up again.

    You have to strike a balance between keeping things up to date and
    spending too much time keeping things up to date.  I try to settle
    on a SunOS release that seems reasonably stable and stay there for
    a long time, for example.  We were at SunOS 3.5.1 for a very long
    time.  We're at 4.1 now, and I'm still searching for a point of
    stability...:-)  

    Build tools to do things, rather than doing it "by hand" - if you
    do something once, you'll do it again.

    Beg, borrow and steal (only kidding) software from others when you
    can. 

(Mark came up with the last two suggestions...)

--- Steve