How to use Sun RPC for large slow procedure calls

[Gregory N. Bond]

This is on Sun 3/60 and Sun 3/260, running SunOs 3.5, soon to be on a
Solbourne server, running SunOs 4.0.3 all round.

I have an application that occasionally requires lookups to a
database.  These lookups are often quite lengthy, taking up to a
minute or more.  As the database code is quite large, and a lot of
processing code is associated with it, I'd rather not include all this
code in every copy of the application.  (Not to mention that a
database backend for each copy of the application would murder the
server in no time.)

So I have used the Sun RPC compiler to create a database server task
that runs on the main machine, and an rpc client library to link into
the application.

Now this works quite well for the small fast calls, as they are
usually answered within the timeouts.  However, on the more complex
queries (and it is not easy to tell what's complex from the client as
a lot of complexity is hidden in the server), the RPC times out and
resends.  This banks up extra work for the server that isn't
necessary, doing one difficult job 5 times.  What is worse, the rpc
client routines often timeout totally, and return an error, even
though the server is happily chugging away on the request.

Also, if the server machine is busy, or the server process has been
swapped etc, or there is lock contention for the tables, then even the
short queries time out and fail, only to work when immediately
retried.  This is quite un-friendly for the (largely non-technical)
user base!

I have tried using the TCP transport, which one would have thought
meant that timeouts were not appropriate, but that doesn't seem to be
the case.  Setting very large timeouts is also not an option as I
would like quick indication that the server is unavailable.

It seems that Sun RPC is designed for many small and quick calls (e.g.
NFS), rather than large slow calls (as in my example, or for things
like numerical compute engines inverting large matricies).  This is
shown by the heritage of using UDP and timeouts rather than the flow
control and reliability in TCP.

Does anyone have any idea how to approach this sort of RPC
application?  Is there some trick or section of TFM that I have
overlooked?  Does anyone have (oh joy oh bliss) some code for this
type of RPC work?  Or do I drop Sun RPC entirely as the wrong tool and
handcraft something using "raw" TCP sockets?

Greg.  The network sort of computes!
--
Gregory Bond, Burdett Buckeridge & Young Ltd, Melbourne, Australia
Internet: gnb at melba.bby.oz.au    non-MX: gnb%melba.bby.oz at uunet.uu.net
Uucp: {uunet,pyramid,ubc-cs,ukc,mcvax,prlb2,nttlab...}!munnari!melba.bby.oz!gnb