Paging-space problems

[Dennis Ferguson]

In article <MCCALPIN.90Nov15150308 at pereland.cms.udel.edu> mccalpin at perelandra.cms.udel.edu (John D. McCalpin) writes:
>>On 15 Nov 90 18:09:19 GMT,dennis at gpu.utcs.utoronto.ca (Dennis Ferguson) said:
>Dennis> run System V systems with no page space at all if need be, and
>Dennis> that the total in-use memory allowed is related to (physical
>Dennis> memory + page space) rather than just page space. 
>
>Dennis> I think AIX exhibits the latter behaviour exactly.  

>This does not mesh with my experience with AIX.  Under AIX 3.1 on my
>RS/6000, I find that I cannot run jobs for which there is not enough
>paging space available on the disk --- even though there is plenty of
>memory to contain the job.
[some interesting examples]
>So what do I mean by *active working set*?  Well, I'm not sure how the
>O/S figures it out, but the following program runs until the part of
>the array that is *actually used* gets too big for the currently
>available paging space:

John,

Figuring out what memory has been modified is fairly simple since the
memory management hardware keeps track of this.  Note that the only
memory which may need to go out to paging space is modified data.  Text
and unmodified, initialized data pages can be paged in from the binary, while
unmodified, uninitialized data pages need not exist at all until they
are touched.  BSD kernels don't worry so much about any of this (i.e. they
allocate page space in advance for all (potentially modifiable) data
pages), but I think System V kernels do.  If you allocate page space
on the fly you can do things like allocate huge chunks of memory which
you only use little bits of without having to have a huge, mostly
unused swap area to back it.  Of course, if you do this what you
lose are the nice "No memory" messages and NULL return values from
malloc() when you run out.

You are right that your examples do not match my memory of the
behaviour of System V kernels, in fact they seem to exhibit a combination
of some of the worst characteristics of both System V and BSD paging
strategies.  What would be interesting to know, however, is whether yo
were ignoring or catching the SIGwhatever which indicates low memory when
you were running these and, if you weren't, whether the behaviour you see
changes if you do ignore this.  If the latter is true what you are seeing may
be the result of an IBM value-added feature rather than a characteristic
of the underlying System V kernel.

Indeed, what would be really interesting is if someone who actually
knew what they were talking about would explain how AIX paging works
to both of us.  What ever they have done, the results are often
unpleasant and are not helped by the greedy memory consumption
of AIX and some of its utilities.

Dennis Ferguson
University of Toronto