Paging-space problems

[Dennis Ferguson]

In article <1990Nov14.223820.29154 at arnor.uucp> rhoover at cirrus.watson.ibm.com (Roger Hoover) writes:
>In article <MARC.90Nov14153807 at marc.watson.ibm.com>, marc at arnor.uucp writes:
>|> malloc fails when the request causes the heap to exceed the 
>|> ulimit for data.  It has nothing to do with paging space.
>|> 
> Well, this is not true under sunos.  For example, consider the
> following program (called big.c):
[...]

It has been a while since I understood this really well, but I think
what is being described is a System V versus BSD Unix variation.

Under BSD Unix (and very old System V derivatives?), sufficient space on
the paging device to back all active virtual memory is allocated when
the memory is allocated, no matter whether the paging space is ever used
or not.  The effect of this is that you run out of paging space only when
allocating new virtual memory, i.e. when exec()ing a new program (in
which case the shell probably emits a "No memory" message) or when growing
an existing process (in which case malloc() returns a NULL value).  If
a process is started successfully it will never be terminated due to
paging space exhaustion, though requests for more memory may be denied.
Other implications are that you can't run a BSD Unix system with no
paging space, and if the size of the paging space doesn't exceed the
size of your physical memory you won't be able to use all of the latter.

System V (or at least the release I was familiar with) doesn't do this.
Instead it allocates page space dynamically, when you need to page
something out.  Running processes have no page space allocation unless
they actually have pages out on the backing store.  The good effects
of this are that you can run System V systems with no page space at
all if need be, and that the total in-use memory allowed is related
to (physical memory + page space) rather than just page space.  The
problem is that System V doesn't know if page space is exhausted
when it allocates new memory, but rather finds this out only when
it needs to page something out.  To avoid deadlock, the process which
is being paged out is killed.

I think AIX exhibits the latter behaviour exactly.  Malloc() never
returns NULL because the kernel doesn't know page space is exhausted
at that point.  Sometime later, however, a process will die to pay
for this.  Note that the process which dies is hardly ever the
process which grew itself, since the latter process is obviously
active and needs its pages, but rather something that was recently
active but which is now idle, like your shell, the window system
or a daemon.  Something has to die when you get to this point, since
it isn't normally possible to free memory back to the system.  It
sounds like the SIGDANGER thing was added to AIX to give the (more
likely to be guilty) active process a chance to commit hara kiri
before an innocent dies.

To tell the truth, I too like the BSD behaviour a lot better (though
the implementation in a vanilla BSD Unix is old, grotty, and still
suspects all the world is a vax).  Having random processes die is
truly annoying.

Dennis Ferguson
University of Toronto