Paging-space problems

[Mark D. Rogers/100000]

Regarding Dennis Ferguson's request for an explanation
of how AIX paging space allocation works, here goes:

- paging disk slots are allocated on first reference to a page
  (early disk allocation, similar to the BSD-style Dennis described,
   although we do not implement quotas).

- malloc() (actually brk()/sbrk()) Do permit over-allocation of 
  paging space. SIGDANGER was quite correctly interpreted by Mr. Ferguson
  in a prior posting as being something that allows a process to
  `gracefully' exit. I forget what the algorithm for determining which
   process to kill, is, but you understand the mechanism correctly.

SIGDANGER was actually invented for the IBM RT, and migrated to
the RISC System/6000. The RT Virtual Memory Subsystem did Late Paging
Space Allocation, and allowed malloc() to over-allocate, exactly like
the System V description.

For the RISC System/6000, a completely new file system which has directory
journalling and is tightly coupled with the paging supervisor was written.
The pager does most of the file i/o via internally mapped files.
Early allocation was done on the new pager, essentially, for two reasons:

- to allow for potential future accounting mechanisms to be implemented
  which take advantage of the fact that virtual memory is always backed
  one-to-one with a disk slot (what you guys want). I don't know what we
  are going to do in this area yet (if anything).

- the new journalled file system/paging subsystem is quite complex, 
  having many objects with many states in and among themselves, to manage.

Early allocation simplified the paging subsystem design, and allows for
potential to implement accounting of sorts. 
As to whether we will or will not do accounting/quotas, I really don't
know at the moment, as I am not the person tracking that.

With regard to the malloc() issue -

- you are quite right in saying that we have customers who
  want `sparse' virtual memory. That really is a big deal among
  certain applications. Being able to have a large memory object
  and only pay for what you really use can be a nice programming
  construct.

- We did provide a `safemalloc()' which goes and touches all the
  pages & checks for SIGDANGER. I thought we shipped that as a sample.

We have at Least two very distinct classes of customers where
paging space allocation is concerned:

1.  Those such as yourselves, who want your application to either
    run, or not, based upon how much backing storage you have.
    If your app. doesn't run, go buy another disk & add paging space.

2.  Customers who like to allocate all the virtual memory they can,
    knowing that it will never all be used. This, alleviates the
    need for any complicated run-time memory management schemes in
    their model. It is a very convenient programming construct.

>From a historical perspective (for your information) we had
a number of `type 2' customers Early on on the RT, and
that influenced the System V-like behaviour of the RT somewhat.

It also has something to do with why we, on the RISC System/6000
still want to allow large virtual memory objects efficiently.

Basically what we have on the RISC System/6000, is a hybrid attempt
at allowing both types of customers. SIGDANGER is a compromise attempt
to allow them to co-exist. Admittedly, it is not perfect, however,
no matter what route one goes in, on this issue, we have found
thorns in the path. We are continuing to investigate the entire
issue, and welcome any comments.

Mark D. Rogers
AIX Operating System Architecture
Austin, Texas