xmallocing

[jim frost]

feighery at cyclone.mitre.org (P. D. Feighery) writes:
>The program simply xmalloced until the xmalloc failed.  Then I xfreed bits and
>pieces from various areas and tried to xmalloc the sum of the pieces I xfreed.
>Well, I had to xmalloc 230 Meg until the xmalloc failed.  What I think is
>happening is that the IBM is paging back and forth to the regions on our
>disk which haven't been allocated to a particular partition.  Is this true?

No.  Both xmalloc and malloc allow overcommit on the RS/6000.  This
means that you can allocate more space than you actually have swap
available for and the allocation will succeed (up to a full segment,
which is 256Mb, which is probably why it failed at 230Mb).  The pages
will not actually be allocated until they have been touched.  If you
touch enough pages to exceed available space, SIGDANGER will be sent
to all processes.  Processes are supposed to free whatever memory they
can when this happens.  If no one frees space, or not enough space is
freed, in a reasonable amount of time (I'm not sure what "reasonable"
is but it's not very long) then the kernel picks a process and kills
it with SIGKILL.  It appears that it kills the largest process that is
not catching SIGDANGER.  On my system that's usually either the linker
or the X server.  It's pretty annoying when it's the X server.

It's been argued as to whether or not it was a good idea to allow
overcommit.  In my opinion it would have been nice to supply a
different malloc interface that allowed overcommit (for those
applications which really want it) rather than breaking nearly all
current applications.  Alas it's done and we have to live with it.

If you retry your test but write a byte or a word every 1k (or some
other number less than or equal to the page size of the machine -- I
can't remember how big the pages are anymore) inside every allocated
area, I believe you will see that SIGDANGER happens long before you
hit the 256Mb mark (assuming that you don't have 200+Mb of swap -- but
few people do).  I'm not sure what will happen to the kernel if you're
allocating in kernel space but it'll be readily apparent to user
processes.

Hope this sheds light on things.

jim