Possible bug in MSC malloc (Large Memory Model)

[Wes Whitnah]

   There are bugs in MSC and/or DOS (verified VERBALLY to me by
   some MicroSoft Engineers a few years back -- see description
   below for details...) when heavily manipulating memory using
   the Large Memory Model -- by heavily manipulating memory I
   mean allocating and freeing memory VERY often.   Most memory
   bugs I have seen though are in the application itself.

   (Description of my personal experiences to follow... )
   

   I worked on an application which uses a LARGE amount of memory
   for windows, data structures, file information, etc. and of
   course uses the large model with MSC.  After many minutes of
   intense use (usually ~30 mins) the system would come up with
   either an "out-of-memory" or "memory-corrupted" error.  Either
   way after exiting the application a "memory-corrupted" message
   would be displayed and the system would halt.

   I double-checked all use and manipulation of memory, and set
   traces on the allocation and freeing of the memory.  Everything
   looked O.K. .  I suspected some fragmentation and made sure
   memory allocation and freeing were in the proper order.  No
   change in the results.

   After many days working through the problem it began to look like
   either a library or system problem (What! question MSC or DOS? :-)
   I created a test program which allocated random-sized chunks of
   memory up to a maximum limit and then freed them in reverse order,
   repeating this sequence a given number of times.  Using either
   malloc() or direct DOS calls produced the same results -- system
   memory corruption at the third pass (or less).  Before the fatal
   error condition memory was *VERY* fragmented.  Freeing blocks
   in the same order of allocation, or in random order did not
   significantly change the results.

   Now that I didn't trust MSC or DOS to correctly manipulate many
   blocks of memory correctly I wrote my own memory manager which
   called malloc() for LARGE blocks of memory and it managed the
   many requests for memory from this large-block pool.  I then
   ran my above test with this memory manager and have had no
   troubles to this day (after debugging the memory manager of
   course! :-).

   I confronted some MicroSoft engineers with my findings, and
   after a long discussion ("Did you check this?", "Yes";...)
   they said, "Oh, you found *THAT* memory problem!  Yes we know
   that DOS (and malloc() which uses DOS calls) has problems when
   memory is heavily used, but not many people come across it,
   so we don't intend to fix it!"  You might guess my reaction to
   that!  When I asked what developers and users should do when they
   come across this problem they told me, "They should write their own
   memory manager!".

   The above events transpired nearly two years ago, so I don't know
   if MicroSoft has really put much effort into solving these problems.
   From what I have seen of recent DOS versions and MSC 5.0 the old
   memory bugs are still there.  I've not come across them very often
   unless many blocks of memory are allocated and freed with FAR
   pointers, and the first noticeable problem is memory fragmentation.
   The best bet after *THOUROUGHLY* verifying your program's memory
   useage is to write your own memory manager and avoid confusing the
   DOS manager.

   Wes Whitnah
   wesw at ozvax.WV.TEK.COM