free form texual database

[Jim O'Connor]

In article <451 at marob.MASA.COM>, daveh at marob.MASA.COM (Dave Hammond) writes:
> In article <379 at fsc2086.FSC.COM> jim at fsc2086.FSC.COM (Jim O'Connor) writes:
  [ discussion of using the filesystem as the textual database ]
> 
> I went this route on a project a few years ago, and was sorry later that
> I did.  The advantage of data manipulation with standard tools was far
> overshadowed by the tremendously inefficient disk usage.  Because of the
> filesystem inode limit, we were bound to a maximum of ~16,000 inodes on
> a 30mb partition.  With the average database entry size under 1K, the
> partition was effectively "filled" at ~16Mb, or half capacity.

The number of inodes in a file system is configurable when you make the file
system.  Altos's utility to init new hard disks (and hence filesystems) ask
for the "number of bytes/inode" to use.  The default is 2048 (sounds like your
system) but can be set to 1024 if you want.  Or, with if you use "mkfs"
directly, you can just specify the number of inodes that you want.

You are right about inefficient capacity usage with small files, though, since
space is allocated in block units. If you expect many small "records", a lot
of space is "wasted" in those partially filled blocks.

Possible solution:
Have the program that "creates records" store "small" records together in a
single file, with some sort of index scheme to keep track of where each record
is.   When you need the record, extract it into a single file, process it, and
then perhaps put it back.  Using "ar" might be a candidate for this.

--jim
------------- 
James B. O'Connor			jim at FSC.COM
Filtration Sciences Corporation		615/821-4022 x. 651