Raw disk I/O

[Frank Mayhar]

I recently replied to an article here, regarding raw disk I/O.  I
received the following
response:

(I wrote:)
}}We were looking into this as well.  After testing, we decided to
}}continue to use the block device, since it was at least three times
}}faster than the raw device.  I concluded then that the block device
}}was doing some disk access optimizations that the raw device wasn't
}}doing.  Also, using the block device has the added advantage that
}}any new optimizations in the driver would automatically be used
}}by the application.
}}Of course, we're using System V.  I think you'll probably see the same thing
}}under BSD, though.

}Were you using the raw device in an efficient way?  Or were you just trying
}to use it like the block device?
}
}I.e., did you rewrite your application to have a writer process that blocks
}for the raw write, allowing the write to proceed asyncronously?  If you just
}wrote a big block directly to disk in the main application, then you have
}no disk cache in operation, the application blocks, and of course the
}raw throughput looks lousy.
}
}All the companies that do databases under Unix use raw partitions,
}and one of the reasons is throughput is much higher.
}
}I suggest you read Rochkind, "Advanced Unix Programming", specifically
}p. 47ff, where he states
}
}"As you can see, the speed advantages of raw I/O are enormous."
}---
}A. Lester Buck     buck at siswat.lonestar.org  ...!texbell!moray!siswat!buck

I sent Mr. Buck the following reply, but never received any response.  Can
anyone else answer my questions?

(My reply:)
Well, we don't seem to have _Advanced_Unix_Programming_ around here.  I can
say that we weren't doing anything special, just blocking I/O.  We got about
40k-45k/s.  With block-mode I/O, we get from around 90 k/s to as much as
200 k/s or a little more, depending on the test parameters.  Are you saying
that we could exceed these figures using raw I/O?

One of the real problems is that we have to do this for multiple users.  It's
not easy to justify two processes per user, nor did I want to bottleneck
everything through a single I/O process.  I _could_ do that, but how
would I get requests back and forth?  Message queues?  (I envision a single
[or even multiple] I/O processes, communicating with clients via message
queues, possibly using an elevator algorithm, certainly doing buffering.)
I still fail to see how this is faster than letting the block-mode device
driver do the same thing, though.  Plus, the device driver knows about the
idiosyncracies of the hardware, and can take advantage of them, where my
I/O process would have to be written for the lowest common denominator.
--
Frank Mayhar  fmayhar at hermes.ladc.bull.com (..!{uunet,hacgate}!ladcgw!fmayhar)
              Bull HN Information Systems Inc.  Los Angeles Development Center
              5250 W. Century Blvd., LA, CA  90045    Phone:  (213) 216-6241