2.11.19 w/dbz not rejecting articles

[Doug Blair]

A few weeks ago I described a condition in which news 2.11.19 with the dbz
code included did not properly reject duplicate articles received from
a second news site.  Thanks to all of you who replied and added your
thoughtful insights. Summary follows:

The most helpful comments came from karl at ficc.ferranti.com (Karl Lehenbauer):

I have determined the cause of this problem.  I had to learn way more about the
internals of news than I ever wanted to, but the problem is the optimizer of the
C compiler.  I forget which news source file it is, but when used with dbz
(System V/386 3.0 and 3.2) it screws up a calling argument and causes trashed
article ID's to be written into the dbz database.  Then, whenever the same
duplicate article is presented, it isn't found in the database, so news treats
it as a new article.

The lazy fix, at least for a test, is to compile news without optimization.

To which Jon Zeeff, dbz author, added:

Or you could avoid the optimizer problem by compiling with gcc.

I don't have gcc, but I did recompile without the -O flag and it seems to
be working just fine these days!

Several (noteable Conor Cahill) recommended cnews.  

sneaky.lonestar.org!gordon (Gordon Burditt) also sent:

There is a define in the dbz routines which controls the size of the
database file.  It's supposed to be prime and several times the
number of articles in the history file.  I had problems with duplicate
articles being accepted, and discovered that this number was about
half of the number of articles in the history file - so half of the
articles would be "not there" for the purpose of duplicate checking.
I would expect dbz performance to start getting real terrible around
the time file size <= 1.05 * # of articles in history file.

Whether you have this problem depends on what groups you receive, and
how long you keep history, but I expect my site to be a bit low on
both of these compared with the bigger sites.  

Raise the number in the dbz source file.  Recompile expire, inews,
and anything else that uses it (rn?).  REBUILD THE DATABASE using
expire -R after installing these, and before letting any articles
arrive.

And m2c.m2c.org!jjmhome!jjm (Jim Murray) sent:

You said that you put dbmclose
in both expire and inews.  You only need dbmclose in expire if you did
things correctly.  You should only link the copy of dbz compiled with INCORE
for expire.  The other programs should be linked with a version of dbz
compiled without the INCORE option.

Also you should be using at least version 1.9 of dbz.  This is the earliest
version that I believe had a working version of INCORE.

Again, thanks to all who replied.  Wait til I tell you about my next problem!!

--
Doug
 ___  _           _  _             _    
|   || |_  ___  _| ||_| ___  __  _| |_  Doug Blair    Obedient Software Corp.
| | ||  .\/ ._\/.  || |/ ._\|  \|_   _| 1007 Naperville Rd, Wheaton IL  60187 
|___||___/\___/\___||_|\___/|_|_| |_|   708-653-5527  blair at obdient.chi.il.us