Author: David Mitchell
Date: 22:49:53 02/17/05
Go up one level in this thread
On February 17, 2005 at 14:03:30, Tord Romstad wrote: >A couple of days ago, a well-known programmer and regular >poster here on the CCC claimed that a good opening book >was worth at least 700 Elo points. I thought this number >looked completely outrageous, and decided to do a simple >experiment. > >I am the author of a basic and minimalistic UCI chess engine >called Glaurung. Source code and executables for Mac OS X, >Linux and Windows can be found at the following URL: > >http://www.math.uio.no/~romstad/glaurung/glaurung.html > >Recently, I have played some test matches with Glaurung >against the strongest engine I have on my compter: Hiarcs >9.6. Not surprisingly, all such matches end in crushing >victories for Hiarcs. The last match I played ended >75-25 in Hiarcs' favor. Your test is so flawed in design, with almost no control factors in place, that your conclusions are irrelevant, imo. Glaurung may lose 3:1 to Hiarcs for a huge number of reasons, or a mixture of several. It may lose with it's book, for quite different reasons than it loses when playing without it's book, to Hiarcs. In your post at least, I see no mention of establishing a valid relationship between these data points. We have to be so careful to ferret out the real reasons here. Superficially, it may appear that Glaurung running without it's book, is somehow, someway, equivalent to Hirarcs running without it's book. That is a horribly flawed leap of reasoning, imo. Congrats on Glaurung, however. Dave > >As a crude test of the "good book=700 Elo" claim, I have >now repeated the match with identical program versions >and conditions, except that Hiarcs was now playing without >an opening book. Assuming that Hiarcs' book is worth 700 >Elo, the expected result of this second match would be >something like 95-5 in _Glaurung's_ favor. > >The actual result of the second match was very close to >the first match: Hiarcs won by 72-28. > >As far as I can see, this means that at least one of the >following must be true: > >a) The statement "good book=700 Elo" is lightyears away >from the truth. > >b) Hiarcs has an extremely bad opening book, and with a >half decent opening book it would be several hundred >rating points ahead of Shredder. > > >Tord
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.