Author: Vasik Rajlich
Date: 01:47:26 02/22/05
Go up one level in this thread
On February 20, 2005 at 16:48:48, Peter Berger wrote: >On February 18, 2005 at 05:30:25, Vasik Rajlich wrote: > >>On February 18, 2005 at 04:25:37, Sune Fischer wrote: >> >>> >>>>>These so called "killer books" are always used by very strong engines on top >>>>>hardware. >>>>>The killer books seem to work best against weaker engines on weaker hardware. >>>>>Now isn't that odd? :) >>>> >>>>Killer books are in most of the cases for rich people who can pay for book >>>>makers so it is not a surprise that they work against engines with weaker >>>>hardware that do not have time to prepare. >>> >>>That could be the reason, but the point is that it won't show a lot because >>>these engines are so strong already that they would have won even with a bad >>>book. >>> >>>It would be far more interesting to give Beowulf a 700 Elo book and then go >>>wipeout Shredder & co in the big tournaments. :) >>> >>>>I also do not beliebe in 700 elo but I certainly believe that they can improve >>>>the performance by 100-200 elo in a tournament when part of the opponents. >>>> >>>>If you have statistics of a lot of games that you get 70% against some program >>>>with line A and 50% against the same program with line B then choosing line A is >>>>important. >>>> >>>>If you have similar statistics for many programs then it may give you >>>>significant increase in performance in tournaments. >>> >>>I tend to think the effect is mostly psychological. >>> >>>I can see how comming out of book with a +0.6 score against an equal opponent >>>must feel like half a victory, and it's not hard for me to imagine that some >>>would call that totally winning :) >>> >>>>Even with no special preperation against specific opponent you may have >>>>statistics that your program score 60% with 1.e4 c6 2.d4 d5 3.f3 and only 50% >>>>against the same opponents with 1.e4 c6 2.d4 d5 3.e5 so of course opening >>>>preperation can help significantly in tournament. >>> >>>I believe such is thing is possible, but increasing your score from 50% to 60% >>>is still not much more than 50-100 Elo. >>> >>>-S. >> >>I quite agree here. >> >>The other thing to keep in mind is just what is involved in increasing the score >>from 50% to 60%. It means that every fifth game you flat out win in the opening. >>Or - every "2.5th" game (ie 40% of games), your position is so great that you'll >>score 75% from it. (Or would have equal chances against an engine 200 points >>above you.) >> >>Chess just doesn't work like this. >> >>Anyway I think this entire debate is psychological. Promoting the value of >>hand-crafted books is a way to promote the value of individual tournaments over >>long matches/rating lists - and from a sporting/spectating point of view it's >>hard to argue with that. Statistical significance is just not that exciting ... >> >>Vas > >I agree with your conclusion that this debate is mainly psychological, but not >with all your reasons. > >E.g. are the statistically significant matches really good predictors for actual >tournaments? I am often surprised how readily people accept the idea that more >games mean more valid results per se. If you look at SSDF results or long >matches done in other user tournaments, there should be a huge influence of the >book learners, to an extent that maybe they are sometimes more the thing tested >than the engines themselves. True - although I doubt that the difference between learning algorithms makes much of a difference. This should just be another peanut ... > >In other tournaments ( like Nunn matches) the books are eliminated altogether, >which again makes them something different. > >Concerning the great quality of the special (and secret) tournament books, the >situation is even worse btw. Except the book authors, who partly worked on them >for dozens of years, and thus are strongly involved emotionally, and >understandibly so, no one even knows them per definition. I doubt that there is >any special book that provides more than 50 points ELO compared to a reasonable >book ( tbd later), guesswork of course. But 50 ELOs is nothing to be sneezed at >anyway. I absolutely agree here. Once we start throwing around crazy numbers like 200 rating points (ie. flat-out losing every other game), 50 rating points seems like nothing - but it's important. How many engine releases do we need to get 50 points? > >To the psychological effect - if a game is won ( or lost ) directly out of book, >this makes a very strong impression of course. And it happens. Just not so often >. And of course one has to take into account the lost points too. Last summer I >had a look at a lot of computer games played at official events, with the book >authors battling it out. The battle ground has limitted to the Najdorf mainly, >sometimes to an extent that the engine doesn't even matter *that* much anymore. >Is this optimal? It invites a lot of randomness, and also means that engines are >often playing for three results. A recent example would be Falcon-Shredder IMHO, >but there is at least a dozen of important tournament games that were influenced >by book, mainly in battles between Kure, Noomen and Necchi books , but for the >strongest programs it is not at all clear to me if this is a good idea. > Yes, I also completely agree here. For example, the Shredder book likes to rip through theoretical lines, getting all sorts of lost positions in the process. It's not that the book is bad - it's just the nature of going deep in those positions. We can give Stefan&Sandro the benefit of the doubt and say that they're taking a conscious decision here. Perhaps they even simply run Shredder at home and take its results, so that at least the result is at least not worse than letting the engine play in a tournament. Interestingly, since hiring GM Lutz to do their preparation, the Hydra team has scaled back the depth of their book. (Look at the games in Ali Dhubai & Bilbao.) Maybe he's come to the conclusion that at some depth, his input isn't as good as Hydra itself. One thing is for sure - somebody in the Hydra camp decided that - it would be interesting to know who :) >What would be a reasonable book ? Mainly one that avoids all stuff that is >well-known to not work too well with computers, has common lines that can be >used by humans ( or other bookers) to compete successfully against computers, >and that takes strengths and weaknesses of the engine into account. Let's say >this is two weeks of work and will lead to measurable effects ( I remember >Andrew Williams did this with his engine and reported an improvement of 100 >points, but I am not sure if this was tested to significance.) There must be an >additional effect, when you continue work, check all the latest news and sources >and work on further improvements - I just think there is not as many points to >be found there as most people believe. But of course I can only guess, same as >everyone else. This includes the book authors themselves, because they fight a >moving target ( the opponent isn't forced to play into the preparations, nor >even likely to, unless it's *another* Najdorf ;) ) - so they can't test either. > The part I don't get here is the "takes the strengths and weaknesses of the engine into account". Maybe if some amateur engine has a bug, sure - although in this case - fix the bug. I just don't buy this stuff about choosing lines which suit the engine. Anyway - this too could be tested. For example, Junior has a reputation as being strong in dynamic positions. IMHO this is nonsense - Junior is just more aggressive in those positions, which is a completely different thing. Can we show that Junior really performs better than Shredder, for example in a match from a preselected set of dynamic positions? I highly doubt it. IMHO - an amateur engine shouldn't bother with book until let's say four years go by. At that point, some professional chess player should be hired who will start from an automatically generated book and spend some month or so making some adjustments, preferably in the range of moves 5-15. In addition to this, it may make sense to keep running the engine and "pre-computing" some results, which can be spot-checked as appropriate. This last step will especially help at fast time controls. Vas >Peter
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.