Author: Dieter Buerssner
Date: 18:38:58 07/31/01
Go up one level in this thread
On July 30, 2001 at 21:26:59, Robert Hyatt wrote: >On July 30, 2001 at 10:44:22, Frank Quisinsky wrote: > >>On July 30, 2001 at 09:31:28, Robert Hyatt wrote: >> >>>On July 29, 2001 at 10:50:47, Frank Quisinsky wrote: >>> >>>> >>>>Hi Bob, >>>> >>>>the same discuss then for 2 years here :-) >>>> >>>>I, the ex non ponderer, will say the following: >>>> >>>>01. >>>>You have right if you say that with ponder = off engines have time manangment >>>>problems, not all and not in all games but it's an important point. I know that >>>>this is for statistics not sooo important (ELO statistic). >>>> >>>>02. >>>>Match without ponder on single system with Athlon 1Ghz: >>>>Engine A = 1Ghz >>>>Engine B = 1Ghz >>>> >>>>Match with ponder on a single system with Athlon 1Ghz: >>>>Engine A = ~ 497-500Mhz >>>>Engine B = ~ 497-500Mhz >>>> >>>>Now we have 25-40% ponder hits if the engines play on the same level (after my >>>>experiments). >>>> >>>>Result: >>>>500Mhz + 25-40% ponder hits = ~700MHz. >>>> >>>>With ponder = on, the matches are running with 700Mhz on 1Ghz Athlon! >>>>With ponder = off, the matches are running with 1Ghz on 1Ghz Athlon! >>>> >>>>I believe better is to play with ponder = off on single system. OK, the time >>>>managment is a good point so say please with ponder but 300 MHz is a better >>>>point to say please without ponder. >>>> >>>>03. >>>>4-piece and 5-piece tablebases and engine-engine matches on a Dual system ... >>>> >>>>This is not very clear. >>>>We have engines witch used tablebases very aggressive or not very aggressive. >>>> >>>>Example: >>>>Gromit - Patzer with 5-pieces ... >>>> >>>>Gromit played move 50 in 2:45 with ponder = on and 5-pieces. >>>>Only 20-30% processor time in this game if Gromit used 5-pieces. >>>>Patzer played move 50 in 1.25 with ponder = on and 5-pieces. >>>> >>>>Now the same positions with 4-pieces ... >>>> >>>>Gromit play move 50 in 1.58! with ponder = on and *4-pieces*. >>>>Patzer played move 50 in 1.13 with ponder = on and *4-pieces*. >>>> >>>>Without 4-piece tablesbases are the differents not very important. >>>> >>>>In my opinion it is better to play eng-eng matches with ponder on a dual system >>>>with 4-piece tablebases. >>>> >>>>Best >>>>Frank >>> >>> >>>Your math is fine. >> >>Hi Bob, >> >>good, for two years you say my math is bad, but in this case you have right (I >>mean for two years) :-)) >> >>But your reasoning is wrong. Which would you rather do: >> >>>(1) play a match between two programs, using their strongest settings, and using >>>two 700mhz processors; >> >>>(2) play a match between two programs, using weaker settings on at least one, if >>>not both, using two 1ghz processors? >>> >>> >>>I vote for (1) because of using the strongest settings. You are voting for (2) >>>to avoid wasting compute cycles. I think (1) is more important... >> >>Yes, this is a good idea and a nice experiment. >>I have an Dual Pentium III 733 MHz and an Dual Pentium III 1GHz. >> >>But it is not 100%ig clear which test exactly! >> >>What do you think about the following match idea: >>Only with one CPUs to point 1-4. >> >>01. >>Crafty 18.10 on Dual Pentium III 733Mhz with ponder against Yace 0.99.50 on Dual >>Pentium III 1.05GHz (the machine is running with FSB = 140) without ponder, 40 >>moves in 40 minutes, 50 games. >> >>02. >>Yace 0.99.50 on Dual Pentium III 733Mhz with ponder against Crafty 18.10 on Dual >>Pentium III 1.05Ghz without ponder, 40 moves in 40 minutes, 50 games. >> >>03. >>The same match but Crafty 18.10 without ponder (1.05Ghz) and Yace 0.99.50 >>(1.05Ghz) without ponder, 50 games. >> >>04. >>After this matches, a new match Crafty 18.10 (1.05Ghz) with ponder - Yace >>0.99.50 (1.05 Ghz) with ponder, 50 games. >> >>In 2-3 months, after my CCE tourney I can make a test. >>If you have a better idea please write. >> >>I think 128 MB for hashtables, and 4pieces tablebases is good enough for the >>experiment. Tablebases with 4Mb cache. No lean options in configuration files. >> >>Best >>Frank > > >I don't think the experiment is valid... Because you don't know how much >crafty and yace are affected by ponder=off. It is possible that both do a bit >worse on time management. So that the matches match pretty well in score. But >does that mean ponder=off is then ok? Only for those two programs. You have to >repeat this for _every_ pair of engines you want to test like that... Until now, I tried to trim the citations in my messages. It takes some editing. Nobody seems to care. Why? I think, if everybody would do this, we all could read this forum much more effectively. If only a minority is doing this, it is just work for this minority. I give up ... Then - you have stated many times, that engine matches on one computer make not much sense. Other people have stated, that it may make sense, and discussed, if ponder on or off would be more sensible. So, who is right? You more or less say, that the "one computer may get sensible results" fraction is wrong. Frank suggests an experiment (on 2 computers). You, from the beginning call this experiment invalid. With the same right, the "one computer may get sensible results" fraction can say, that you should show evidence, that engine matches on one computer, with ponder off will show no reasonable results. So, they can ask you, to show an example, where it really makes a difference. How to solve this question? You allways seem to assume, that all chess engines are only developed with ponder on in mind. I can say, that this is not true. Yace started with ponder off, and later I implemented a rather unsophisticated ponder mode. Yace doesn't have the puzzling mode, that Crafty has. The time management was written, while Yace had no pondering. I just added very few lines of code (certainly less then 10), to make the engine use more time when pondering is on. So, perhaps, these two engines would be a good test for the question, how pondering affects engine strength. Especially, because our approach seems to be very different. I do not have 2 (reasonable) computers to play engine matches. Neither do I operate an account at some ICS myself. All my testing is with ponder off. You, on the other hand, do not care at all about ponder off (which I totally respect). So, this seems to be a well thought out example match. Unfortunately, the pages of Volker Pittlik are not available right now. He has already done many ponder on/off tests. Sure - they wer on one dual computer. My interpretation of his results were nevertheless, it really does not make a difference. And, as a last point. Many people seem to have fun running engine matches. Only a minor fraction of those will have two computers at hand. Why take them all the fun? When sombody sends me Crafty-Yace 1-0 on a single computer and a long time control, I try to look into the game. It is interesting information for me, as well as the result of a whole match (I am not too interested in games won by my program ...). With best regards, Dieter Bürßner
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.