Experimental Results

Before the start of the experiment we decided to broaden our behavioural studies of deep searches with respect to Hyatt and Newborn's original setup. Because of these extended objectives we let DARKTHOUGHT record more detailed information than just the search depth at which the best move changed as compared with the previous iteration. While searching each of the 343 corrected test positions to a fixed depth of 14 plies, DARKTHOUGHT also determined whether the new best move after the current iteration was ``fresh'' in the sense that it never constituted the best move at the end of any former iteration. Moreover, DARKTHOUGHT tracked all the instances where the new best move after the current iteration was the same as the best move of two or three iterations ago. In these cases there are typically several good moves with nearly equal scores that compete for the pole position and cause frequent changes of the best moves in consecutive iterations. The well-known and wide-spread instability of searches at odd and even depths belongs to this behavioural category with the new best move after the current iteration being the same as the best move of two iterations ago.

In order to formalize the above explanations of what DARKTHOUGHT recorded, we let B(i) denote the best move after iteration i. The columns ``Best Change'', ``Fresh Best'', ``(I - 2) Best'', and ``(I - 3) Best'' of the forthcoming tables (Table 1.5 - Table 1.12) list the numbers of positions for which the following conditions held after the search completed iteration i.

We also derived these four relations for CRAFTY by applying a short Perl script to Hyatt and Newborn's publicly available result file (see Section 1.5.5). The script automatically computed the complete result data of CRAFTY as listed in the tables of this section. The data includes all the additional information for CRAFTY that equals the special recordings of DARKTHOUGHT. Therefore and because we eliminated the doubles from the original set of test positions, we could not reuse the experimental results as originally published by Hyatt and Newborn in 1997. Furthermore, Hyatt and Newborn's numbers differ from those that our Perl script produces for CRAFTY from Hyatt and Newborn's original result file which spans the full set of 347 test positions (see Appendix 1.5.11). Hence, we consistently refer to our own numbers throughout the whole section.

The experimental results as published by other researchers correspond exactly to our ``Best Change'' columns. Up to now, nobody except Junghanns et ever mentioned any further characteristics of the new best moves. In this respect we advance the state-of-the-art by studying some interesting relations between all new best moves (``Fresh Best'') and sequences of two or three successive new best moves (``I - 2 / I - 3 Best'') that occur during single searches of modern chess programs. The remainder of this section presents the collected data and discusses our interpretation of the raw numbers in detail. For a description of the exact experimental setup of DARKTHOUGHT please see Appendix 1.5.9.



 

Created by Ernst A. Heinz, Thu Dec 16 23:28:11 EST 1999