Experimental Results for the Remaining Positions


 
Table 1.11: Results of DarkThought for the 130 Remaining Positions.
Search Best    Fresh    (I - 2)   (I - 3)  
Depth Change (#) Best  (#) Best  (#) Best  (#)
2 38.46% (50) 100.00% (50)    0.00% (0)   0.00% (0)
3 30.00% (39) 87.18% (34) 12.82% (5) 0.00% (0)
4 29.23% (38) 57.89% (22) 26.32% (10) 15.79% (6)
5 23.85% (31) 70.97% (22) 12.90% (4) 12.90% (4)
6 26.92% (35) 62.86% (22) 14.29% (5) 5.71% (2)
7 22.31% (29) 62.07% (18) 27.59% (8) 3.45% (1)
8 21.54% (28) 35.71% (10) 32.14% (9) 7.14% (2)
9 16.92% (22) 54.55% (12) 18.18% (4) 13.64% (3)
10 17.69% (23) 47.83% (11) 13.04% (3) 8.70% (2)
11 15.38% (20) 50.00% (10) 20.00% (4) 10.00% (2)
12 16.15% (21) 42.86% (9) 19.05% (4) 4.76% (1)
13 11.54% (15) 46.67% (7) 20.00% (3) 13.33% (2)
14 13.08% (17) 41.18% (7) 11.76% (2) 17.65% (3)
 


 
Table 1.12: Results of Crafty for the 130 Remaining Positions.
Search Best    Fresh    (I - 2)   (I - 3)  
Depth Change (#) Best  (#) Best  (#) Best  (#)
2 44.62% (58) 100.00% (58)    0.00% (0)   0.00% (0)
3 30.77% (40) 75.00% (30) 25.00% (10) 0.00% (0)
4 26.92% (35) 82.86% (29) 8.57% (3) 8.57% (3)
5 23.85% (31) 67.74% (21) 25.81% (8) 3.23% (1)
6 23.85% (31) 51.61% (16) 29.03% (9) 3.23% (1)
7 22.31% (29) 51.72% (15) 24.14% (7) 3.45% (1)
8 20.77% (27) 40.74% (11) 22.22% (6) 14.81% (4)
9 13.08% (17) 47.06% (8) 23.53% (4) 5.88% (1)
10 11.54% (15) 40.00% (6) 33.33% (5) 0.00% (0)
11 12.31% (16) 62.50% (10) 18.75% (3) 0.00% (0)
12 12.31% (16) 43.75% (7) 12.50% (2) 22.22% (2)
13 13.85% (18) 33.33% (6) 16.67% (3) 12.50% (2)
14 13.85% (18) 38.89% (7) 27.78% (5) 5.56% (1)
 

When removing the 106 opening positions and the 107 middlegame positions from the full set of 343 corrected test positions, you end up with a subset of 130 remaining positions that occurred at the 35th, 42nd, ..., and 70th moves of real chess games. This final subset contains mostly endgame positions complemented by a few ones from late middlegames. Table 1.11 presents the experimental results of DARKTHOUGHT for these 130 remaining positions alone. Table 1.12 lists the corresponding numbers of CRAFTY as automatically computed by our Perl script from Hyatt and Newborn's publicly available result file. The numbers exhibit the following unusual yet clearly sporadic fluctuations.

We attribute these fluctuations to the reduced size of the test subset as in the cases of the opening and the middlegame positions (see Section 1.5.6 and Section 1.5.6). Up to iteration #8 inclusively the results for the remaining positions alone closely resemble those for the whole test set which Section 1.5.6 and Section 1.5.6 already discussed in great detail. Starting with iteration #9, however, the ``Best Change'' rates of CRAFTY and DARKTHOUGHT for the remaining subset dropped markedly below that of all other positions and stayed lower on average until iteration #14. Given the strong empirical evidence from the experimental results of both CRAFTY and DARKTHOUGHT, we expect this more stable behaviour at high search depths in late middlegames and endgames to hold for modern chess programs in general.

Thanks to our additionally recorded data we can actually explain the drops of the ``Best Change'' rates from iteration #9 onwards. In comparison with their preceding siblings for the opening and the middlegame positions Table 1.11 and Table 1.12 reveal that the numbers of ``(I - 2) Best'' moves for the subset of remaining positions differ substantially from those of the other subsets. Starting with iteration #9, both CRAFTY and DARKTHOUGHT produced only three ``(I - 2) Best'' moves per iteration on average for the remaining subset (50%-100% less than usual) while the overall numbers of ``Fresh Best'' moves and ``(I - 3) Best'' moves remained roughly the same as for the other subsets.1.10 Therefore, the notable drops of the ``Best Change'' rates spring almost entirely from the smaller number of ``(I - 2) Best'' moves. This means that the odd/even instability of both CRAFTY and DARKTHOUGHT decreased with less material on the chess board.



Created by Ernst A. Heinz, Thu Dec 16 23:28:11 EST 1999