Experimental Results for the Opening Positions
Table 1.7:
Results of DarkThought for the 106 Opening Positions.
| Search |
Best |
|
Fresh |
|
(I - 2) |
|
(I - 3) |
|
| Depth |
Change |
(#) |
Best |
(#) |
Best |
(#) |
Best |
(#) |
| 2 |
37.74% |
(40) |
100.00% |
(40) |
0.00% |
(0) |
0.00% |
(0) |
| 3 |
50.94% |
(54) |
83.33% |
(45) |
16.67% |
(9) |
0.00% |
(0) |
| 4 |
39.62% |
(42) |
54.76% |
(23) |
35.71% |
(15) |
9.52% |
(4) |
| 5 |
33.96% |
(36) |
44.44% |
(16) |
38.89% |
(14) |
0.00% |
(0) |
| 6 |
25.47% |
(27) |
55.56% |
(15) |
25.93% |
(7) |
3.70% |
(1) |
| 7 |
16.98% |
(18) |
44.44% |
(8) |
27.78% |
(5) |
22.22% |
(4) |
| 8 |
25.47% |
(27) |
59.26% |
(16) |
11.11% |
(3) |
7.41% |
(2) |
| 9 |
25.47% |
(27) |
37.04% |
(10) |
29.63% |
(8) |
11.11% |
(3) |
| 10 |
29.25% |
(31) |
25.81% |
(8) |
41.94% |
(13) |
9.68% |
(3) |
| 11 |
18.87% |
(20) |
15.00% |
(3) |
55.00% |
(11) |
15.00% |
(3) |
| 12 |
17.93% |
(19) |
42.11% |
(8) |
26.32% |
(5) |
15.79% |
(3) |
| 13 |
16.04% |
(17) |
41.18% |
(7) |
35.29% |
(6) |
5.88% |
(1) |
| 14 |
13.21% |
(14) |
35.71% |
(5) |
14.29% |
(2) |
14.29% |
(2) |
|
Table 1.8:
Results of Crafty for the 106 Opening Positions.
| Search |
Best |
|
Fresh |
|
(I - 2) |
|
(I - 3) |
|
| Depth |
Change |
(#) |
Best |
(#) |
Best |
(#) |
Best |
(#) |
| 2 |
34.91% |
(37) |
100.00% |
(37) |
0.00% |
(0) |
0.00% |
(0) |
| 3 |
49.06% |
(52) |
65.38% |
(34) |
34.62% |
(18) |
0.00% |
(0) |
| 4 |
32.08% |
(34) |
55.88% |
(19) |
38.24% |
(13) |
5.88% |
(2) |
| 5 |
33.02% |
(35) |
60.00% |
(21) |
31.43% |
(11) |
8.57% |
(3) |
| 6 |
31.13% |
(33) |
60.61% |
(20) |
15.15% |
(5) |
12.12% |
(4) |
| 7 |
30.19% |
(32) |
43.75% |
(14) |
34.38% |
(11) |
9.38% |
(3) |
| 8 |
25.47% |
(27) |
29.63% |
(8) |
44.44% |
(12) |
11.11% |
(3) |
| 9 |
22.64% |
(24) |
12.50% |
(3) |
41.67% |
(10) |
4.17% |
(1) |
| 10 |
19.81% |
(21) |
52.38% |
(11) |
28.57% |
(6) |
4.76% |
(1) |
| 11 |
19.81% |
(21) |
38.10% |
(8) |
23.81% |
(5) |
14.29% |
(3) |
| 12 |
22.64% |
(24) |
45.83% |
(11) |
33.33% |
(8) |
0.00% |
(0) |
| 13 |
16.98% |
(18) |
27.78% |
(5) |
22.22% |
(4) |
22.22% |
(4) |
| 14 |
19.81% |
(21) |
33.33% |
(7) |
38.10% |
(8) |
4.76% |
(1) |
|
The 343 corrected test positions contain a subset of 106 opening
positions that occurred at the 7th move (51 positions) and the 14th move
(55 positions) of real chess games. Table 1.7 presents
the experimental results of DARKTHOUGHT for these 106 opening
positions alone. Table 1.8 lists the corresponding
numbers of CRAFTY as automatically computed by our Perl script from
Hyatt and Newborn's publicly available result file. The numbers
exhibit the following unusual yet clearly sporadic fluctuations.
- DARKTHOUGHT - ``Best Change'' at iteration #7, ``Fresh
Best'' at iteration #11, ``(I - 2) Best'' at iteration #8, and
``(I - 2) Best'' iteration #11 and iteration #14.
- CRAFTY - ``Fresh Best'' at iter. #9 and ``(I - 2)
Best'' at iteration #6.
We attribute these fluctuations to the reduced size of the test subset.
Other than that the results for the opening positions alone closely
resemble those for the whole test set which Section 1.5.6 and
Section 1.5.6 already discussed in great detail. The analyses
of the overall results mostly apply to the results for the opening
subset as well. Only the ``Best Change'' rates of CRAFTY for the
opening positions do not really fit into the general picture because
they stayed around 20% up to iteration #14 inclusively. In contrast to
the overall results, DARKTHOUGHT executed much stabler searches in
the opening positions than CRAFTY from iteration #11 onwards. This
is probably due to a more aggressive evaluation of development and tempi
of CRAFTY in the opening phase of the game. The special behaviour of
DARKTHOUGHT that drops its ``Best Change'' rate to nearly 13% in
iteration #14 is also visible for the opening subset alone.
Created by Ernst A. Heinz, Thu Dec 16 23:28:11 EST 1999