Experimental Results for the Middlegame Positions
Table 1.9:
Results of DarkThought for the 107 Middlegame Positions.
| Search |
Best |
|
Fresh |
|
(I - 2) |
|
(I - 3) |
|
| Depth |
Change |
(#) |
Best |
(#) |
Best |
(#) |
Best |
(#) |
| 2 |
28.97% |
(31) |
100.00% |
(31) |
0.00% |
(0) |
0.00% |
(0) |
| 3 |
40.19% |
(43) |
86.05% |
(37) |
13.95% |
(6) |
0.00% |
(0) |
| 4 |
27.10% |
(29) |
51.72% |
(15) |
31.03% |
(9) |
17.24% |
(5) |
| 5 |
31.78% |
(34) |
55.88% |
(19) |
20.59% |
(7) |
20.59% |
(7) |
| 6 |
20.56% |
(22) |
81.82% |
(18) |
18.18% |
(4) |
9.09% |
(2) |
| 7 |
24.30% |
(26) |
46.15% |
(12) |
30.77% |
(8) |
11.54% |
(3) |
| 8 |
28.97% |
(31) |
54.84% |
(17) |
29.03% |
(9) |
0.00% |
(0) |
| 9 |
23.36% |
(25) |
32.00% |
(8) |
36.00% |
(9) |
16.00% |
(4) |
| 10 |
27.10% |
(29) |
41.38% |
(12) |
44.83% |
(13) |
6.90% |
(2) |
| 11 |
18.69% |
(20) |
30.00% |
(6) |
35.00% |
(7) |
5.00% |
(1) |
| 12 |
12.15% |
(13) |
53.85% |
(7) |
15.38% |
(2) |
7.69% |
(1) |
| 13 |
23.36% |
(25) |
40.00% |
(10) |
28.00% |
(7) |
12.00% |
(3) |
| 14 |
14.95% |
(16) |
25.00% |
(4) |
50.00% |
(8) |
6.25% |
(1) |
|
Table 1.10:
Results of Crafty for the 107 Middlegame Positions.
| Search |
Best |
|
Fresh |
|
(I - 2) |
|
(I - 3) |
|
| Depth |
Change |
(#) |
Best |
(#) |
Best |
(#) |
Best |
(#) |
| 2 |
35.51% |
(38) |
100.00% |
(38) |
0.00% |
(0) |
0.00% |
(0) |
| 3 |
31.78% |
(34) |
76.47% |
(26) |
23.53% |
(8) |
0.00% |
(0) |
| 4 |
33.64% |
(36) |
58.33% |
(21) |
30.56% |
(11) |
11.11% |
(4) |
| 5 |
35.51% |
(38) |
52.63% |
(20) |
34.21% |
(13) |
7.89% |
(3) |
| 6 |
28.04% |
(30) |
56.67% |
(17) |
30.00% |
(9) |
10.00% |
(3) |
| 7 |
21.50% |
(23) |
47.83% |
(11) |
34.78% |
(8) |
8.70% |
(2) |
| 8 |
21.50% |
(23) |
43.48% |
(10) |
26.09% |
(6) |
4.35% |
(1) |
| 9 |
20.56% |
(22) |
36.36% |
(8) |
45.45% |
(10) |
4.55% |
(1) |
| 10 |
21.50% |
(23) |
30.43% |
(7) |
34.78% |
(8) |
8.70% |
(2) |
| 11 |
18.69% |
(20) |
60.00% |
(12) |
30.00% |
(6) |
0.00% |
(0) |
| 12 |
16.82% |
(18) |
33.33% |
(6) |
22.22% |
(4) |
22.22% |
(4) |
| 13 |
13.08% |
(14) |
35.71% |
(5) |
28.57% |
(4) |
7.14% |
(1) |
| 14 |
13.08% |
(14) |
50.00% |
(7) |
28.57% |
(4) |
7.14% |
(1) |
|
The 343 corrected test positions contain a subset of 107 middlegame
positions that occurred at the 21st move (55 positions) and the 28th
move (52 positions) of real chess games. Table 1.7
presents the experimental results of DARKTHOUGHT for these 107
middlegame positions alone. Table 1.8 lists the
corresponding numbers of CRAFTY as automatically computed by our Perl
script from Hyatt and Newborn's publicly available result file. The
numbers exhibit the following unusual yet clearly sporadic fluctuations.
- DARKTHOUGHT - ``Best Change'' at iteration #12 and
iteration #13, ``Fresh Best'' at iteration #6 and iteration #12,
``(I - 2) Best'' at iteration #6, and ``(I - 2) Best'' at iteration
#12 and iteration #14.
- CRAFTY - ``Fresh Best'' at iteration #11 and iteration
#14.
We attribute these fluctuations to the reduced size of the test subset
as in the case of the opening positions (see Section 1.5.6).
Other than that the results for the middlegame positions alone closely
resemble those for the whole test set which Section 1.5.6 and
Section 1.5.6 already discussed in great detail. The analyses
of the overall results mostly apply to the results for the middlegame
subset as well. However, the ``Fresh Best'' rates of CRAFTY and
DARKTHOUGHT remained larger on average for the middlegame positions
alone than for all other positions at high search depths of 9-14
plies. As probably expected by most experts, the real middlegame
positions were therefore clearly the hardest for both programs to decide
on. Moreover, the behaviour of CRAFTY for the middlegame subset does
not lead to an increase of its ``Best Change'' rate at the final
iteration #14.
Created by Ernst A. Heinz, Thu Dec 16 23:28:11 EST 1999