Appendix C  --  Published Results of Crafty from 1997

Table 1.14 lists the ``Best Change'' rates as originally published by Hyatt and Newborn in 1997 [114]. Because Hyatt and Newborn did not provide any explicit numbers for the 107 middlegame positions alone, we calculated them as described in Appendix 1.5.11 below. The numbers in parantheses represent the rates that our automatic Perl script computed when we applied it to Hyatt and Newborn's original result file of CRAFTY for the full set of 347 test positions. The slight differences do not affect the overall interpretation of the results and we regard them as constituting some omnipresent imprecisions of measurement. Please note that we debugged our Perl script thoroughly, performed many cross-checks during its execution and really trust its automatically generated output.

 
Table 1.14: ``Best Change'' Rates as Published by Hyatt and Newborn.
Search Crafty (Auto-) Crafty (Auto-) Crafty (Auto-) Crafty (Auto-)
Depth All (matic) Open (matic) Middle (matic) Rest (matic)
2 38.90% (38.90%) 35.50% (35.45%) 35.50% (35.51%) 44.60% (44.62%)
3 37.80% (36.89%) 49.10% (49.09%) 33.80% (31.78%) 31.50% (30.77%)
4 30.00% (30.55%) 30.00% (31.82%) 34.60% (33.64%) 26.20% (26.92%)
5 30.50% (30.55%) 33.60% (33.64%) 35.50% (35.51%) 23.80% (23.85%)
6 27.40% (27.38%) 30.00% (30.91%) 28.10% (28.04%) 24.60% (23.85%)
7 23.30% (25.07%) 30.00% (31.82%) 19.50% (21.50%) 20.80% (22.31%)
8 23.30% (22.48%) 26.40% (25.45%) 24.10% (21.50%) 20.00% (20.77%)
9 17.30% (18.16%) 21.80% (21.82%) 19.70% (20.56%) 11.50% (13.08%)
10 17.90% (17.00%) 20.00% (19.09%) 22.50% (21.50%) 12.30% (11.54%)
11 16.70% (16.43%) 20.00% (19.09%) 17.70% (18.69%) 13.10% (12.31%)
12 17.00% (17.00%) 23.60% (22.73%) 15.90% (16.82%) 12.30% (12.31%)
13 14.40% (14.70%) 17.30% (17.27%) 13.00% (13.08%) 13.10% (13.85%)
14 15.30% (15.56%) 19.10% (20.00%) 13.20% (13.08%) 13.80% (13.85%)
 

Results of Crafty for the Middlegame Positions in 1997  


Unfortunately, Hyatt and Newborn did not present the experimental results of CRAFTY for the 107 middlegame positions alone in their article. Therefore, we derived the ``Best Change'' rates of CRAFTY for these positions by ourselves from the available data of the other positions. The absolute number of new best moves as chosen by CRAFTY in the middlegame positions equals the number of new best moves for all test positions minus the new best moves for the opening positions minus the new best moves of the remaining positions. Table 1.15 shows our according calculations in detail and lists the resulting ``Best Change'' rates of CRAFTY for the 107 middlegame positions from iteration #2 to iteration #14.

 
Table 1.15: Calculated Results of Crafty for the 107 Middlegame Positions.
Search Calculation of the ``Best Change'' Rates   Best 
Depth for the 107 Middlegame Positions   Change
2 (0.389*347 - 0.355*110 - 0.446*130) / 107 = 35.50%
3 (0.378*347 - 0.491*110 - 0.315*130) / 107 = 33.80%
4 (0.300*347 - 0.300*110 - 0.262*130) / 107 = 34.60%
5 (0.305*347 - 0.336*110 - 0.238*130) / 107 = 35.50%
6 (0.274*347 - 0.300*110 - 0.246*130) / 107 = 28.10%
7 (0.233*347 - 0.300*110 - 0.208*130) / 107 = 19.50%
8 (0.233*347 - 0.264*110 - 0.200*130) / 107 = 24.10%
9 (0.173*347 - 0.218*110 - 0.115*130) / 107 = 19.70%
10 (0.179*347 - 0.200*110 - 0.123*130) / 107 = 22.50%
11 (0.167*347 - 0.200*110 - 0.131*130) / 107 = 17.70%
12 (0.170*347 - 0.236*110 - 0.123*130) / 107 = 15.90%
13 (0.144*347 - 0.173*110 - 0.131*130) / 107 = 13.00%
14 (0.153*347 - 0.191*110 - 0.138*130) / 107 = 13.20%
 



Created by Ernst A. Heinz, Thu Dec 16 23:28:11 EST 1999