wrong mate scores when searching with fixed nodes

[robertnurnberg]

It looks like The Huntsman 1 reports some wrong mates for the test suite from https://github.com/vondele/matetrack. The problematic positions are:

• k7/Pp1ppp2/1P4p1/7p/4P3/1p6/pP1P1PPP/K7 w - - where Huntsman reports #-12 instead of #11
• 7n/1p1K1p1N/bp3p1p/1p2prbr/1P2Rn1p/B6P/1PB2PP1/k7 w - - where Huntsman reports #20 instead of #21

The development version does not suffer from the same problem.

I guess it would be prudent to replace the latest release with a bug fix version.

Steps to reproduce:

wget https://github.com/joergoster/Stockfish-old/archive/refs/tags/h1.zip
unzip h1.zip
cd Stockfish-old-h1/src/ && make -j profile-build
cd ../..
git clone https://github.com/vondele/matetrack && cd matetrack
python matecheck.py --engine ../Stockfish-old-h1/src/huntsman >& log.txt &

The resulting output file log.txt:

6560 FENs loaded...

Matetrack started...
Found mate #-12 (wrong sign) for FEN "k7/Pp1ppp2/1P4p1/7p/4P3/1p6/pP1P1PPP/K7 w - -".
Found mate #20 (better) for FEN "7n/1p1K1p1N/bp3p1p/1p2prbr/1P2Rn1p/B6P/1PB2PP1/k7 w - -".

Using ../Stockfish-old-h1/src/huntsman with 1000000 nodes
Total fens:    6560
Found mates:   5240
Best mates:    3868
Better mates:  1
Wrong mates:   1

[joergoster]

No problem here:
info depth 85 seldepth 22 multipv 1 score mate 11 nodes 4605669 nps 1534711 hashfull 483 tbhits 0 time 3001 pv f2f4 d7d6 d2d4 d6d5 e4d5 h5h4 g2g4 h4g3 h2g3 f7f5 d5d6 e7d6 d4d5 g6g5 f4g5 f5f4 g5g6 f4g3 g6g7 g3g2 g7g8q
bestmove f2f4 ponder d7d6

info depth 34 seldepth 44 multipv 1 score mate 20 nodes 914337 nps 850546 hashfull 341 tbhits 0 time 1075 pv e4e1 a1a2 c2b1 a2a1 b1e4 a1a2 e1c1 a2b3 e4c2 b3a2 c2b1 a2a1 b1e4 a1a2 h7f8 a2b3 e4c2 b3a2 c2b1 a2a1 b1e4 a1a2 d7c8 a2b3 f8d7 b3a2 c8b8 a2b3 b8a7 h8g6 a7b6 f4d3 e4d5 b3a4 c1c3 g5e3 f2e3 g6f8 d5b3

Note: running with a fixed number of nodes may stop the search anywhere and the resulting scores may be unreliable.
Closing this as a non-issue.

[robertnurnberg]

Thanks to Huntsman for detecting an incorrect mate score in the matetrack test suite. :)

See vondele/matetrack#26

[joergoster]

I'm also re-opening this issue. Although I still think go mate x or go infinite is the main use case, there is definitely room for improvement for the mentioned go nodes and also go movetime cases.

[joergoster]

Just pushed my final fix for The Huntsman, so closing again.