the engine doesn't respond with bestmove #3532
Comments
|
Have you first insert the uci command?. SF responds then with the know options and uciok. Then it works. Tested with the current master 'Reduce in LMR reduction on PvNode' (self compiled). |
|
Sure the version is changed, try those commands with the version: Makefile: Extend sanitize support |
|
@voyag I can't reproduce the problem on my Mac. Could you please show us the |
|
I have used this version from abrok: https://abrok.eu/stockfish/builds/3802cdf9b69dd0a8256dbed7c25141ac5757d151/linux64avx2/stockfish_21060511_x64_avx2.zip |
|
OK. Could you try the output of |
|
Stockfish 050621 by the Stockfish developers (see AUTHORS file) Compiled by g++ (GNUC) 7.5.0 on Linux |
|
I have compiled the same version by myself and i get a result but it stucks at depth 9 and we never reach depth 10 (at least not in the time i waited). Stockfish 080621 by the Stockfish developers (see AUTHORS file) Compiled by g++ (GNUC) 9.2.1 on Linux The same holds for the bmi2 compile. EDIT: the version from abrok have the same behavior |
|
Some remarks:
So this is a difficult position for SF, she is searching and searching and hitting draws by 50 moves rule all over the place. But in fact everything is fine, as the move she will eventually output after the Depending on the exact version and the exact hash amount the command |
|
would be interesting to see how long we need to reach depth 10 on that position. Eventually there should be some output. I wonder if we have some extensions that are not limited one way or another. |
|
I have done more degugging and the search goes for root depth 10 at least to depth 144. ANd the reason is the singular extension. I try here to restrict this: https://tests.stockfishchess.org/tests/view/60bfb0b1457376eb8bcaa88c |
|
Here constrain the extension=2 case to captures/promotions: https://tests.stockfishchess.org/tests/view/60bfb3b6457376eb8bcaa895. Here we reach around depth 24. |
|
Do you think it would be possible to compare somehow the current level reached in search (as given by |
|
@snicolet Here we reach around depth 18. |
|
|
|
In this position every move by Black is probably singular, because Black has only the a8 and b7 squares to hold the draw. So Black takes the extension at every node in line 1073 of search.cpp (and probably even the double extension, hence the search explosion): I think I will play with the idea to take the extensions with probability 50% (something like that), and see what happens. |
|
What happens is that the bench goes from https://tests.stockfishchess.org/tests/view/60bfc823457376eb8bcaa8a6 |
|
In Crystal, I added the condition: && depth < rootDepth - ss->ply. YMMV. |
|
Another possibility is to keep track of the total extension of a subtree and stop allowing to extend further at some point. |
|
Most likely, yes. +2 extensions are really dangerous. TBH, I was surprised when this made it in without comment. I had to do something about it in Crystal because the combo of the +2 here and on game cycles was preventing bench from completing. |
|
since this doesn't appear to be a problem with SF13, do we know which recent commit has triggered this? I've started two more tests: just disabling extension by 2 seems not possible for now. |
|
It's also worth noting that even when reaching higher depths SF doesn't give drawish scores.
Older versions have no problem. (Stockfish 6)
|
|
sf 6 switches at 21 from 215 to 6, sf 7 - sf11 have a bit unstable eval, last high eval is 350 at depth 13 for sf11. I would see this as a independent issue, seems like nnue doesn't know this endgame, and we don't switch to classical? |
|
This might not be an issue for play, where the search is cut by maximum allowed time, but is an issue for analysis. Perhaps a fix could be applied only to infinite analysis, where the user relies on periodic updates. |
|
Interesting position. As a human you can quickly prove it's a draw with perfect play as follows: without the black pawn on b5 it's a well-known draw (wrong-colored bishop, doesn't matter there are two a-pawns, black plays Ka8, Kb7 alternatingly till hell freezes). The only thing white can try is to force black to play b4, in an attempt to transform the pawn on a3 into a b-pawn, but this is only possible after trapping the black king in the corner with e.g. Kb6 (for white), but then after b4 axb4 is stalemate. Not taking the pawn on b4 of course amounts to nothing for white, black can play b3, b2 etc. next and force a capture or blockade... 4c02998 explains why we don't switch to classical as often as before (before this simplification, we would always have switched to classical, as apart from pawns and kings, there is only a single bishop present, which is the precise corner case checked in the code), but I agree this "switching behavior" in itself is a separate issue. [speculation] |
|
I'm currently testing at LTC (after passed STC) https://tests.stockfishchess.org/tests/view/60c07eac457376eb8bcaa965 which make sure extensions terminate. It fixes the testcase, even if it is still a bit slower. It does appear to gain a little Elo. While search during game play will terminate, if this continues for a few moves, one will run out of time quickly. |
|
With classical eval:
As you can see classical eval does know it's a draw almost immediately. And because of this, search doesn't go into this pathological behaviour. Again, search and eval are much more connected than one might think. ;-) I have no doubt, though, that in some way or another we must guard against doing too many extensions at once. |
|
I've scheduled a test which enables classical eval again for low piece endgames, and this leads to the right eval for this endgame. However, it won't pass fishtest https://tests.stockfishchess.org/tests/view/60c0ea80457376eb8bcaa9bd |
|
The test here passed STC and LTC, and improves the situation (in particular the testcase above): https://tests.stockfishchess.org/tests/view/60c07eac457376eb8bcaa965 However, it is still not fully fixed, a related testcase takes very long to complete depth 25. Maybe there are better alternatives? |
|
@vondele How is my cap_ext patch (see fishtest) doing on that related testcase? |
|
let me first paste the testcase: |
|
@joergoster passes with your branch. |
|
Great! Thanks for testing. |
|
Try to tackle this by excluding endgames from big extension (The problem should be there more lilkely). |
|
The following fix passed testing, and should be a robust way to avoid the explosion. https://tests.stockfishchess.org/tests/view/60c196fb457376eb8bcaaa6b I'll make a PR. |
double extensions can lead to search explosions, for specific positions. Currently, however, this double extensions is worth about 10Elo and can't be removed. This patch instead limits the number of double extensions given to a maximum of 3. This fixes official-stockfish#3532 where the following testcase was shown to be problematic: ``` uci setoption name Hash value 4 setoption name Contempt value 0 ucinewgame position fen 8/Pk6/8/1p6/8/P1K5/8/6B1 w - - 37 130 go depth 20 ``` passed STC: https://tests.stockfishchess.org/tests/view/60c13161457376eb8bcaaa0f LLR: 2.94 (-2.94,2.94) <-2.50,0.50> Total: 166440 W: 5559 L: 5594 D: 155287 Ptnml(0-2): 106, 4921, 73197, 4894, 102 passed LTC: https://tests.stockfishchess.org/tests/view/60c196fb457376eb8bcaaa6b LLR: 2.95 (-2.94,2.94) <-2.50,0.50> Total: 73256 W: 6114 L: 6062 D: 61080 Ptnml(0-2): 222, 4912, 26306, 4968, 220 Bench: 5067605
double extensions can lead to search explosions, for specific positions. Currently, however, this double extensions is worth about 10Elo and can't be removed. This patch instead limits the number of double extensions given to a maximum of 3. This fixes official-stockfish#3532 where the following testcase was shown to be problematic: ``` uci setoption name Hash value 4 setoption name Contempt value 0 ucinewgame position fen 8/Pk6/8/1p6/8/P1K5/8/6B1 w - - 37 130 go depth 20 ``` passed STC: https://tests.stockfishchess.org/tests/view/60c13161457376eb8bcaaa0f LLR: 2.95 (-2.94,2.94) <-2.50,0.50> Total: 73256 W: 6114 L: 6062 D: 61080 Ptnml(0-2): 222, 4912, 26306, 4968, 220 passed LTC: https://tests.stockfishchess.org/tests/view/60c196fb457376eb8bcaaa6b LLR: 2.94 (-2.94,2.94) <-2.50,0.50> Total: 166440 W: 5559 L: 5594 D: 155287 Ptnml(0-2): 106, 4921, 73197, 4894, 102 Bench: 5067605
This patch detects some search explosions due to double extensions in our search algorithm which can happen in some pathological positions, and takes measure to ensure progress even in these pathological situations. While a small number of double extensions can be useful during search (for example to resolve a tactical sequence), a sustained regime of double extensions leads to search explosion and a non-finishing search. See the discussion in official-stockfish/Stockfish#3544 and the issue official-stockfish/Stockfish#3532 . The implemented algorithm is as follows: a) at each node during search, store the current depth in the stack. Double extensions are by definition levels of the stack where the depth at ply N is strictly higher than depth at ply N-1. b) during search, calculate for each thread a running average of the number of double extensions in the last 4096 visited nodes. c) if one thread has more than 2% of double extensions for a sustained period of time (6 millions consecutive nodes, or about 4 seconds on my iMac), we decide that this thread is in an explosion state and we calm down this thread by preventing it to do any double extension for the next 6 millions nodes. ----------- Example where the patch solves a search explosion: ``` ./stockfish ucinewgame position fen 8/Pk6/8/1p6/8/P1K5/8/6B1 w - - 37 130 go infinite ``` This algorithm does not affect search in normal, non-pathological positions. We verified that usual bench is unchanged up to depth 20 at least, for instance, and that the node numbers are unchanged for a search of the starting position at depth 32. ------------- Bench: 5575265
This patch detects some search explosions due to double extensions in our search algorithm which can happen in some pathological positions, and takes measure to ensure progress even in these pathological situations. While a small number of double extensions can be useful during search (for example to resolve a tactical sequence), a sustained regime of double extensions leads to search explosion and a non-finishing search. See the discussion in official-stockfish/Stockfish#3544 and the issue official-stockfish/Stockfish#3532 . The implemented algorithm is as follows: a) at each node during search, store the current depth in the stack. Double extensions are by definition levels of the stack where the depth at ply N is strictly higher than depth at ply N-1. b) during search, calculate for each thread a running average of the number of double extensions in the last 4096 visited nodes. c) if one thread has more than 2% of double extensions for a sustained period of time (6 millions consecutive nodes, or about 4 seconds on my iMac), we decide that this thread is in an explosion state and we calm down this thread by preventing it to do any double extension for the next 6 millions nodes. ----------- Example where the patch solves a search explosion: ``` ./stockfish ucinewgame position fen 8/Pk6/8/1p6/8/P1K5/8/6B1 w - - 37 130 go infinite ``` This algorithm does not affect search in normal, non-pathological positions. We verified that usual bench is unchanged up to depth 20 at least, for instance, and that the node numbers are unchanged for a search of the starting position at depth 32. ------------- Bench: 5575265
This patch detects some search explosions due to double extensions in our search algorithm which can happen in some pathological positions, and takes measure to ensure progress even in these pathological situations. While a small number of double extensions can be useful during search (for example to resolve a tactical sequence), a sustained regime of double extensions leads to search explosion and a non-finishing search. See the discussion in official-stockfish/Stockfish#3544 and the issue official-stockfish/Stockfish#3532 . The implemented algorithm is as follows: a) at each node during search, store the current depth in the stack. Double extensions are by definition levels of the stack where the depth at ply N is strictly higher than depth at ply N-1. b) during search, calculate for each thread a running average of the number of double extensions in the last 4096 visited nodes. c) if one thread has more than 2% of double extensions for a sustained period of time (6 millions consecutive nodes, or about 4 seconds on my iMac), we decide that this thread is in an explosion state and we calm down this thread by preventing it to do any double extension for the next 6 millions nodes. ----------- Example where the patch solves a search explosion: ``` ./stockfish ucinewgame position fen 8/Pk6/8/1p6/8/P1K5/8/6B1 w - - 37 130 go infinite ``` This algorithm does not affect search in normal, non-pathological positions. We verified that usual bench is unchanged up to depth 20 at least, for instance, and that the node numbers are unchanged for a search of the starting position at depth 32. ------------- See official-stockfish/Stockfish#3714 Bench: 5575265
This patch detects some search explosions (due to double extensions in search.cpp) which can happen in some pathological positions, and takes measures to ensure progress in search even for these pathological situations. While a small number of double extensions can be useful during search (for example to resolve a tactical sequence), a sustained regime of double extensions leads to search explosion and a non-finishing search. See the discussion in #3544 and the issue #3532 . The implemented algorithm is the following: a) at each node during search, store the current depth in the stack. Double extensions are by definition levels of the stack where the depth at ply N is strictly higher than depth at ply N-1. b) during search, calculate for each thread a running average of the number of double extensions in the last 4096 visited nodes. c) if one thread has more than 2% of double extensions for a sustained period of time (6 millions consecutive nodes, or about 4 seconds on my iMac), we decide that this thread is in an explosion state and we calm down this thread by preventing it to do any double extension for the next 6 millions nodes. To calculate the running averages, we also introduced a auxiliary class generalizing the computations of ttHitAverage variable we already had in code. The implementation uses an exponential moving average of period 4096 and resolution 1/1024, and all computations are done with integers for efficiency. ----------- Example where the patch solves a search explosion: ``` ./stockfish ucinewgame position fen 8/Pk6/8/1p6/8/P1K5/8/6B1 w - - 37 130 go infinite ``` This algorithm does not affect search in normal, non-pathological positions. We verified, for instance, that the usual bench is unchanged up to depth 20 at least, and that the node numbers are unchanged for a search of the starting position at depth 32. ------------- See #3714 Bench: 5575265
This patch detects some search explosions (due to double extensions in search.cpp) which can happen in some pathological positions, and takes measures to ensure progress in search even for these pathological situations. While a small number of double extensions can be useful during search (for example to resolve a tactical sequence), a sustained regime of double extensions leads to search explosion and a non-finishing search. See the discussion in official-stockfish#3544 and the issue official-stockfish#3532 . The implemented algorithm is the following: a) at each node during search, store the current depth in the stack. Double extensions are by definition levels of the stack where the depth at ply N is strictly higher than depth at ply N-1. b) during search, calculate for each thread a running average of the number of double extensions in the last 4096 visited nodes. c) if one thread has more than 2% of double extensions for a sustained period of time (6 millions consecutive nodes, or about 4 seconds on my iMac), we decide that this thread is in an explosion state and we calm down this thread by preventing it to do any double extension for the next 6 millions nodes. To calculate the running averages, we also introduced a auxiliary class generalizing the computations of ttHitAverage variable we already had in code. The implementation uses an exponential moving average of period 4096 and resolution 1/1024, and all computations are done with integers for efficiency. ----------- Example where the patch solves a search explosion: ``` ./stockfish ucinewgame position fen 8/Pk6/8/1p6/8/P1K5/8/6B1 w - - 37 130 go infinite ``` This algorithm does not affect search in normal, non-pathological positions. We verified, for instance, that the usual bench is unchanged up to depth 20 at least, and that the node numbers are unchanged for a search of the starting position at depth 32. ------------- See official-stockfish#3714 Bench: 5575265
With the current development version, from console with the following commands the engine stall and doesn't respond with bestmove
setoption name Hash value 4 setoption name Contempt value 0 ucinewgame position fen 8/Pk6/8/1p6/8/P1K5/8/6B1 w - - 37 130 go depth 10The text was updated successfully, but these errors were encountered: