add alphazero time manager #1201

MelleKoning · 2020-04-13T15:47:10Z

as per #968 but now on the new time manager factory.

startup with param:
--time-manager=alphazero

Comment copied over from #968 below:

Optionally play with alphazero time settings.

After talking with ASilver on Discord, found the explanation that the original alphazero used some different time setting, which is just use 1/20th (thus 5%) of the remaining time available for the upcoming move.

Additionally I just added the wtime (the time increment) for use directly for the upcoming move as well here.

Tests done:
This time setting performs worse than the default lczero time setting.

In only 10 games (60sec+1sec) the result was 3 wins for lczero time, 1 for aztime, 6 draws.
Other 10 games (10sec+0.1sec) the result was even worse:

Score of lc0.net.42850 vs lc0az.net.42850: 5 - 1 - 4 [0.700]
Elo difference: 147.19 +/- 197.89

10 of 10 games finished.

However, this is just an option that does not affect engine strength if you do not enable it, so might be interesting for people who want to play a bit with the setting. You can alter the value used by Alphazero, dividing remaining time always by 20 to other values from 2: take half the time left for upcoming move, to 100: take 1% of time left for upcoming move, to see what difference that makes...

Have fun testing! :)

MelleKoning · 2020-04-13T16:34:08Z

output of --help seems confusing

startup with --time-manager=alphazero works but --help gives

--alphazero-time-value=2.00..100.00
               Remaining time is divided by this value. Default value of 20 uses alphazero
               time.Lower values will spend more time in beginning of game, higher values will
               save more timefor later in the game
               [UCI: AlphazeroTimeValue  DEFAULT: 20.00  MIN: 2.00  MAX: 100.00]

       --time-manager=STRING
               Name and config of atime manager.
               [UCI: TimeManager  DEFAULT: legacy]

       --alphazero=STRING
               Makes the engine use AlphaZero time. The alphazero engine always budgeted 5
               percent of total time left for the first upcoming move, thereby gradually taking
               less time for each consecutive move in the game..
               [UCI: Alphazero  DEFAULT: alphazero]

Any ideas to make this better?

mooskagh · 2020-04-13T17:45:57Z

Like with backend options which are different per different backend, time manager options are not thought to be discoverable for now other than in external documentation (e.g. see http://lczero.org/dev/docs/timemgr/#configuration-parameters for "smooth" time manager).

Also it doesn't really good to have global options that only work in some cases (e.g. in your case alphazero-time-value won't affect anything with the default time manager, but still will be displayed). So for now the recommended way is to access parameters through OptionsDict passed to the TimeManager.

There are some thoughts to change it, but for now that's what we have.

MelleKoning · 2020-04-13T20:07:03Z

@mooskagh updated accordingly, extended the --help documentation a bit.

Will gladly update the documentation on lczero.org when possible and/or when merged.

src/mcts/stoppers/alphazero.cc

src/mcts/stoppers/factory.cc

MelleKoning · 2020-04-26T10:49:12Z

@mooskagh merged with master; it had a change in TimeManager base class which caused merge conflict, is now resolved.

MelleKoning · 2020-04-30T07:33:59Z

@Naphthalin @mooskagh @Tilps Have merged with latest 0.25, no merge conflicts.
Can you please approve and merge?

src/mcts/stoppers/factory.cc

Update with suggested description.

MelleKoning · 2020-05-01T07:27:20Z

Link to (draft) documentation available:
LeelaChessZero/lczero.org#29

borg323 · 2020-05-03T22:42:56Z

src/mcts/stoppers/alphazero.cc

+ public:
+  AlphazeroStopper(int64_t deadline_ms, int64_t* time_piggy_bank)
+      : TimeLimitStopper(deadline_ms), time_piggy_bank_(time_piggy_bank) {}
+  virtual void OnSearchDone(const IterationStats& stats) override {


Why virtual?

It's just as copied over from legacy.cc (assuming that's what is wanted)

have removed the virtual now

mooskagh · 2020-05-08T20:26:40Z

src/mcts/stoppers/alphazero.cc

+ public:
+  AlphazeroTimeManager(int64_t move_overhead, const OptionsDict& params)
+      : move_overhead_(move_overhead),
+        alphazerotimevalue_(params.GetOrDefault<float>("alphazero-time-value", 20.0f)),


The variable (both configuration and C++ variable) name is quite too generic. :)
time-use-fraction or something like that might be better, but also not good as it's inverted fraction..

borg323 · 2020-05-08T21:16:13Z

src/mcts/stoppers/alphazero.cc

+  // Make sure we don't exceed current time limit with what we calculated.
+  auto deadline =
+      std::min(static_cast<int64_t>(this_move_time), *time + increment - move_overhead_);
+  return std::make_unique<AlphazeroStopper>(deadline, &time_spared_ms_);


This can be just return std::make_unique<TimeLimitStopper>(deadline); since time_spared_ms_ is updated but not used, and the AlphazeroStopper can be removed.

* Time management refactoring (LeelaChessZero#1195) * Appended files. * Compiles. * Compiles again. * Make smart pruning use smoothed nps. * Seems to be fully implemented. * Mistype. * One more bug. * Found discrepancy with documentaiton. * Bugfixes. * Don't smooth nps during the first move. * Too large default for timeuse decay. * Bugfix. * Fix build. * Relax defaults a bit. Add fixed to logging. * Remove "smooth" to "smooth-experimental" for now. * MLH verbose stats - Issue 1200 (LeelaChessZero#1230) * Add M effect logic to output section * Fix missing prefixes and semicolons * Some fixes. * Slight format improvement? Co-authored-by: Tilps <[email protected]> * Start TempDecay only after a given number of moves (LeelaChessZero#1212) * Added TempDecayStartMove for starting temp decay only after a given number of moves. This allows keeping initial game up for a few moves and still use decay. * Doesn't allow temperature to fall below endgame temp during temp decay. Still allows initial temp to be below endgame temp. * Doesn't allow temperature to fall below endgame temp during temp decay. Still allows initial temp to be below endgame temp. * Hide temp options * renamed TempDecayStartMove to TempDecayDelayMoves Co-authored-by: Alexis Olson <[email protected]> * Changelog for 0.25.0-rc2. (LeelaChessZero#1233) * Changelog for 0.25.0-rc2. * Add one more PR to the changelog. * Cuda winograd (LeelaChessZero#1228) * custom winograd convolution for cuda backends * custom winograd fixes - fix a bug to make it work for non-SE networks - enable by default only with fp32. * address review comments * remove random line in comment * remove unused constants - W,H are hardcoded to 8 - because there are assumptions in the code based on that. No point in defining constants. * cuda winograd fixes (LeelaChessZero#1238) * cuda winograd fixes - don't typecast directly to half datatype in CPU side code as older CUDA runtime doesn't support that. - don't use gemmEx version on GPUs older than Maxwell generation (not supported). - modify the check to enable custom_winograd setting. It should be faster in most cases - except presently on RTX GPUs when using fp16. * Allow most parts of fen to be optional. (LeelaChessZero#1234) Default to white to move, no castling, no en passant, 0 rule50ply, 1 total move. Also convert other string to std::string and removing using. * Fix UpdateNps to actually smooth the nps and correctly handle time_since_movestart_ms == 0 (LeelaChessZero#1243) * Update changelog for 0.25.0 final release. (LeelaChessZero#1244) * Always report at least 1 depth. (LeelaChessZero#1247) * Fix un-intended regression for GTX GPUs (LeelaChessZero#1246) * memory optimization for cudnn custom_winograd (LeelaChessZero#1250) * memory optimization for cudnn custom_winograd - don't save untransformed weights - print warning message when low memory is detected. * address review comments * fix warning message * fix total weight size calculation 2 layers per residual block! * keep pdb files only for release builds (LeelaChessZero#1256) * doc update (LeelaChessZero#1267) * Include verbose stats for the node. (LeelaChessZero#1268) Use printing lambdas for parts of the verbose output to share between the newly outputted node and its children. * add alphazero time manager (LeelaChessZero#1201) * Updated FLAGS.md with logfile flag (LeelaChessZero#1275) * Fixed a typo in CONTRIBUTING.md (LeelaChessZero#1274) * Update Readme about using git (LeelaChessZero#1265) * Make `wl_` double. (LeelaChessZero#1280) * Move move filter population to a constructor. (LeelaChessZero#1281) * Filter out illegal searchmoves to avoid crashing. (LeelaChessZero#1282) * Clear policy for terminal loss. (LeelaChessZero#1285) * Allow smart pruning to terminate search if win is known. (LeelaChessZero#1284) * Allow smart pruning to terminate search if win is known. * Minor tweak, better safe than sorry. * Fix bug where pv might not update for best move change. (LeelaChessZero#1286) * Fix bug where pv might not update. * Fix... Co-authored-by: Alexander Lyashuk <[email protected]> Co-authored-by: Tilps <[email protected]> Co-authored-by: Naphthalin <[email protected]> Co-authored-by: Ankan Banerjee <[email protected]> Co-authored-by: Ed Lee <[email protected]> Co-authored-by: borg323 <[email protected]> Co-authored-by: Hace <[email protected]> Co-authored-by: Kip Hamiltons <[email protected]> Co-authored-by: nguyenpham <[email protected]>

@Naphthalin

* Time management refactoring (#1195) * Appended files. * Compiles. * Compiles again. * Make smart pruning use smoothed nps. * Seems to be fully implemented. * Mistype. * One more bug. * Found discrepancy with documentaiton. * Bugfixes. * Don't smooth nps during the first move. * Too large default for timeuse decay. * Bugfix. * Fix build. * Relax defaults a bit. Add fixed to logging. * Remove "smooth" to "smooth-experimental" for now. * MLH verbose stats - Issue 1200 (#1230) * Add M effect logic to output section * Fix missing prefixes and semicolons * Some fixes. * Slight format improvement? Co-authored-by: Tilps <[email protected]> * Start TempDecay only after a given number of moves (#1212) * Added TempDecayStartMove for starting temp decay only after a given number of moves. This allows keeping initial game up for a few moves and still use decay. * Doesn't allow temperature to fall below endgame temp during temp decay. Still allows initial temp to be below endgame temp. * Doesn't allow temperature to fall below endgame temp during temp decay. Still allows initial temp to be below endgame temp. * Hide temp options * renamed TempDecayStartMove to TempDecayDelayMoves Co-authored-by: Alexis Olson <[email protected]> * Changelog for 0.25.0-rc2. (#1233) * Changelog for 0.25.0-rc2. * Add one more PR to the changelog. * Cuda winograd (#1228) * custom winograd convolution for cuda backends * custom winograd fixes - fix a bug to make it work for non-SE networks - enable by default only with fp32. * address review comments * remove random line in comment * remove unused constants - W,H are hardcoded to 8 - because there are assumptions in the code based on that. No point in defining constants. * cuda winograd fixes (#1238) * cuda winograd fixes - don't typecast directly to half datatype in CPU side code as older CUDA runtime doesn't support that. - don't use gemmEx version on GPUs older than Maxwell generation (not supported). - modify the check to enable custom_winograd setting. It should be faster in most cases - except presently on RTX GPUs when using fp16. * Allow most parts of fen to be optional. (#1234) Default to white to move, no castling, no en passant, 0 rule50ply, 1 total move. Also convert other string to std::string and removing using. * Fix UpdateNps to actually smooth the nps and correctly handle time_since_movestart_ms == 0 (#1243) * Update changelog for 0.25.0 final release. (#1244) * Always report at least 1 depth. (#1247) * Fix un-intended regression for GTX GPUs (#1246) * memory optimization for cudnn custom_winograd (#1250) * memory optimization for cudnn custom_winograd - don't save untransformed weights - print warning message when low memory is detected. * address review comments * fix warning message * fix total weight size calculation 2 layers per residual block! * keep pdb files only for release builds (#1256) * doc update (#1267) * Include verbose stats for the node. (#1268) Use printing lambdas for parts of the verbose output to share between the newly outputted node and its children. * add alphazero time manager (#1201) * Updated FLAGS.md with logfile flag (#1275) * Fixed a typo in CONTRIBUTING.md (#1274) * Update Readme about using git (#1265) * Make `wl_` double. (#1280) * Move move filter population to a constructor. (#1281) * Filter out illegal searchmoves to avoid crashing. (#1282) * Clear policy for terminal loss. (#1285) * Allow smart pruning to terminate search if win is known. (#1284) * Allow smart pruning to terminate search if win is known. * Minor tweak, better safe than sorry. * Fix bug where pv might not update for best move change. (#1286) * Fix bug where pv might not update. * Fix... * Catch up to master (#6) * Time management refactoring (#1195) * Appended files. * Compiles. * Compiles again. * Make smart pruning use smoothed nps. * Seems to be fully implemented. * Mistype. * One more bug. * Found discrepancy with documentaiton. * Bugfixes. * Don't smooth nps during the first move. * Too large default for timeuse decay. * Bugfix. * Fix build. * Relax defaults a bit. Add fixed to logging. * Remove "smooth" to "smooth-experimental" for now. * MLH verbose stats - Issue 1200 (#1230) * Add M effect logic to output section * Fix missing prefixes and semicolons * Some fixes. * Slight format improvement? Co-authored-by: Tilps <[email protected]> * Start TempDecay only after a given number of moves (#1212) * Added TempDecayStartMove for starting temp decay only after a given number of moves. This allows keeping initial game up for a few moves and still use decay. * Doesn't allow temperature to fall below endgame temp during temp decay. Still allows initial temp to be below endgame temp. * Doesn't allow temperature to fall below endgame temp during temp decay. Still allows initial temp to be below endgame temp. * Hide temp options * renamed TempDecayStartMove to TempDecayDelayMoves Co-authored-by: Alexis Olson <[email protected]> * Changelog for 0.25.0-rc2. (#1233) * Changelog for 0.25.0-rc2. * Add one more PR to the changelog. * Cuda winograd (#1228) * custom winograd convolution for cuda backends * custom winograd fixes - fix a bug to make it work for non-SE networks - enable by default only with fp32. * address review comments * remove random line in comment * remove unused constants - W,H are hardcoded to 8 - because there are assumptions in the code based on that. No point in defining constants. * cuda winograd fixes (#1238) * cuda winograd fixes - don't typecast directly to half datatype in CPU side code as older CUDA runtime doesn't support that. - don't use gemmEx version on GPUs older than Maxwell generation (not supported). - modify the check to enable custom_winograd setting. It should be faster in most cases - except presently on RTX GPUs when using fp16. * Allow most parts of fen to be optional. (#1234) Default to white to move, no castling, no en passant, 0 rule50ply, 1 total move. Also convert other string to std::string and removing using. * Fix UpdateNps to actually smooth the nps and correctly handle time_since_movestart_ms == 0 (#1243) * Update changelog for 0.25.0 final release. (#1244) * Always report at least 1 depth. (#1247) * Fix un-intended regression for GTX GPUs (#1246) * memory optimization for cudnn custom_winograd (#1250) * memory optimization for cudnn custom_winograd - don't save untransformed weights - print warning message when low memory is detected. * address review comments * fix warning message * fix total weight size calculation 2 layers per residual block! * keep pdb files only for release builds (#1256) * doc update (#1267) * Include verbose stats for the node. (#1268) Use printing lambdas for parts of the verbose output to share between the newly outputted node and its children. * add alphazero time manager (#1201) * Updated FLAGS.md with logfile flag (#1275) * Fixed a typo in CONTRIBUTING.md (#1274) * Update Readme about using git (#1265) * Make `wl_` double. (#1280) * Move move filter population to a constructor. (#1281) * Filter out illegal searchmoves to avoid crashing. (#1282) * Clear policy for terminal loss. (#1285) * Allow smart pruning to terminate search if win is known. (#1284) * Allow smart pruning to terminate search if win is known. * Minor tweak, better safe than sorry. * Fix bug where pv might not update for best move change. (#1286) * Fix bug where pv might not update. * Fix... Co-authored-by: Alexander Lyashuk <[email protected]> Co-authored-by: Tilps <[email protected]> Co-authored-by: Naphthalin <[email protected]> Co-authored-by: Ankan Banerjee <[email protected]> Co-authored-by: Ed Lee <[email protected]> Co-authored-by: borg323 <[email protected]> Co-authored-by: Hace <[email protected]> Co-authored-by: Kip Hamiltons <[email protected]> Co-authored-by: nguyenpham <[email protected]> * Change defaults and unhide MLH options * Update values per @Naphthalin's comments Co-authored-by: Alexander Lyashuk <[email protected]> Co-authored-by: Tilps <[email protected]> Co-authored-by: Naphthalin <[email protected]> Co-authored-by: Ankan Banerjee <[email protected]> Co-authored-by: Ed Lee <[email protected]> Co-authored-by: borg323 <[email protected]> Co-authored-by: Hace <[email protected]> Co-authored-by: Kip Hamiltons <[email protected]> Co-authored-by: nguyenpham <[email protected]>

add alphazero time manager

a07b3f2

MelleKoning mentioned this pull request Apr 13, 2020

Optional alphazero time #968

Closed

add aphazero.cc to meson.build

5f06d4c

remove unnecessary options

43f1e85

mooskagh requested changes Apr 17, 2020

View reviewed changes

src/mcts/stoppers/alphazero.cc Outdated Show resolved Hide resolved

src/mcts/stoppers/factory.cc Outdated Show resolved Hide resolved

remove code and update helptext

3c2c7ad

MelleKoning mentioned this pull request Apr 19, 2020

add alphazero time management documentation LeelaChessZero/lczero.org#29

Merged

MelleKoning added 2 commits April 26, 2020 11:23

Merge branch 'master' into alphazerotime2

10e007f

adapt to master base class change

0672f61

MelleKoning requested a review from mooskagh April 26, 2020 10:53

Merge branch 'master' into alphazerotime2

c9ef22d

mooskagh reviewed Apr 30, 2020

View reviewed changes

src/mcts/stoppers/factory.cc Outdated Show resolved Hide resolved

Update time manager description

e03d504

Update with suggested description.

borg323 reviewed May 3, 2020

View reviewed changes

remove virtual from func

31ea1f7

mooskagh approved these changes May 8, 2020

View reviewed changes

mooskagh merged commit b6b3011 into LeelaChessZero:master May 8, 2020

borg323 reviewed May 8, 2020

View reviewed changes

AlexisOlson pushed a commit to AlexisOlson/lc0 that referenced this pull request May 10, 2020

add alphazero time manager (LeelaChessZero#1201)

d98eceb

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add alphazero time manager #1201

add alphazero time manager #1201

MelleKoning commented Apr 13, 2020 •

edited

Loading

MelleKoning commented Apr 13, 2020

mooskagh commented Apr 13, 2020

MelleKoning commented Apr 13, 2020 •

edited

Loading

MelleKoning commented Apr 26, 2020

MelleKoning commented Apr 30, 2020

MelleKoning commented May 1, 2020

borg323 May 3, 2020

MelleKoning May 7, 2020

MelleKoning May 8, 2020

mooskagh May 8, 2020

borg323 May 8, 2020

add alphazero time manager #1201

add alphazero time manager #1201

Conversation

MelleKoning commented Apr 13, 2020 • edited Loading

MelleKoning commented Apr 13, 2020

mooskagh commented Apr 13, 2020

MelleKoning commented Apr 13, 2020 • edited Loading

MelleKoning commented Apr 26, 2020

MelleKoning commented Apr 30, 2020

MelleKoning commented May 1, 2020

borg323 May 3, 2020

Choose a reason for hiding this comment

MelleKoning May 7, 2020

Choose a reason for hiding this comment

MelleKoning May 8, 2020

Choose a reason for hiding this comment

mooskagh May 8, 2020

Choose a reason for hiding this comment

borg323 May 8, 2020

Choose a reason for hiding this comment

MelleKoning commented Apr 13, 2020 •

edited

Loading

MelleKoning commented Apr 13, 2020 •

edited

Loading