This repository has been archived by the owner on Dec 18, 2023. It is now read-only.
Enabling algorithm-specific default num_adaptive_samples #1353
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Summary: The number of adaptive samples for all algorithm is set to 0 by default, which is okay for algorithms like ancestral MH or NMC, but not for HMC and NUTS. Per our previous discussion, instead of using the same default value for all algorithms, a better solution is to let each algorithm determine how many adaptive iterations are necessary. For compositional inference, it will look up the number of adaptive samples required by each inference method in the config and default to the maximum among them (see [Option 3 in Warmup Design](https://fb.quip.com/jzFvAai8ogWV#temp:C:WMIc97e2ffd843b4ab19bbe40c7c) for details). This diff implement this idea by introducing a `_get_default_num_adaptive_samples` private method on `BaseInference`, which is default to 0. Algorithms can override the default behavior by overloading the method. Differential Revision: D34404061 fbshipit-source-id: 5c073abb3abb993cf96ba5ef13e0923e859e6b09
facebook-github-bot
added
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D34404061 |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
The number of adaptive samples for all algorithm is set to 0 by default, which is okay for algorithms like ancestral MH or NMC, but not for HMC and NUTS. Per our previous discussion, instead of using the same default value for all algorithms, a better solution is to let each algorithm determine how many adaptive iterations are necessary. For compositional inference, it will look up the number of adaptive samples required by each inference method in the config and default to the maximum among them (see Option 3 in Warmup Design for details).
This diff implement this idea by introducing a
_get_default_num_adaptive_samplesprivate method onBaseInference, which is default to 0. Algorithms can override the default behavior by overloading the method.Differential Revision: D34404061