Backendbench assistant with TC-dependent output. #1546
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
My batchsize test matches have shown a strong dependency of optimal Clippy threshold to average nodes/move (results on Discord: https://discord.com/channels/425419482568196106/530486338236055583/821040507077394483). Now the assistant shows 3 proposals for best batchsize for the most common use cases.
Example:
NPS reported by backendbench are used for the calculation of expected nodes/move in games and 3 different thresholds are calculated to determine each of the 3 batchsizes. Threshold formula is also adjusted to account for the fact that real game NPS are higher than backendbench NPS.
Multi-GPU scenario is not considered here (even higher NPS in games compared to backendbench NPS on 1GPU), I'm not sure if this is necessary.