maintain big.Float precision with multiplication and modulo #75
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There are probably other places where arbitrarily large values can
overflow the default precision creating unexpected results, but
these two points are the easiest to encounter. There is also
the issue that many functions drop down to operating on native float64
or int internally (
floor
,log
,pow
, etc.). I chose to only handlethe
Multiply
andModulo
functions at this time, since they werealready using arbitrary precision values throughout.
When working with big.Float values, it is imperative to ensure the
desired precision for the operations is maintained throughout. The
default precision for a Float is set initially by the max argument
precision when creating the non-zero value. If this is coming from a
Float which was set via SetInt, and that Int value was created by
SetInt64, then the precision is only 64 bits.
The product of the arguments may require more precision than either
argument. The working precision was chosen to be 512, since that is
the value used with parsing large numbers from a string.