Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support encoding decimals as numbers #465

Merged
merged 1 commit into from
Jul 4, 2023
Merged

Support encoding decimals as numbers #465

merged 1 commit into from
Jul 4, 2023

Conversation

jcrist
Copy link
Owner

@jcrist jcrist commented Jul 4, 2023

Previously we always encoded decimal.Decimal values as strings. While this is the best format for these types (prevents precision loss, widely supported), sometimes there is a need to encode Decimal objects the same as numbers. This adds a new decimal_format option to all Encoder classes. This value defaults to "string" (to encode decimals as strings), but may be set to "number" to encode them the same as the protocol's numeric types.

We opt not to support this configuration in the top-level encode functions for now, since this is a less-common setting.

Fixes #440.

Previously we always encoded `decimal.Decimal` values as strings. While
this is the best format for these types (prevents precision loss, widely
supported), sometimes there is a need to encode `Decimal` objects the
same as numbers. This adds a new `decimal_format` option to all
`Encoder` classes. This value defaults to `"string"` (to encode decimals
as strings), but may be set to `"number"` to encode them the same as the
protocol's numeric types.

We opt not to support this configuration in the top-level `encode`
functions for now, since this is a less-common setting.
@jcrist
Copy link
Owner Author

jcrist commented Jul 4, 2023

A quick demo:

In [1]: import msgspec, decimal

In [2]: x = decimal.Decimal("1.3")

In [3]: msgspec.json.encode(x)  # by default encodes as a string
Out[3]: b'"1.3"'

In [4]: encoder = msgspec.json.Encoder(decimal_format="number")  # create an encoder to override

In [5]: encoder.encode(x)  # now encodes as a number
Out[5]: b'1.3'

In [6]: import json

In [7]: decimal.Decimal(json.loads(b'1.3'))  # downstream tools may have trouble loading decimals from numbers without precision loss
Out[7]: Decimal('1.3000000000000000444089209850062616169452667236328125')

In [8]: msgspec.json.decode(b'1.3', type=decimal.Decimal)  # msgspec has no issues with this though, but other tools may
Out[8]: Decimal('1.3')

@jcrist
Copy link
Owner Author

jcrist commented Jul 4, 2023

If you wish to try this out before the next release, please see our instructions for installing from GitHub.

@s7clarke10
Copy link

Brilliant, thank you very much for taking this on board and providing the ability to output as a decimal. I do take you point that there may be a loss of precision. I will utilize configuration to select whether we wish to output as a string or decimal using this feature.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Option to JSON encode Decimal types as numbers, not strings
2 participants