-
-
Notifications
You must be signed in to change notification settings - Fork 632
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Deprecate TimeDelta serialization_type
parameter
#2654
Open
ddelange
wants to merge
3
commits into
marshmallow-code:dev
Choose a base branch
from
ddelange:timedelta-deserialize
base: dev
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -1435,45 +1435,52 @@ def _make_object_from_format(value, data_format): | |
|
||
|
||
class TimeDelta(Field): | ||
"""A field that (de)serializes a :class:`datetime.timedelta` object to an | ||
integer or float and vice versa. The integer or float can represent the | ||
number of days, seconds or microseconds. | ||
|
||
:param precision: Influences how the integer or float is interpreted during | ||
(de)serialization. Must be 'days', 'seconds', 'microseconds', | ||
'milliseconds', 'minutes', 'hours' or 'weeks'. | ||
:param serialization_type: Whether to (de)serialize to a `int` or `float`. | ||
:param kwargs: The same keyword arguments that :class:`Field` receives. | ||
"""A field that (de)serializes a :class:`datetime.timedelta` object to a `float`. | ||
The `float` can represent any time unit that the :class:`datetime.timedelta` constructor | ||
supports. | ||
|
||
Integer Caveats | ||
--------------- | ||
Any fractional parts (which depends on the precision used) will be truncated | ||
when serializing using `int`. | ||
:param precision: The time unit used for (de)serialization. Must be one of 'weeks', | ||
'days', 'hours', 'minutes', 'seconds', 'milliseconds' or 'microseconds'. | ||
:param kwargs: The same keyword arguments that :class:`Field` receives. | ||
|
||
Float Caveats | ||
------------- | ||
Use of `float` when (de)serializing may result in data precision loss due | ||
to the way machines handle floating point values. | ||
Precision loss may occur when serializing a highly precise :class:`datetime.timedelta` | ||
object using a big ``precision`` unit due to floating point arithmetics. | ||
|
||
Regardless of the precision chosen, the fractional part when using `float` | ||
will always be truncated to microseconds. | ||
For example, `1.12345` interpreted as microseconds will result in `timedelta(microseconds=1)`. | ||
When necessary, the :class:`datetime.timedelta` constructor rounds `float` inputs | ||
to whole microseconds during initialization of the object. As a result, deserializing | ||
a `float` might be subject to rounding, regardless of `precision`. For example, | ||
``TimeDelta().deserialize("1.1234567") == timedelta(seconds=1, microseconds=123457)``. | ||
|
||
.. versionchanged:: 2.0.0 | ||
Always serializes to an integer value to avoid rounding errors. | ||
Add `precision` parameter. | ||
.. versionchanged:: 3.17.0 | ||
Allow (de)serialization to `float` through use of a new `serialization_type` parameter. | ||
`int` is the default to retain previous behaviour. | ||
Allow serialization to `float` through use of a new `serialization_type` parameter. | ||
Defaults to `int` for backwards compatibility. Also affects deserialization. | ||
.. versionchanged:: 4.0.0 | ||
Deprecate `serialization_type` parameter, always serialize to float. | ||
""" | ||
|
||
WEEKS = "weeks" | ||
DAYS = "days" | ||
HOURS = "hours" | ||
MINUTES = "minutes" | ||
SECONDS = "seconds" | ||
MICROSECONDS = "microseconds" | ||
MILLISECONDS = "milliseconds" | ||
MINUTES = "minutes" | ||
HOURS = "hours" | ||
WEEKS = "weeks" | ||
MICROSECONDS = "microseconds" | ||
|
||
# cache this mapping on class level for performance | ||
_unit_to_microseconds_mapping = { | ||
WEEKS: 1000000 * 60 * 60 * 24 * 7, | ||
DAYS: 1000000 * 60 * 60 * 24, | ||
HOURS: 1000000 * 60 * 60, | ||
MINUTES: 1000000 * 60, | ||
SECONDS: 1000000, | ||
MILLISECONDS: 1000, | ||
MICROSECONDS: 1, | ||
} | ||
|
||
#: Default error messages. | ||
default_error_messages = { | ||
|
@@ -1484,49 +1491,38 @@ class TimeDelta(Field): | |
def __init__( | ||
self, | ||
precision: str = SECONDS, | ||
serialization_type: type[int | float] = int, | ||
serialization_type: typing.Any = missing_, | ||
**kwargs, | ||
): | ||
) -> None: | ||
precision = precision.lower() | ||
units = ( | ||
self.DAYS, | ||
self.SECONDS, | ||
self.MICROSECONDS, | ||
self.MILLISECONDS, | ||
self.MINUTES, | ||
self.HOURS, | ||
self.WEEKS, | ||
) | ||
|
||
if precision not in units: | ||
msg = 'The precision must be {} or "{}".'.format( | ||
", ".join([f'"{each}"' for each in units[:-1]]), units[-1] | ||
) | ||
if precision not in self._unit_to_microseconds_mapping: | ||
units = ", ".join(self._unit_to_microseconds_mapping) | ||
msg = f"The precision must be one of: {units}." | ||
raise ValueError(msg) | ||
|
||
if serialization_type not in (int, float): | ||
raise ValueError("The serialization type must be one of int or float") | ||
if serialization_type is not missing_: | ||
warnings.warn( | ||
"The 'serialization_type' argument to TimeDelta is deprecated.", | ||
RemovedInMarshmallow4Warning, | ||
stacklevel=2, | ||
) | ||
|
||
self.precision = precision | ||
self.serialization_type = serialization_type | ||
super().__init__(**kwargs) | ||
|
||
def _serialize(self, value, attr, obj, **kwargs): | ||
def _serialize(self, value, attr, obj, **kwargs) -> float | None: | ||
if value is None: | ||
return None | ||
|
||
base_unit = dt.timedelta(**{self.precision: 1}) | ||
# limit float arithmetics to a single division to minimize precision loss | ||
microseconds: int = utils.timedelta_to_microseconds(value) | ||
microseconds_per_unit: int = self._unit_to_microseconds_mapping[self.precision] | ||
return microseconds / microseconds_per_unit | ||
Comment on lines
+1518
to
+1521
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. apart from float precision issues with the old In [1]: from marshmallow import utils
...: from datetime import timedelta
...:
...: WEEKS = "weeks"
...: DAYS = "days"
...: HOURS = "hours"
...: MINUTES = "minutes"
...: SECONDS = "seconds"
...: MILLISECONDS = "milliseconds"
...: MICROSECONDS = "microseconds"
...:
...: _unit_to_microseconds_mapping = {
...: WEEKS: 1000000 * 60 * 60 * 24 * 7,
...: DAYS: 1000000 * 60 * 60 * 24,
...: HOURS: 1000000 * 60 * 60,
...: MINUTES: 1000000 * 60,
...: SECONDS: 1000000,
...: MILLISECONDS: 1000,
...: MICROSECONDS: 1,
...: }
...:
...: precision = WEEKS
...:
...: def serialize_old(value):
...: base_unit = timedelta(**{precision: 1})
...: return value.total_seconds() / base_unit.total_seconds()
...:
...: def serialize_new(value):
...: microseconds: int = utils.timedelta_to_microseconds(value)
...: microseconds_per_unit: int = _unit_to_microseconds_mapping[precision]
...: return microseconds / microseconds_per_unit
...:
...: value = timedelta(weeks=1, microseconds=1)
In [2]: %timeit serialize_old(value)
558 ns ± 3.16 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)
In [3]: %timeit serialize_new(value)
113 ns ± 0.583 ns per loop (mean ± std. dev. of 7 runs, 10,000,000 loops each) |
||
|
||
if self.serialization_type is int: | ||
delta = utils.timedelta_to_microseconds(value) | ||
unit = utils.timedelta_to_microseconds(base_unit) | ||
return delta // unit | ||
assert self.serialization_type is float | ||
return value.total_seconds() / base_unit.total_seconds() | ||
|
||
def _deserialize(self, value, attr, data, **kwargs): | ||
def _deserialize(self, value, attr, data, **kwargs) -> dt.timedelta: | ||
try: | ||
value = self.serialization_type(value) | ||
value = float(value) | ||
ddelange marked this conversation as resolved.
Show resolved
Hide resolved
|
||
except (TypeError, ValueError) as error: | ||
raise self.make_error("invalid") from error | ||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
unified this msg formatting with the Enum field ref.