Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use msgspec.UNSET for tracking unset fields #350

Merged
merged 8 commits into from
Mar 23, 2023
Merged

Use msgspec.UNSET for tracking unset fields #350

merged 8 commits into from
Mar 23, 2023

Conversation

jcrist
Copy link
Owner

@jcrist jcrist commented Mar 23, 2023

msgspec.UNSET is a singleton object used to indicate that a field has no set value. This is useful for cases where you need to differentiate between a message where a field is missing and a message where the field is explicitly None.

>>> from msgspec import Struct, UnsetType, UNSET, json

>>> class Example(Struct):
...     x: int
...     y: int | None | UnsetType = UNSET  # a field, defaulting to UNSET

During encoding, any field containing UNSET is omitted from the message.

>>> json.encode(Example(1))  # y is UNSET
b'{"x":1}'

>>> json.encode(Example(1, None))  # y is None
b'{"x":1,"y":null}'

>>> json.encode(Example(1, 2))  # y is 2
b'{"x":1,"y":2}'

During decoding, if a field isn't explicitly set in the message, the default value of UNSET will be set instead. This lets downstream consumers determine whether a field was left unset, or explicitly set to None

>>> json.decode(b'{"x": 1}', type=Example)  # y defaults to UNSET
Example(x=1, y=UNSET)

>>> json.decode(b'{"x": 1, "y": null}', type=Example)  # y is None
Example(x=1, y=None)

>>> json.decode(b'{"x": 1, "y": 2}', type=Example)  # y is 2
Example(x=1, y=2)

UNSET fields are supported for msgspec.Struct, dataclasses, and attrs types. It is an error to use msgspec.UNSET or msgspec.UnsetType anywhere other than a field for one of these types.

For ease of use, UNSET is falsey and a singleton - usage in predicates should be fairly similar to how Python users already check for None:

def on_example(ex: Example):
    if ex.y is UNSET:
        # Handle y being unset
        ...
    elif ex.y is None:
        # Handle y being explicitly None
        ...
    else:
        # Handle y with actual values
        ...

Note that this repurposes the existing msgspec.UNSET singleton, and is thus a breaking change. Anyone who was previously making use of msgspec.UNSET should move to using msgspec.NODEFAULT instead. My hope is this breakage affects no users, as the previous msgspec.UNSET was fairly new, and was a pretty internal API.

Fixes #344.

This makes the internal `NODEFAULT` singleton public, and switches the
all previous usage of `msgspec.UNSET` to use `msgspec.NODEFAULT`
instead.

This is a *breaking change*. In the unlikely case a user was using
`msgspec.UNSET` directly (it's a fairly fringe API to use), they should
move to using `msgspec.NODEFAULT` instead. This breaking change is
needed so we can reclaim the `msgspec.UNSET` singleton for an
alternate purpose.
This adds support for encoding object-like types (Struct, dataclasses,
attrs) where some fields may contain the `msgspec.UNSET` singleton. In
this case, these fields are omitted from the encoded message.

Encoding `msgspec.UNSET` in locations other than a direct object field
(e.g. in `[1, 2, msgspec.UNSET]`) will result in a type error.
This makes UNSET's semantics the same as `None`, except it's a different
singleton.
Copy link

@SyntaxColoring SyntaxColoring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs make sense to me! Thanks. A couple of suggestions.

msgspec/__init__.pyi Outdated Show resolved Hide resolved
docs/source/supported-types.rst Show resolved Hide resolved
This lets type checkers do a better job of type narrowing. For ease of
implementation we only make this change in the type stubs. In normal
usage the singleton object and types should be treated as opaque objects
by users, lieing in the type stubs here seems unlikely to cause an
issue.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Losslessly represent omitted fields
2 participants