Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

first attempt at an adjacently tagged union for consideration #94

Draft
wants to merge 60 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
921e8e4
first attempt at an adjacently tagged union for consideration
altendky Mar 12, 2020
4f98346
black
altendky Mar 12, 2020
14c7273
raise on extra keys when deserializing adjacently tagged
altendky Mar 12, 2020
cd0f7b6
remove trailing comma on single line, thanks black
altendky Mar 12, 2020
08c6e0c
correct TypeTagField.field type hint
altendky Mar 12, 2020
ddb5df4
add str and decimal.Decimal examples
altendky Mar 12, 2020
2492519
break by adding a couple of different list examples
altendky Mar 12, 2020
1ce7e67
remove explicit registry collision check
altendky Mar 13, 2020
cac94de
correct tests, better demonstrate failures, prep for additional regis…
altendky Mar 13, 2020
f2cffbe
black...
altendky Mar 13, 2020
84cd5ed
maybe interesting
altendky May 30, 2020
e76a271
use typeguard instead, for now at least
altendky May 31, 2020
18e78d3
black
altendky May 31, 2020
b233cf4
cleanup
altendky May 31, 2020
7a44b41
no protocol for now
altendky May 31, 2020
7731c00
remove duplicate code
altendky May 31, 2020
b30170e
pop a little
altendky May 31, 2020
21c9e1b
extract adjacent functions and generalize to TaggedUnion
altendky May 31, 2020
6a4bae8
group adjacently tagged functions
altendky May 31, 2020
8b97cf6
add externally tagged support
altendky May 31, 2020
f63f290
add internally tagged support
altendky May 31, 2020
1b41217
make tagged type and value keys configurable
altendky May 31, 2020
74494eb
complain if other than exactly one hint matches
altendky Jun 1, 2020
d5d7183
isinstance() for str vs. typing.Sequence[str] handling
altendky Jun 1, 2020
39104a7
Merge branch 'master' into 36-altendky-first_attempt
altendky Jul 29, 2021
f29ef21
black
altendky Jul 29, 2021
8633a8e
Merge branch 'main' into 36-altendky-first_attempt
altendky Aug 5, 2021
d76ec24
additional heuristics for List vs. Sequence, 3.7+ only
altendky Aug 5, 2021
4b4892f
Merge branch 'main' into 36-altendky-first_attempt
altendky Aug 5, 2021
39caf6f
make test examples frozen
altendky Aug 5, 2021
e615380
mypy
altendky Aug 5, 2021
3090a14
Merge branch 'main' into 36-altendky-first_attempt
altendky Aug 6, 2021
0e9c3a9
actually use typing-extensions
altendky Aug 6, 2021
419fd30
black
altendky Aug 6, 2021
7034ea2
check
altendky Aug 6, 2021
fc889f6
coverage
altendky Aug 6, 2021
408b21f
create specific exceptions
altendky Aug 6, 2021
cc9279f
check
altendky Aug 6, 2021
a6dfccd
specify field registry protocol and check it
altendky Aug 6, 2021
c7a385e
add test that looks like real user code
altendky Aug 6, 2021
d610e5e
just use Any for now
altendky Aug 6, 2021
47197f6
drop some args/kwargs to be more explicit
altendky Aug 6, 2021
c2cf8a6
docstrings and some more
altendky Aug 9, 2021
6722760
use typing_inspect.get_origin() instead of .__origin__
altendky Aug 17, 2021
47861f9
Merge branch 'main' into 36-altendky-first_attempt
altendky Aug 18, 2021
76bc4da
type hinting catchup
altendky Aug 19, 2021
918f2ab
Merge branch 'main' into 36-altendky-first_attempt
altendky Aug 19, 2021
aa85230
tagged union documentation
altendky Aug 23, 2021
64a20d9
add missing snippets
altendky Aug 23, 2021
c0f1ef6
Merge branch 'main' into 36-altendky-first_attempt
altendky Aug 24, 2021
82f92be
black
altendky Aug 24, 2021
2b8ca42
doc warning tidy
altendky Aug 24, 2021
c256d80
docutils < 0.17 for circular include error
altendky Aug 24, 2021
dc5d731
avoid trailing blank lines in example in docs
altendky Aug 24, 2021
bd64aa0
xfail for working-.__origin__-requiring tests on < 3.7
altendky Aug 24, 2021
f8a377f
parametrize against *_tagged_union[_from_registry] for coverage
altendky Aug 25, 2021
694000f
actually test tagged union examples
altendky Aug 30, 2021
d5acb63
add new example resources
altendky Aug 30, 2021
f6d2314
isort
altendky Aug 30, 2021
ea773b7
CatsAndDogs corrected to CatOrDog
altendky Sep 8, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 83 additions & 0 deletions src/desert/_fields.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
import typing

import attr
import marshmallow.fields


T = typing.TypeVar("T")


@attr.s(frozen=True, auto_attribs=True)
class TypeTagField:
cls: type
tag: str
field: typing.Type[marshmallow.fields.Field]


@attr.s(auto_attribs=True)
class TypeDictRegistry:
the_dict: typing.Dict[typing.Union[type, str], marshmallow.fields.Field] = attr.ib(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not a big the_foo user, perhaps .mapping or something.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heh, yeah, that was a garbage name so as to not think about it and worry about other things. Could also just be .dict

factory=dict
)

def register(self, cls, tag, field):
if any(key in self.the_dict for key in [cls, tag]):
raise Exception()

type_tag_field = TypeTagField(cls=cls, tag=tag, field=field)

self.the_dict[cls] = type_tag_field
self.the_dict[tag] = type_tag_field

# TODO: this type hinting... doesn't help much as it could return
# another cls
def __call__(self, tag: str, field: marshmallow.fields) -> typing.Callable[[T], T]:
return lambda cls: self.register(cls=cls, tag=tag, field=field)

def from_object(self, value):
return self.the_dict[type(value)]

def from_tag(self, tag):
return self.the_dict[tag]


class AdjacentlyTaggedUnion(marshmallow.fields.Field):
def __init__(
self,
*,
from_object: typing.Callable[[typing.Any], TypeTagField],
from_tag: typing.Callable[[str], TypeTagField],
**kwargs,
):
super().__init__(**kwargs)

self.from_object = from_object
self.from_tag = from_tag

def _deserialize(
self,
value: typing.Any,
attr: typing.Optional[str],
data: typing.Optional[typing.Mapping[str, typing.Any]],
**kwargs,
) -> typing.Any:
tag = value["type"]
serialized_value = value["value"]

if len(value) > 2:
raise Exception()

type_tag_field = self.from_tag(tag)
field = type_tag_field.field()

return field.deserialize(serialized_value)

def _serialize(
self, value: typing.Any, attr: str, obj: typing.Any, **kwargs,
) -> typing.Any:
type_tag_field = self.from_object(value)
field = type_tag_field.field()
tag = type_tag_field.tag
serialized_value = field.serialize(attr, obj)

return {"type": tag, "value": serialized_value}
98 changes: 98 additions & 0 deletions tests/test_fields.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
import decimal
import typing

import attr
import marshmallow
import pytest

import desert._fields


# TODO: test that field constructor doesn't tromple Field parameters


@attr.s(auto_attribs=True)
class ExampleData:
object: typing.Any
tag: str
field: typing.Callable[[], marshmallow.fields.Field]


example_data_list = [
ExampleData(object=3.7, tag="float_tag", field=marshmallow.fields.Float),
ExampleData(object="29", tag="str_tag", field=marshmallow.fields.String),
ExampleData(
object=decimal.Decimal("4.2"),
tag="decimal_tag",
field=marshmallow.fields.Decimal,
),
ExampleData(
object=[1, 2, 3],
tag="integer_list_tag",
field=lambda: marshmallow.fields.List(marshmallow.fields.Integer()),
),
ExampleData(
object=['abc', '2', 'mno'],
tag="string_list_tag",
field=lambda: marshmallow.fields.List(marshmallow.fields.String()),
),
]


@pytest.fixture(
name="example_data",
params=example_data_list,
ids=[str(example) for example in example_data_list],
)
def _example_data(request):
return request.param


@pytest.fixture(name="registry", scope="session")
altendky marked this conversation as resolved.
Show resolved Hide resolved
def _registry():
registry = desert._fields.TypeDictRegistry()

for example in example_data_list:
registry.register(
cls=type(example.object), tag=example.tag, field=example.field,
)

return registry


@pytest.fixture(name="adjacently_tagged_field", scope="session")
def _adjacently_tagged_field(registry):
return desert._fields.AdjacentlyTaggedUnion(
from_object=registry.from_object, from_tag=registry.from_tag,
)


def test_adjacently_tagged_deserialize(example_data, adjacently_tagged_field):
serialized_value = {"type": example_data.tag, "value": example_data.object}

deserialized_value = adjacently_tagged_field.deserialize(serialized_value)

assert (type(deserialized_value) == type(example_data.object)) and (
deserialized_value == example_data.object
)


def test_adjacently_tagged_deserialize_extra_key_raises(
example_data, adjacently_tagged_field,
):
serialized_value = {
"type": example_data.tag,
"value": example_data.object,
"extra": 29,
}

with pytest.raises(expected_exception=Exception):
adjacently_tagged_field.deserialize(serialized_value)


def test_adjacently_tagged_serialize(example_data, adjacently_tagged_field):
obj = {"key": example_data.object}

serialized_value = adjacently_tagged_field.serialize("key", obj)

assert serialized_value == {"type": example_data.tag, "value": example_data.object}