GenConverter fails to structure when init=False attributes are present

[bushkov]

• cattrs version: 1.8.0
• Python version: 3.8.11
• Operating System: macOS 11.5.2

Description

I am trying to structure an attrs class that contains an attribute which shouldn't be initialized on instance creation. When I use Converter, the structuring works fine, but when I use GenConverter it fails.

What I Did

Here is a simple example that fails:

import attr
import cattr


c = cattr.GenConverter()

@attr.s
class A:
    x: int = attr.ib()
    y: int = attr.ib(init=False)


d = {"x": 0}

c.structure(d, A)

It fails with:

Traceback (most recent call last):
  File "example.py", line 15, in <module>
    c.structure(d, A)
  File "cattr/converters.py", line 294, in structure
    return self._structure_func.dispatch(cl)(obj, cl)
  File "<cattrs generated structure __main__.A>", line 4, in structure_A
KeyError: 'y'

On a related note it's not clear to me from the documentation in which situation one should prefer to use Converter and in which situation one should prefer GenConverter. In particular the following points are unclear:

GenConverter differs from the old cattr.Converter in the following ways:

• structuring and unstructuring of attrs classes is slower the first time, but faster every subsequent time
• structuring and unstructuring can be customized
• support for easy overriding collection unstructuring

By 'faster every subsequent time' does it mean that GenConverter is faster than Converter or does it mean that it's getting faster with every structure/unstructure, but not faster than Converter?

About 'structuring and unstructuring can be customized' and support for easy overriding collection unstructuring, it would be helpful to have some concrete examples showing what one can do with GenConverter which cannot be done with Converter.

[Tinche]

Hello.

init=False attributes are kinda weird in a serialization and deserialization context. I don't really know what the right approach would be when de/serializing them, so the code doesn't deal with them at all. The fact the Converter works and GenConverter doesn't is simply due to the differences in how they work, it's not intentional. For that reason, I would advise to not use these fields in classes you're de/serializing. If you still want to use them you need to specify the behavior yourself, and I can help with that (it would involve hook factories, with which you're familiar).

To answer your other questions. GenConverter will basically write an un/structuring function for you. So the first time it sees a class, this functions needs to be generated, and that's a little slow. But then this function gets cached and reused. Now, this function is essentially what you'd write yourself, so it's basically the fastest it can possibly be in pure Python.

Converter is basically a legacy component which I don't plan on deprecating since it has users. The difference is it doesn't generate these functions, but always uses a prewritten handler. The prewritten handler is significantly slower (how much depends on the number of fields, mostly). Generally we're still talking single or double digit microseconds, so you might not care.

For overriding collection unstructuring, the docs are here: https://cattrs.readthedocs.io/en/latest/unstructuring.html#customizing-collection-unstructuring

[Tinche]

Ok to close this?

[bushkov]

@Tinche yes, it's ok to close this one.

[artem30801]

Hello! What's the best workaround here? Using override(omit=True) doesn't seem to work

[Tinche]

@artem30801 omit=True not working is news to me. Feel free to show an example

[artem30801]

That's a minimal (I think) example of my problem, @Tinche:

import attr
import cattr
from cattr.gen import make_dict_structure_fn, override


@attr.define()
class A:
    name: str = attr.field(init=False)
    data: str = attr.field()


converter = cattr.GenConverter()


def structure(cls):
    overrides = {a.name: override(omit=True) for a in attr.fields(cls) if not a.init}
    return make_dict_structure_fn(cls, converter, **overrides)


converter.register_structure_hook_factory(attr.has, structure)

d = {"data": "value"}
converter.structure(d, A)

Traceback when I run the code:

Traceback (most recent call last):
  File "xxx\test.py", line 23, in <module>
    converter.structure(d, A)
  File "xxx\.venv\lib\site-packages\cattr\converters.py", line 294, in structure
    return self._structure_func.dispatch(cl)(obj, cl)
  File "<cattrs generated structure __main__.A>", line 3, in structure_A
KeyError: 'name'

I think that code shoud omit name field, though it doesn't seem to work

Edit:

converter.register_structure_hook(A, make_dict_structure_fn(A, converter, name=override(omit=False)))

Yields same results

[Tinche]

@artem30801 You're right, it doesn't work for structuring. It's even documented here: https://cattrs.readthedocs.io/en/latest/customizing.html#omit

That said, I don't remember why it doesn't work for structuring in the first place. I guess no one asked for it yet (and I misunderstood the OP). I can add it in the next couple of days.

[Tinche]

@artem30801 The master branch has support for structuring omit. Feel free to give it a test to see if it serves your needs

[artem30801]

Thank you very much!
Tested out master branch, it seems to work without any issues for my cases
Also allows you to have fields with recursive type hints now (excluded from init of course).

[otaj]

Hi @Tinche , would it be possible to release a new version with this fix released? Thanks!

[otaj]

I just noticed, that there already is a version of 1.9.0, just not yet released to PyPI. Can you please release it there? Thanks!

[Tinche]

1.9.0 hasn't been released yet at all. I can get around to it over the weekend.

[otaj]

Oh, right, it just has a version in pyproject.toml. If you could release it over the weekend (or just some minor version with patch for this included), that would be awesome! Thank you!

[Tinche]

@otaj Released!