Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pickling generic annotations errors #317

Closed
valtron opened this issue Dec 20, 2019 · 6 comments
Closed

Pickling generic annotations errors #317

valtron opened this issue Dec 20, 2019 · 6 comments

Comments

@valtron
Copy link
Contributor

valtron commented Dec 20, 2019

import typing, cloudpickle
cloudpickle.dumps(typing.Optional[int])

gives:

_pickle.PicklingError: Can't pickle typing.Union[int, NoneType]: it's not the same object as typing.Union

While pickling annotations directly isn't common, this is a problem when trying to pickle a class with annotations whose __module__ == '__main__'. For example, this fails with the same error:

import typing, cloudpickle
class C:
	prop: typing.Optional[int]
cloudpickle.dumps(C)
@ogrisel
Copy link
Contributor

ogrisel commented Dec 20, 2019

On Python 3.7 with cloudpickle master:

>>> import typing, cloudpickle 
... cloudpickle.dumps(typing.Optional[int])                                            
b'\x80\x04\x95\x86\x00\x00\x00\x00\x00\x00\x00\x8c\t_operator\x94\x8c\x07getitem\x94\x93\x94\x8c\x06typing\x94\x8c\x05Union\x94\x93\x94\x8c\x08builtins\x94\x8c\x03int\x94\x93\x94\x8c\x17cloudpickle.cloudpickle\x94\x8c\r_builtin_type\x94\x93\x94\x8c\tClassType\x94\x85\x94R\x94N\x85\x94R\x94\x86\x94\x86\x94R\x94.'

Furthermore:

>>> from pickle import loads                                                           
>>> loads(cloudpickle.dumps(typing.Optional[int]))                                     
typing.Union[int, NoneType]

What version of Python and cloudpickle did you use?

@valtron
Copy link
Contributor Author

valtron commented Dec 21, 2019

Sorry, forgot to mention it's Python 3.6, cloudpickle 1.2.2.

@ogrisel
Copy link
Contributor

ogrisel commented Dec 21, 2019

Type annotations are not supported for Python 3.6:

https://github.com/cloudpipe/cloudpickle#note-about-function-annotations

However the contract would be to just drop them silently instead of raising an exception. Feel free to submit a PR to do so.

@valtron
Copy link
Contributor Author

valtron commented Dec 22, 2019

Just made #318 to better support pickling annotations. I'm not sure if there are any problems with the approach, but it seems to work.

@jrbourbeau
Copy link
Member

It seems #318 may have closed this issue. I'm able to reproduce on Python 3.6 with the latest dev version of cloudpickle:

In [1]: import typing, cloudpickle
   ...: cloudpickle.dumps(typing.Optional[int])
Out[1]: b'\x80\x04\x95z\x00\x00\x00\x00\x00\x00\x00\x8c\x17cloudpickle.cloudpickle\x94\x8c\x1e_create_parametrized_type_hint\x94\x93\x94\x8c\x06typing\x94\x8c\x05Union\x94\x93\x94\x8c\x08builtins\x94\x8c\x03int\x94\x93\x94h\x06\x8c\x04type\x94\x93\x94N\x85\x94R\x94\x86\x94\x86\x94R\x94.'

@valtron @ogrisel is it safe to close this issue?

@valtron
Copy link
Contributor Author

valtron commented Sep 25, 2021

@jrbourbeau Yep, forgot this issue existed. Thanks for the reminder.

@valtron valtron closed this as completed Sep 25, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants