pyiceberg pydantic failed with new typing-extension package

[fritzb]

Apache Iceberg version

1.2.1 (latest release)

Query engine

None

Please describe the bug 🐞

I have typing-extensions 4.6.1, and this version caused pydantic to failed when importing BooleanExpression from pyiceberg. Downgrading to typing-extension 4.5.0 is the workaround

To reproduce:
from pyiceberg.expressions import BooleanExpression

Exception generated:

[2023-05-31, 20:10:31 UTC] {pod_manager.py:342} INFO - Traceback (most recent call last):
[2023-05-31, 20:10:31 UTC] {pod_manager.py:342} INFO -   File "/usr/local/bin/iceberg_converter", line 5, in <module>
[2023-05-31, 20:10:31 UTC] {pod_manager.py:342} INFO -     from tasks.iceberg_converter.app import main
[2023-05-31, 20:10:31 UTC] {pod_manager.py:342} INFO -   File "/usr/local/lib/python3.8/dist-packages/tasks/iceberg_converter/app.py", line 18, in <module>
[2023-05-31, 20:10:31 UTC] {pod_manager.py:342} INFO -     from pyiceberg.expressions import BooleanExpression
[2023-05-31, 20:10:31 UTC] {pod_manager.py:342} INFO -   File "/usr/local/lib/python3.8/dist-packages/pyiceberg/expressions/__init__.py", line 32, in <module>
[2023-05-31, 20:10:31 UTC] {pod_manager.py:342} INFO -     from pyiceberg.expressions.literals import (
[2023-05-31, 20:10:31 UTC] {pod_manager.py:342} INFO -   File "/usr/local/lib/python3.8/dist-packages/pyiceberg/expressions/literals.py", line 33, in <module>
[2023-05-31, 20:10:31 UTC] {pod_manager.py:342} INFO -     from pyiceberg.types import (
[2023-05-31, 20:10:31 UTC] {pod_manager.py:342} INFO -   File "/usr/local/lib/python3.8/dist-packages/pyiceberg/types.py", line 251, in <module>
[2023-05-31, 20:10:31 UTC] {pod_manager.py:342} INFO -     class StructType(IcebergType):
[2023-05-31, 20:10:31 UTC] {pod_manager.py:342} INFO -   File "pydantic/main.py", line 198, in pydantic.main.ModelMetaclass.__new__
[2023-05-31, 20:10:31 UTC] {pod_manager.py:342} INFO -   File "pydantic/fields.py", line 506, in pydantic.fields.ModelField.infer
[2023-05-31, 20:10:31 UTC] {pod_manager.py:342} INFO -   File "pydantic/fields.py", line 436, in pydantic.fields.ModelField.__init__
[2023-05-31, 20:10:31 UTC] {pod_manager.py:342} INFO -   File "pydantic/fields.py", line 552, in pydantic.fields.ModelField.prepare
[2023-05-31, 20:10:31 UTC] {pod_manager.py:342} INFO -   File "pydantic/fields.py", line 668, in pydantic.fields.ModelField._type_analysis
[2023-05-31, 20:10:31 UTC] {pod_manager.py:342} INFO -   File "/usr/lib/python3.8/typing.py", line 774, in __subclasscheck__
[2023-05-31, 20:10:31 UTC] {pod_manager.py:342} INFO -     return issubclass(cls, self.__origin__)
[2023-05-31, 20:10:31 UTC] {pod_manager.py:342} INFO - TypeError: issubclass() arg 1 must be a class
[2023-05-31, 20:10:31 UTC] {pod_manager.py:410} ERROR - Error parsing timestamp (no timestamp in message ''). Will continue execution but won't update timestamp
[2023-05-31, 20:10:31 UTC] {pod_manager.py:342} INFO - 

[Fokko]

Auch, it looks like some downstream dependency is updated.

➜  python git:(fd-bump-poetry) ✗ poetry show typing-extensions  
 name         : typing-extensions                                      
 version      : 4.5.0                                                  
 description  : Backported and Experimental Type Hints for Python 3.7+ 

required by
 - aioitertools >=4.0
 - azure-core >=4.3.0
 - azure-storage-blob >=4.0.1
 - pydantic >=4.2.0
 - rich >=4.0.0,<5.0

Can you check if the error persists with 4.6.2? That one has just been released. Otherwise, we'll just pin it on 4.5.0.

[Fokko]

I've created a PR that will at least bring visibility to our CI: #7747 Unfortunately this is something that sometimes happens in Python land. For PyIceberg typing-extensions is not a direct dependency, but we can pin it if the error persists. Thanks again @fritzb for reporting this.

[Fokko]

Looks like it is a bug in Pydantic itself: pydantic/pydantic#5826 Bumping to the latest version of Pydantic will fix the issue as well.