You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
With the release of version 0.6.0, pyiceberg is set to expand its capabilities in handling a greater variety of table properties, encompassing both read and write operations. (like #358#363 )
It would be beneficial to have a place to manage the names and default values of all table properties. Similar to Apache Iceberg Java's TableProperties and PropertyUtil classes.
We can create a new file named properties.py under pyiceberg.table to hold properties' name and default, and another file named property_util.py under pyiceberg.utils to parse properties into int, bool, float, or Optional[T].
This will ensure consistent parsing of properties across the codebase and simplify future updates or changes in property names and defaults.
The text was updated successfully, but these errors were encountered:
I think this is a great idea @HonahX. I try to avoid creating a lot of new files, since imports in Python are slow. What do you think of adding a class TableProperties to table/__init__.py. This way we can call it TableProperties.MAX_PARQUET_DICTIONARY_SIZE.
@Fokko Sounds good. My initial thought was to use a new file because table/init.py already has a substantial amount of content, and it's likely to grow as we develop write support. However, considering the importance of import speed, integrating it into table/__init__.py seems like the more efficient approach.
Feature Request / Improvement
With the release of version 0.6.0, pyiceberg is set to expand its capabilities in handling a greater variety of table properties, encompassing both read and write operations. (like #358 #363 )
It would be beneficial to have a place to manage the names and default values of all table properties. Similar to Apache Iceberg Java's TableProperties and PropertyUtil classes.
We can create a new file named
properties.py
underpyiceberg.table
to hold properties' name and default, and another file namedproperty_util.py
underpyiceberg.utils
to parse properties intoint
,bool
,float
, orOptional[T]
.This will ensure consistent parsing of properties across the codebase and simplify future updates or changes in property names and defaults.
The text was updated successfully, but these errors were encountered: