Skip to content

Commit

Permalink
perf: lazily import pyiceberg and unity catalog if available
Browse files Browse the repository at this point in the history
  • Loading branch information
Jay Chia committed Dec 13, 2024
1 parent 35ed63c commit 9b964b5
Showing 1 changed file with 19 additions and 12 deletions.
31 changes: 19 additions & 12 deletions daft/catalog/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,21 +45,12 @@

from daft.dataframe import DataFrame

_PYICEBERG_AVAILABLE = False
try:
from pyiceberg.catalog import Catalog as PyIcebergCatalog

_PYICEBERG_AVAILABLE = True
except ImportError:
pass
from typing import TYPE_CHECKING

_UNITY_AVAILABLE = False
try:
if TYPE_CHECKING:
from pyiceberg.catalog import Catalog as PyIcebergCatalog

Check warning on line 51 in daft/catalog/__init__.py

View check run for this annotation

Codecov / codecov/patch

daft/catalog/__init__.py#L51

Added line #L51 was not covered by tests
from daft.unity_catalog import UnityCatalog

_UNITY_AVAILABLE = True
except ImportError:
pass

__all__ = [
"read_table",
Expand Down Expand Up @@ -136,6 +127,22 @@ def register_python_catalog(catalog: PyIcebergCatalog | UnityCatalog, name: str
>>> daft.catalog.register_python_catalog(catalog, "my_daft_catalog")
"""
_PYICEBERG_AVAILABLE = False
try:
from pyiceberg.catalog import Catalog as PyIcebergCatalog

Check warning on line 132 in daft/catalog/__init__.py

View check run for this annotation

Codecov / codecov/patch

daft/catalog/__init__.py#L130-L132

Added lines #L130 - L132 were not covered by tests

_PYICEBERG_AVAILABLE = True
except ImportError:
pass

Check warning on line 136 in daft/catalog/__init__.py

View check run for this annotation

Codecov / codecov/patch

daft/catalog/__init__.py#L134-L136

Added lines #L134 - L136 were not covered by tests

_UNITY_AVAILABLE = False
try:
from daft.unity_catalog import UnityCatalog

Check warning on line 140 in daft/catalog/__init__.py

View check run for this annotation

Codecov / codecov/patch

daft/catalog/__init__.py#L138-L140

Added lines #L138 - L140 were not covered by tests

_UNITY_AVAILABLE = True
except ImportError:
pass

Check warning on line 144 in daft/catalog/__init__.py

View check run for this annotation

Codecov / codecov/patch

daft/catalog/__init__.py#L142-L144

Added lines #L142 - L144 were not covered by tests

python_catalog: PyIcebergCatalog
if _PYICEBERG_AVAILABLE and isinstance(catalog, PyIcebergCatalog):
from daft.catalog.pyiceberg import PyIcebergCatalogAdaptor
Expand Down

0 comments on commit 9b964b5

Please sign in to comment.