Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(taps): Change SQLStream.schema into a cached property #1745

Merged
merged 12 commits into from
Jun 5, 2023
Merged

refactor(taps): Change SQLStream.schema into a cached property #1745

Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
8dac344
Change schema into a cached_property
mjsqu Jun 2, 2023
d9d223e
Fix missing sys import
mjsqu Jun 2, 2023
c7f1e09
Changes for 3.7 compatibility
mjsqu Jun 2, 2023
2fb7a9c
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 2, 2023
7d92c49
Add parentheses to lru_cache for py 3.7
mjsqu Jun 2, 2023
e4cf099
Merge branch 'main' into feature/sql_schema_cached_property
edgarrmondragon Jun 5, 2023
40f9e66
Fix schema return type
mjsqu Jun 5, 2023
323a59b
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 5, 2023
d7831d6
Fix for mypy
mjsqu Jun 5, 2023
55312ca
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 5, 2023
7c7ee95
Re-add removed TYPE_CHECKING
mjsqu Jun 5, 2023
d3c8da5
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 5, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions singer_sdk/streams/sql.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ class SQLStream(Stream, metaclass=abc.ABCMeta):
"""Base class for SQLAlchemy-based streams."""

connector_class = SQLConnector
_cached_schema: dict | None = None

def __init__(
self,
Expand Down Expand Up @@ -74,7 +75,7 @@ def metadata(self) -> MetadataMapping:
"""
return self._singer_catalog_entry.metadata

@property
@property # TODO: Investigate @cached_property after py > 3.7
def schema(self) -> dict:
"""Return metadata object (dict) as specified in the Singer spec.

Expand All @@ -83,7 +84,13 @@ def schema(self) -> dict:
Returns:
The schema object.
"""
return t.cast(dict, self._singer_catalog_entry.schema.to_dict())
if not self._cached_schema:
self._cached_schema = t.cast(
dict,
self._singer_catalog_entry.schema.to_dict(),
)

return self._cached_schema

@property
def tap_stream_id(self) -> str:
Expand Down