Skip to content

Commit

Permalink
feat: Add to_remote_storage method to RetrievalJob
Browse files Browse the repository at this point in the history
Signed-off-by: Achal Shah <[email protected]>
  • Loading branch information
achals committed Jul 6, 2022
1 parent 38fd001 commit 1dddebc
Showing 1 changed file with 17 additions and 0 deletions.
17 changes: 17 additions & 0 deletions sdk/python/feast/infra/offline_stores/offline_store.py
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,23 @@ def metadata(self) -> Optional[RetrievalMetadata]:
"""
pass

def supports_remote_storage_export(self) -> bool:
"""
This method should return True if the RetrievalJob supports `to_remote_storage()`.
"""
return False

def to_remote_storage(self) -> List[str]:
"""
This method should export the result of this RetrievalJob to
remote storage (such as S3, GCS, HDFS, etc).
Implementations of this method should export the results as
multiple parquet files, each file sized appropriately
depending on how much data is being returned by the retrieval
job.
"""
raise NotImplementedError()


class OfflineStore(ABC):
"""
Expand Down

0 comments on commit 1dddebc

Please sign in to comment.