-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add s3 support (with custom endpoints) #1789
Conversation
Signed-off-by: Willem Pienaar <[email protected]>
Signed-off-by: Willem Pienaar <[email protected]>
Signed-off-by: Willem Pienaar <[email protected]>
Codecov Report
@@ Coverage Diff @@
## master #1789 +/- ##
===========================================
- Coverage 85.37% 63.92% -21.45%
===========================================
Files 88 89 +1
Lines 6583 6700 +117
===========================================
- Hits 5620 4283 -1337
- Misses 963 2417 +1454
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
Signed-off-by: Willem Pienaar <[email protected]>
Signed-off-by: Willem Pienaar <[email protected]>
Signed-off-by: Willem Pienaar <[email protected]>
f: Any | ||
minio: DockerContainer | ||
bucket = "feast-test" | ||
access_key = "AKIAIOSFODNN7EXAMPLE" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can't really get around hardcoding access_key
and secret
in two places right now. The test suite doesn't allow us to inject configuration. cc @achals it would be nice if we could propagate a config (context-like) object.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed yeah I'll make a task for that
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/lgtm
return zip(schema.names, map(str, schema.types)) | ||
|
||
@staticmethod | ||
def prepare_path(path: str, s3_endpoint_override: str): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit, type annotations for the output of this method?
sdk/python/tests/integration/feature_repos/universal/data_sources/file.py
Show resolved
Hide resolved
f: Any | ||
minio: DockerContainer | ||
bucket = "feast-test" | ||
access_key = "AKIAIOSFODNN7EXAMPLE" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed yeah I'll make a task for that
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: achals, woop The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Signed-off-by: Willem Pienaar <[email protected]>
New changes are detected. LGTM label has been removed. |
Signed-off-by: Willem Pienaar <[email protected]>
What this PR does / why we need it:
Refactored #1549 to work with the new test suite. Also added support for testing S3 using Minio.
Added @qooba as co-author.
Which issue(s) this PR fixes:
Fixes #1548
Does this PR introduce a user-facing change?: