-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add feature and feature set labels, for metadata #536
Merged
feast-ci-bot
merged 25 commits into
feast-dev:master
from
imjuanleonard:python-sdk-for-labels
Apr 30, 2020
Merged
Changes from 1 commit
Commits
Show all changes
25 commits
Select commit
Hold shift + click to select a range
e2d731c
Add labels column to feature field
imjuanleonard 0cdb63b
Add labels to feature spec proto
imjuanleonard 6802e0a
Implement labels on feature model
imjuanleonard 60e4cc8
Implement list labels
imjuanleonard b597799
Implement set_label and remove_label from feature client
imjuanleonard 4e69b2e
Refactor ingest to only accept featureset and dtype
imjuanleonard 258560c
Add equality on labels field
imjuanleonard ced6609
Add tests for labels apply
imjuanleonard 1a06833
Add Optional type hint to labels
imjuanleonard 9e768a1
Add empty labels key validation
imjuanleonard 108de14
Add label when generating feature spec for specServiceTest to test eq…
imjuanleonard c6cf42c
corrected convention (push test)
b9fd9af
corrected review comments
f241a4f
Merge branch 'master' into python-sdk-for-labels
dd88b6c
corrected lint-python check
2f7c86a
corrected lint-python 2
30d63fa
back out python SDK changes
4b7ec66
Implemented labels on a feature set level
b3a6317
added empty keys validation
de1153e
corrected review comments (storing empty json for features if labels …
a756929
Updated the comment to match the logic
suwik 18edc12
added e2e tests for feature and feature set labels
5e10c94
Merge branch 'python-sdk-for-labels' of https://github.com/imjuanleon…
02197f8
moved e2e tests for feature and feature set labels
8d6d4f4
changed tests ordering
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,90 @@ | ||
import pytest | ||
import grpc | ||
import uuid | ||
|
||
import feast.core.CoreService_pb2 | ||
from feast import Feature | ||
from feast.core.CoreService_pb2_grpc import CoreServiceStub | ||
from feast.feature_set import FeatureSet | ||
from feast import ValueType | ||
|
||
PROJECT_NAME = 'batch_' + uuid.uuid4().hex.upper()[0:6] | ||
LAST_VERSION = 0 | ||
GRPC_CONNECTION_TIMEOUT = 3 | ||
LABEL_KEY = "my" | ||
LABEL_VALUE = "label" | ||
|
||
|
||
@pytest.fixture(scope="module") | ||
def core_url(pytestconfig): | ||
return pytestconfig.getoption("core_url") | ||
|
||
|
||
@pytest.fixture(scope="module") | ||
def core_service_stub(core_url): | ||
if core_url.endswith(":443"): | ||
core_channel = grpc.secure_channel( | ||
core_url, grpc.ssl_channel_credentials() | ||
) | ||
else: | ||
core_channel = grpc.insecure_channel(core_url) | ||
|
||
try: | ||
grpc.channel_ready_future(core_channel).result(timeout=GRPC_CONNECTION_TIMEOUT) | ||
except grpc.FutureTimeoutError: | ||
raise ConnectionError( | ||
f"Connection timed out while attempting to connect to Feast " | ||
f"Core gRPC server {core_url} " | ||
) | ||
core_service_stub = CoreServiceStub(core_channel) | ||
return core_service_stub | ||
|
||
|
||
def apply_feature_set(core_service_stub, feature_set_proto): | ||
try: | ||
apply_fs_response = core_service_stub.ApplyFeatureSet( | ||
feast.core.CoreService_pb2.ApplyFeatureSetRequest(feature_set=feature_set_proto), | ||
timeout=GRPC_CONNECTION_TIMEOUT, | ||
) # type: ApplyFeatureSetResponse | ||
except grpc.RpcError as e: | ||
raise grpc.RpcError(e.details()) | ||
return apply_fs_response.feature_set | ||
|
||
|
||
def get_feature_set(core_service_stub, name, project): | ||
try: | ||
get_feature_set_response = core_service_stub.GetFeatureSet( | ||
feast.core.CoreService_pb2.GetFeatureSetRequest( | ||
project=project, name=name.strip(), version=LAST_VERSION | ||
) | ||
) # type: GetFeatureSetResponse | ||
except grpc.RpcError as e: | ||
raise grpc.RpcError(e.details()) | ||
return get_feature_set_response.feature_set | ||
|
||
|
||
@pytest.mark.timeout(45) | ||
def test_feature_set_labels(core_service_stub): | ||
feature_set_name = "test_feature_set_labels" | ||
feature_set_proto = FeatureSet(feature_set_name, PROJECT_NAME).to_proto() | ||
feature_set_proto.spec.labels[LABEL_KEY] = LABEL_VALUE | ||
apply_feature_set(core_service_stub, feature_set_proto) | ||
|
||
retrieved_feature_set = get_feature_set(core_service_stub, feature_set_name, PROJECT_NAME) | ||
|
||
assert LABEL_KEY in retrieved_feature_set.spec.labels | ||
assert retrieved_feature_set.spec.labels[LABEL_KEY] == LABEL_VALUE | ||
|
||
|
||
def test_feature_labels(core_service_stub): | ||
feature_set_name = "test_feature_labels" | ||
feature_set_proto = FeatureSet(feature_set_name, PROJECT_NAME, features=[Feature("rating", ValueType.INT64)])\ | ||
.to_proto() | ||
feature_set_proto.spec.features[0].labels[LABEL_KEY] = LABEL_VALUE | ||
apply_feature_set(core_service_stub, feature_set_proto) | ||
|
||
retrieved_feature_set = get_feature_set(core_service_stub, feature_set_name, PROJECT_NAME) | ||
retrieved_feature = retrieved_feature_set.spec.features[0] | ||
|
||
assert LABEL_KEY in retrieved_feature.labels | ||
assert retrieved_feature.labels[LABEL_KEY] == LABEL_VALUE |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The tests look good, but possibly add a
TODO
to migrate this over to the python sdk once that has been updated?