feat: Add /write-to-online-store method to the python feature server

[tpvasconcelos]

This PR adds a /write-to-online-store HTTP POST method to the python feature server.

Example: using curl to push fresh feature vectors to the online store:

curl -X POST "http://localhost:6566/write-to-online-store" -d '{
    "feature_view_name": "driver_hourly_stats",
    "df": {
            "driver_id": [1001],
            "event_timestamp": ["2022-05-13 10:59:42"],
            "created": ["2022-05-13 10:59:42"],
            "conv_rate": [1.0],
            "acc_rate": [1.0],
            "avg_daily_trips": [1000]
    }
  }'

Check the table below for the output difference of the /get-online-features before and after the operation above:

$ curl -X POST "http://localhost:6566/get-online-features" -d '{
    "features": [
      "driver_hourly_stats:conv_rate",
      "driver_hourly_stats:acc_rate",
      "driver_hourly_stats:avg_daily_trips"
    ],
    "entities": {
      "driver_id": [1001]
    }
  }' | jq

Before:	After:
{ "metadata": { "feature_names": [ "driver_id", "avg_daily_trips", "conv_rate", "acc_rate" ] }, "results": [ { "values": [ 1001, 3, 0.15759889781475067, 0.5574429631233215 ], "statuses": [...], "event_timestamps": [ "1970-01-01T00:00:00Z", "2022-02-04T10:00:00Z", "2022-02-04T10:00:00Z", "2022-02-04T10:00:00Z" ] } ] }	{ "metadata": { "feature_names": [ "driver_id", "avg_daily_trips", "conv_rate", "acc_rate" ] }, "results": [ { "values": [ 1001, 1000, 1, 1 ], "statuses": [...], "event_timestamps": [ "1970-01-01T00:00:00Z", "2022-05-13T10:59:42Z", "2022-05-13T10:59:42Z", "2022-05-13T10:59:42Z" ] } ] }

Signed-off-by: Tomas Pereira de Vasconcelos [email protected]
@tpvasconcelos

[tpvasconcelos]

@adchia I used the same naming convention as in store.write_to_online_store() for the endpoint name and JSON fields. WDYT?

[codecov-commenter]

Codecov Report

Merging #2423 (a2c3cc9) into master (37971a4) will decrease coverage by 0.02%.
The diff coverage is 44.44%.

@@            Coverage Diff             @@
##           master    #2423      +/-   ##
==========================================
- Coverage   58.62%   58.60%   -0.03%     
==========================================
  Files         126      126              
  Lines       10850    10868      +18     
==========================================
+ Hits         6361     6369       +8     
- Misses       4489     4499      +10     

Flag	Coverage Δ
unittests	58.60% <44.44%> (-0.03%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
sdk/python/feast/feature_server.py	35.18% <44.44%> (+4.62%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 37971a4...a2c3cc9. Read the comment docs.

[adchia]

@adchia I used the same naming convention as in store.write_to_online_store() for the endpoint name and JSON fields. WDYT?

Hey! Can you actually use the store.push() method instead for this and rename accordingly? Reason why we wanted the name to be a little more general is we'll likely also want to support in the future pushing to the offline store too.

I'll add a warning on the existing write_to_online_store() method in favor of the push() method.

Also: you still have linter issues

[tpvasconcelos]

Hey! Can you actually use the store.push() method instead for this and rename accordingly? Reason why we wanted the name to be a little more general is we'll likely also want to support in the future pushing to the offline store too.

@adchia Will this method allow to only push to the online store? As I understand, the future goal of this method is to push to both the online and offline stores with a single call right? I would prefer still having an option to only push to the online store. If this is the case, should I just add the /push method instead of removing the /write-to-online-store one?

[adchia]

I think that's fine for now as long as both of those methods have an alpha warning attached to the methods.

I think the .push method should probably allow flexibility in pushing to offline or online or both. Today it only pushes to the online store for feature views that depend on a PushSource.

Down the road, one example way of allowing pushing to offline stores might be having a boolean flag that lets you configure pushing to the batch source. e.g.

click_event_source = PushSource(
    name="click_event_source",
    schema=[
       ...
    ],
    batch_source=BigQuerySource(
			table="my_project.my_Dataset.click_event_table"
		),
    push_to_batch=True # will now also create or append to the batch source.
)

And this way, you can have a stream based FeatureView depend on a Union[PushSource, KafkaSource, KinesisSource] etc.

[adchia]

/lgtm

[feast-ci-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: adchia, tpvasconcelos

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [adchia]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment