Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Clean up Rockset Online Store for use #3549

Merged
merged 1 commit into from
Mar 24, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/reference/online-stores/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ Please see [Online Store](../../getting-started/architecture-and-components/onli
[mysql.md](mysql.md)
{% endcontent-ref %}

{% content-ref url="mysql.md" %}
{% content-ref url="rockset.md" %}
[rockset.md](rockset.md)
{% endcontent-ref %}

56 changes: 51 additions & 5 deletions docs/reference/online-stores/rockset.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,12 @@

In Alpha Development.

The [Rockset](https://rockset.com/demo-signup/) online store provides support for materializing feature values within a Rockset collection for serving online features in real-time.
The [Rockset](https://rockset.com/demo-signup/) online store provides support for materializing feature values within a Rockset collection in order to serve features in real-time.

* Each document is uniquely identified by its '_id' value. Repeated inserts into the same document '_id' will result in an upsert.

Rockset indexes all columns allowing for quick per feature look up and also allows for a dynamic typed schema that can change based on any new requirements. ApiKeys can be found in the console
along with host urls which you can find in "View Region Endpoint Urls".
Rockset indexes all columns allowing for quick per feature look up and also allows for a dynamic typed schema that can change based on any new requirements. API Keys can be found in the Rockset console.
You can also find host urls on the same tab by clicking "View Region Endpoint Urls".

Data Model Used Per Doc

Expand All @@ -32,7 +32,53 @@ project: my_feature_app
registry: data/registry.db
provider: local
online_stores
## Basic Configs ##

# If apikey or host is left blank the driver will try to pull
# these values from environment variables ROCKSET_APIKEY and
# ROCKSET_APISERVER respectively.
type: rockset
apikey: MY_APIKEY_HERE
host: api.usw2a1.rockset.com
apikey: <your_api_key_here>
host: <your_region_endpoint_here>

## Advanced Configs ##

# Batch size of records that will be turned per page when
# paginating a batched read.
#
# read_pagination_batch_size: 100

# The amount of time, in seconds, we will wait for the
# collection to become visible to the API.
#
# collection_created_timeout_secs: 60

# The amount of time, in seconds, we will wait for the
# collection to enter READY state.
#
# collection_ready_timeout_secs: 1800

# Whether to wait for all writes to be flushed from log
# and queryable before returning write as completed. If
# False, documents that are written may not be seen
# immediately in subsequent reads.
#
# fence_all_writes: True

# The amount of time we will wait, in seconds, for the
# write fence to be passed
#
# fence_timeout_secs: 600

# Initial backoff, in seconds, we will wait between
# requests when polling for a response.
#
# initial_request_backoff_secs: 2

# Initial backoff, in seconds, we will wait between
# requests when polling for a response.
# max_request_backoff_secs: 30

# The max amount of times we will retry a failed request.
# max_request_attempts: 10000
```
12 changes: 11 additions & 1 deletion sdk/python/feast/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -631,7 +631,17 @@ def materialize_incremental_command(ctx: click.Context, end_ts: str, views: List
"--template",
"-t",
type=click.Choice(
["local", "gcp", "aws", "snowflake", "spark", "postgres", "hbase", "cassandra"],
[
"local",
"gcp",
"aws",
"snowflake",
"spark",
"postgres",
"hbase",
"cassandra",
"rockset",
],
case_sensitive=False,
),
help="Specify a template for the created project",
Expand Down
2 changes: 2 additions & 0 deletions sdk/python/feast/repo_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,8 @@ def __init__(self, **data: Any):
self._online_config = "datastore"
elif data["provider"] == "aws":
self._online_config = "dynamodb"
elif data["provider"] == "rockset":
self._online_config = "rockset"

self._batch_engine = None
if "batch_engine" in data:
Expand Down
6 changes: 4 additions & 2 deletions sdk/python/feast/templates/rockset/bootstrap.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,13 @@ def bootstrap():
data_path.mkdir(exist_ok=True)

rockset_apikey = click.prompt(
"Rockset Api Key (If blank will be read from ROCKSET_APIKEY in ENV):"
"Rockset Api Key (If blank will be read from ROCKSET_APIKEY in ENV):",
default="",
)

rockset_host = click.prompt(
"Rockset Host (If blank will be read from ROCKSET_APISERVER in ENV):"
"Rockset Host (If blank will be read from ROCKSET_APISERVER in ENV):",
default="",
)

replace_str_in_file(config_file, "ROCKSET_APIKEY", rockset_apikey)
Expand Down