Skip to content

Commit

Permalink
docs: Include docs updates for release process. Clean up old docs (#2957
Browse files Browse the repository at this point in the history 
)

* docs: Include docs updates for release process. Clean up old references in docs

Signed-off-by: Danny Chiao <[email protected]>

* clean up third integrations page

Signed-off-by: Danny Chiao <[email protected]>

* clean up lint

Signed-off-by: Danny Chiao <[email protected]>

* add clarification on credentials

Signed-off-by: Danny Chiao <[email protected]>
  • Loading branch information
@adchia
adchia committed Jul 21, 2022
1 parent 57ed370 commit 9e23787
Show file tree
Hide file tree
Showing 13 changed files with 60 additions and 188 deletions.
21 changes: 5 additions & 16 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,7 @@ The list below contains the functionality that contributors are planning to deve

* Items below that are in development (or planned for development) will be indicated in parentheses.
* We welcome contribution to all items in the roadmap!
* Want to speak to a Feast contributor? We are more than happy to jump on a call. Please schedule a time using [Calendly](https://calendly.com/d/x2ry-g5bb/meet-with-feast-team).
* Have questions about the roadmap? Go to the Slack channel to ask on #feast-development

* **Data Sources**
* [x] [Snowflake source](https://docs.feast.dev/reference/data-sources/snowflake)
Expand All @@ -151,7 +151,6 @@ The list below contains the functionality that contributors are planning to deve
* [x] [Postgres (contrib plugin)](https://docs.feast.dev/reference/data-sources/postgres)
* [x] [Spark (contrib plugin)](https://docs.feast.dev/reference/data-sources/spark)
* [x] Kafka / Kinesis sources (via [push support into the online store](https://docs.feast.dev/reference/data-sources/push))
* [ ] HTTP source
* **Offline Stores**
* [x] [Snowflake](https://docs.feast.dev/reference/offline-stores/snowflake)
* [x] [Redshift](https://docs.feast.dev/reference/offline-stores/redshift)
Expand Down Expand Up @@ -184,30 +183,20 @@ The list below contains the functionality that contributors are planning to deve
* **Deployments**
* [x] AWS Lambda (Alpha release. See [RFC](https://docs.google.com/document/d/1eZWKWzfBif66LDN32IajpaG-j82LSHCCOzY6R7Ax7MI/edit))
* [x] Kubernetes (See [guide](https://docs.feast.dev/how-to-guides/running-feast-in-production#4.3.-java-based-feature-server-deployed-on-kubernetes))
* [ ] Cloud Run
* [ ] KNative
* **Feature Serving**
* [x] Python Client
* [x] REST Feature Server (Python) (Alpha release. See [RFC](https://docs.google.com/document/d/1iXvFhAsJ5jgAhPOpTdB3j-Wj1S9x3Ev\_Wr6ZpnLzER4/edit))
* [x] gRPC Feature Server (Java) (See [#1497](https://github.com/feast-dev/feast/issues/1497))
* [x] Push API
* [ ] Java Client
* [ ] Go Client
* [ ] Delete API
* [ ] Feature Logging (for training)
* [x] REST Feature Server (Python) (See [RFC](https://docs.google.com/document/d/1iXvFhAsJ5jgAhPOpTdB3j-Wj1S9x3Ev\_Wr6ZpnLzER4/edit))
* [x] REST / gRPC Feature Server (Go) (Alpha release. See [docs](https://docs.feast.dev/reference/feature-servers/go-feature-retrieval)
* [x] gRPC Feature Server (Java) (Alpha release. See [#1497](https://github.com/feast-dev/feast/issues/1497))
* **Data Quality Management (See [RFC](https://docs.google.com/document/d/110F72d4NTv80p35wDSONxhhPBqWRwbZXG4f9mNEMd98/edit))**
* [x] Data profiling and validation (Great Expectations)
* [ ] Training-serving skew detection (in progress)
* [ ] Metric production
* [ ] Drift detection
* **Feature Discovery and Governance**
* [x] Python SDK for browsing feature registry
* [x] CLI for browsing feature registry
* [x] Model-centric feature tracking (feature services)
* [x] Amundsen integration (see [Feast extractor](https://github.com/amundsen-io/amundsen/blob/main/databuilder/databuilder/extractor/feast_extractor.py))
* [x] DataHub integration (see [DataHub Feast docs](https://datahubproject.io/docs/generated/ingestion/sources/feast/))
* [x] Feast Web UI (Alpha release. See [documentation](https://docs.feast.dev/reference/alpha-web-ui))
* [ ] REST API for browsing feature registry


## 🎓 Important Resources

Expand Down
7 changes: 1 addition & 6 deletions docs/community.md
View file
Original file line number Diff line number Diff line change
@@ -1,16 +1,11 @@
# Community

{% hint style="success" %}
**Speak to us:** Have a question, feature request, idea, or just looking to speak to a real person? Set up a meeting with a Feast maintainer over [here](https://calendly.com/d/x2ry-g5bb/meet-with-feast-team)!
{% endhint %}

## Links & Resources

* [Slack](https://slack.feast.dev): Feel free to ask questions or say hello!
* [Mailing list](https://groups.google.com/d/forum/feast-dev): We have both a user and developer mailing list.
* Feast users should join [[email protected]](mailto:[email protected]) group by clicking [here](https://groups.google.com/g/feast-discuss).
* Feast developers should join [[email protected]](mailto:[email protected]) group by clicking [here](https://groups.google.com/d/forum/feast-dev).
* People interested in the Feast community newsletter should join feast-announce by clicking [here](https://groups.google.com/d/forum/feast-announce).
* [Community Calendar](https://calendar.google.com/calendar/u/0?cid=ZTFsZHVhdGM3MDU3YTJucTBwMzNqNW5rajBAZ3JvdXAuY2FsZW5kYXIuZ29vZ2xlLmNvbQ): Includes community calls and design meetings.
* [Google Folder](https://drive.google.com/drive/u/0/folders/1jgMHOPDT2DvBlJeO9LCM79DP4lm4eOrR): This folder is used as a central repository for all Feast resources. For example:
* Design proposals in the form of Request for Comments (RFC).
Expand All @@ -27,7 +22,7 @@

## Community Calls

We have a user and contributor community call every two weeks (Asia & US friendly).
We have a user and contributor community call every two weeks (US & EU friendly).

{% hint style="info" %}
Please join the above Feast user groups in order to see calendar invites to the community calls
Expand Down
22 changes: 12 additions & 10 deletions docs/getting-started/feast-workshop.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -30,13 +30,15 @@ _See also:_ [_Feast quickstart_](https://docs.feast.dev/getting-started/quicksta

These are meant mostly to be done in order, with examples building on previous concepts.

| Time (min) | Description | Module |
| :--------: | ----------------------------------------------------------------------- | --------------------------------------------------------------------------- |
| 30-45 | Setting up Feast projects & CI/CD + powering batch predictions | [Module 0](https://github.com/feast-dev/feast-workshop/tree/main/module\_0) |
| 15-20 | Streaming ingestion & online feature retrieval with Kafka, Spark, Redis | [Module 1](https://github.com/feast-dev/feast-workshop/tree/main/module\_1) |
| 10-15 | Real-time feature engineering with on demand transformations | [Module 2](https://github.com/feast-dev/feast-workshop/tree/main/module\_2) |
| TBD | Feature server deployment (embed, as a service, AWS Lambda) | TBD |
| TBD | Versioning features / models in Feast | TBD |
| TBD | Data quality monitoring in Feast | TBD |
| TBD | Batch transformations | TBD |
| TBD | Stream transformations | TBD |
See https://github.com/feast-dev/feast-workshop

| Time (min) | Description | Module |
| :--------: | ----------------------------------------------------------------------- |-----------|
| 30-45 | Setting up Feast projects & CI/CD + powering batch predictions | Module 0 |
| 15-20 | Streaming ingestion & online feature retrieval with Kafka, Spark, Redis | Module 1 |
| 10-15 | Real-time feature engineering with on demand transformations | Module 2 |
| TBD | Feature server deployment (embed, as a service, AWS Lambda) | TBD |
| TBD | Versioning features / models in Feast | TBD |
| TBD | Data quality monitoring in Feast | TBD |
| TBD | Batch transformations | TBD |
| TBD | Stream transformations | TBD |
48 changes: 2 additions & 46 deletions docs/getting-started/third-party-integrations.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,55 +11,11 @@ Don't see your offline store or online store of choice here? Check out our guide

## Integrations

### **Data Sources**

* [x] [Snowflake source](https://docs.feast.dev/reference/data-sources/snowflake)
* [x] [Redshift source](https://docs.feast.dev/reference/data-sources/redshift)
* [x] [BigQuery source](https://docs.feast.dev/reference/data-sources/bigquery)
* [x] [Parquet file source](https://docs.feast.dev/reference/data-sources/file)
* [x] [Synapse source (community plugin)](https://github.com/Azure/feast-azure)
* [x] [Hive (community plugin)](https://github.com/baineng/feast-hive)
* [x] [Postgres (contrib plugin)](https://docs.feast.dev/reference/data-sources/postgres)
* [x] [Spark (contrib plugin)](https://docs.feast.dev/reference/data-sources/spark)
* [x] Kafka / Kinesis sources (via [push support into the online store](https://docs.feast.dev/reference/data-sources/push))
* [ ] HTTP source

### Offline Stores

* [x] [Snowflake](https://docs.feast.dev/reference/offline-stores/snowflake)
* [x] [Redshift](https://docs.feast.dev/reference/offline-stores/redshift)
* [x] [BigQuery](https://docs.feast.dev/reference/offline-stores/bigquery)
* [x] [Synapse (community plugin)](https://github.com/Azure/feast-azure)
* [x] [Hive (community plugin)](https://github.com/baineng/feast-hive)
* [x] [Postgres (contrib plugin)](https://docs.feast.dev/reference/offline-stores/postgres)
* [x] [Trino (contrib plugin)](https://github.com/Shopify/feast-trino)
* [x] [Spark (contrib plugin)](https://docs.feast.dev/reference/offline-stores/spark)
* [x] [In-memory / Pandas](https://docs.feast.dev/reference/offline-stores/file)
* [x] [Custom offline store support](https://docs.feast.dev/how-to-guides/adding-a-new-offline-store)

### Online Stores

* [x] [DynamoDB](https://docs.feast.dev/reference/online-stores/dynamodb)
* [x] [Redis](https://docs.feast.dev/reference/online-stores/redis)
* [x] [Datastore](https://docs.feast.dev/reference/online-stores/datastore)
* [x] [SQLite](https://docs.feast.dev/reference/online-stores/sqlite)
* [x] [Azure Cache for Redis (community plugin)](https://github.com/Azure/feast-azure)
* [x] [Postgres (contrib plugin)](https://docs.feast.dev/reference/online-stores/postgres)
* [x] [Custom online store support](https://docs.feast.dev/how-to-guides/adding-support-for-a-new-online-store)
* [x] [Cassandra / AstraDB](https://github.com/datastaxdevs/feast-cassandra-online-store)
* [ ] Bigtable (in progress)

### **Deployments**

* [x] AWS Lambda (Alpha release. See [guide](../reference/alpha-aws-lambda-feature-server.md) and [RFC](https://docs.google.com/document/d/1eZWKWzfBif66LDN32IajpaG-j82LSHCCOzY6R7Ax7MI/edit))
* [x] Kubernetes (See [guide](https://docs.feast.dev/how-to-guides/running-feast-in-production#4.3.-java-based-feature-server-deployed-on-kubernetes))
* [ ] Cloud Run
* [ ] KNative

See [Functionality and Roadmap](../../README.md#-functionality-and-roadmap)

## Standards

In order for a plugin integration to be highlighted on this page, it must meet the following requirements:
In order for a plugin integration to be highlighted, it must meet the following requirements:

1. The plugin must have tests. Ideally it would use the Feast universal tests (see this [guide](../how-to-guides/adding-or-reusing-tests.md) for an example), but custom tests are fine.
2. The plugin must have some basic documentation on how it should be used.
Expand Down
81 changes: 0 additions & 81 deletions docs/project/development-guide.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,85 +11,6 @@ This guide is targeted at developers looking to contribute to Feast:

> Learn How the Feast [Contributing Process](contributing.md) works.
## Project Structure

Feast is composed of [multiple components](../getting-started/architecture-and-components/) distributed into multiple repositories:

<table>
<thead>
<tr>
<th style="text-align:left">Repository</th>
<th style="text-align:left">Description</th>
<th style="text-align:left">Component(s)</th>
</tr>
</thead>
<tbody>
<tr>
<td style="text-align:left"><a href="https://github.com/feast-dev/feast">Main Feast Repository</a>
</td>
<td style="text-align:left">Hosts all required code to run Feast. This includes the Feast Python SDK
and Protobuf definitions. For legacy reasons this repository still contains
Terraform config and a Go Client for Feast.</td>
<td style="text-align:left">
<ul>
<li><b>Python SDK / CLI</b>
</li>
<li><b>Protobuf APIs</b>
</li>
<li><b>Documentation</b>
</li>
<li><b>Go Client</b>
</li>
<li><b>Terraform</b>
</li>
</ul>
</td>
</tr>
<tr>
<td style="text-align:left"><a href="https://github.com/feast-dev/feast-java">Feast Java</a>
</td>
<td style="text-align:left">Java-specific Feast components. Includes the Feast Core Registry, Feast
Serving for serving online feature values, and the Feast Java Client for
retrieving feature values.</td>
<td style="text-align:left">
<ul>
<li><b>Core</b>
</li>
<li><b>Serving</b>
</li>
<li><b>Java Client</b>
</li>
</ul>
</td>
</tr>
<tr>
<td style="text-align:left"><a href="https://github.com/feast-dev/feast-spark">Feast Spark</a>
</td>
<td style="text-align:left">Feast Spark SDK &amp; Feast Job Service for launching ingestion jobs and
for building training datasets with Spark</td>
<td style="text-align:left">
<ul>
<li><b>Spark SDK</b>
</li>
<li><b>Job Service</b>
</li>
</ul>
</td>
</tr>
<tr>
<td style="text-align:left"><a href="https://github.com/feast-dev/feast-helm-charts/">Feast Helm Chart</a>
</td>
<td style="text-align:left">Helm Chart for deploying Feast on Kubernetes &amp; Spark.</td>
<td style="text-align:left">
<ul>
<li><b>Helm Chart</b>
</li>
</ul>
</td>
</tr>
</tbody>
</table>

## Making a Pull Request

{% hint style="info" %}
Expand Down Expand Up @@ -148,5 +69,3 @@ The language specific bindings have to be regenerated when changes are made to t
| :--- | :--- | :--- |
| [Main Feast Repository](https://github.com/feast-dev/feast) | Python | Run `make compile-protos-python` to generate bindings |
| [Main Feast Repository](https://github.com/feast-dev/feast) | Golang | Run `make compile-protos-go` to generate bindings |
| [Feast Java](https://github.com/feast-dev/feast-java) | Java | No action required: bindings are generated automatically during compilation. |

Binary file added docs/project/new_branch_part_1.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/project/new_branch_part_2.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/project/new_branch_part_3.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/project/new_branch_part_4.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/project/new_branch_part_5.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
27 changes: 27 additions & 0 deletions docs/project/release-process.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -28,3 +28,30 @@ For Feast maintainers, these are the concrete steps for making a new release.
4. Try the dry run first with your personal access token. If this succeeds, uncheck `Dry Run` and run the release workflow.
5. All of the jobs should succeed besides the UI job which needs to be released separately. Ping a maintainer on Slack to run the UI release manually.
6. Try to install the feast release in your local environment and test out the `feast init` -> `feast apply` workflow to verify as a sanity check that the release worked correctly.

### (for minor releases) Post-release steps
1. Create a new branch based on master (i.e. v0.22-branch) and push to the main Feast repo. This will be where cherry-picks go for future patch releases and where documentation will point.
2. Write a summary of the release in the GitHub release
1. By default, Semantic Release will pull in messages from commits (features vs fixes, etc). But this is hard to digest still, so it helps to have a high level overview.

### Update documentation

In the Feast Gitbook (ask [Danny Chiao](https://tectonfeast.slack.com/team/U029405HFEU) in Slack for access):
1. Create a new space within the Feast collection
2. Go to the overflow menu on the top -> Synchronize with Git
1. Specify GitHub as the provider

![](new_branch_part_1.png)
2. Configure to point to the new release branch

![](new_branch_part_2.png)
3. Publish the new page for this branch as part of the collection

![](new_branch_part_3.png)
4. Go back to the main Feast collection and go to the overflow menu -> "Customize collection"

![](new_branch_part_3.png)
5. Configure the default space to be your new branch and save

![](new_branch_part_5.png)
6. Verify on docs.feast.dev that this new space is the default (this may take a few minutes to propagate, and your browser cache may be caching the old branch as the default)
20 changes: 7 additions & 13 deletions docs/project/versioning-policy.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,24 +23,18 @@ In general, unless you're committing code that only applies to a particular rele

The following table shows the **status** \(stable, beta, or alpha\) of Feast components.

Application status indicators for Feast:
Component status indicators for Feast:

* **Stable** means that the component has reached a sufficient level of stability and adoption that the Feast community has deemed the component stable. Please see the stability criteria below.
* **Beta** means that the component is working towards a version 1.0 release. Beta does not mean a component is unstable, it simply means the component has not met the full criteria of stability.
* **Alpha** means that the component is in the early phases of development and/or integration into Feast.

| Application | Status | Notes |
| :--- | :--- | :--- |
| [Feast Serving](https://github.com/feast-dev/feast-java) | Beta | APIs are considered stable and will not have breaking changes within 3 minor versions. |
| [Feast Core](https://github.com/feast-dev/feast-java) | Beta | At risk of deprecation |
| [Feast Java Client](https://github.com/feast-dev/feast-java) | Beta | |
| [Feast Python SDK](https://github.com/feast-dev/feast) | Beta | |
| [Feast Go Client](https://github.com/feast-dev/feast) | Beta | |
| [Feast Spark Python SDK](https://github.com/feast-dev/feast-spark) | Alpha | |
| [Feast Spark Launchers](https://github.com/feast-dev/feast-spark) | Alpha | |
| [Feast Job Service](https://github.com/feast-dev/feast-spark) | Alpha | Scheduled for deprecation |
| [Feast Helm Chart](https://github.com/feast-dev/feast-helm-charts) | Beta | |
| | | |
| Component | Status | Notes |
|:---------------------------------------------------------------------------------|:-------| :--- |
| [Feast Python SDK](https://github.com/feast-dev/feast/tree/master/sdk/python) | Stable | |
| [Feast Go Feature Server](https://github.com/feast-dev/feast/tree/master/) | Beta | |
| [Feast Java Feature Server](https://github.com/feast-dev/feast/tree/master/java) | Alpha | |
| | | |

Criteria for reaching _**stable**_ status:

Expand Down
Loading

0 comments on commit 9e23787

Please sign in to comment.