Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add instruction for install Feast on IKS and OpenShift using Kustomize #1534

Merged
merged 2 commits into from
May 18, 2021
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,4 +121,4 @@ See the deployment guide of the respective cloud providers for how to work with
- [Terraform Deployment on Amazon EKS](https://docs.feast.dev/feast-on-kubernetes/getting-started/install-feast/kubernetes-amazon-eks-with-terraform)
- [Terraform Deployment on Azure AKS](https://docs.feast.dev/feast-on-kubernetes/getting-started/install-feast/kubernetes-azure-aks-with-terraform)
- [Terraform Deployment on Google Cloud GKE](https://docs.feast.dev/feast-on-kubernetes/getting-started/install-feast/google-cloud-gke-with-terraform)
- [Helm Deployment on IBM Cloud IKS](https://docs.feast.dev/feast-on-kubernetes/getting-started/install-feast/ibm-cloud-iks-with-helm)
- [Kustomize Deployment on IBM Cloud IKS or OpenShift](https://docs.feast.dev/feast-on-kubernetes/getting-started/install-feast/ibm-cloud-iks-with-kustomize)
2 changes: 1 addition & 1 deletion docs/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@
* [Azure AKS \(with Helm\)](feast-on-kubernetes/getting-started/install-feast/kubernetes-azure-aks-with-helm.md)
* [Azure AKS \(with Terraform\)](feast-on-kubernetes/getting-started/install-feast/kubernetes-azure-aks-with-terraform.md)
* [Google Cloud GKE \(with Terraform\)](feast-on-kubernetes/getting-started/install-feast/google-cloud-gke-with-terraform.md)
* [IBM Cloud Kubernetes Service \(IKS\) \(with Helm\)](feast-on-kubernetes/getting-started/install-feast/ibm-cloud-iks-with-helm.md)
* [IBM Cloud Kubernetes Service \(IKS\) and Red Hat OpenShift \(with Kustomize\)](feast-on-kubernetes/getting-started/install-feast/ibm-cloud-iks-with-kustomize.md)
* [Connect to Feast](feast-on-kubernetes/getting-started/connect-to-feast/README.md)
* [Python SDK](feast-on-kubernetes/getting-started/connect-to-feast/python-sdk.md)
* [Feast CLI](feast-on-kubernetes/getting-started/connect-to-feast/feast-cli.md)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,9 @@ This guide installs Feast into a Google Cloud environment using Terraform. The T

{% page-ref page="google-cloud-gke-with-terraform.md" %}

## IBM Cloud Kubernetes Service \(IKS\) \(using Helm\)
## IBM Cloud Kubernetes Service \(IKS\) and Red Hat OpenShift \(using Kustomize\)

This guide installs Feast into an existing [IBM Cloud Kubernetes Service](https://www.ibm.com/cloud/kubernetes-service) using Helm.
This guide installs Feast into an existing [IBM Cloud Kubernetes Service](https://www.ibm.com/cloud/kubernetes-service) or [Red Hat OpenShift on IBM Cloud](https://www.ibm.com/cloud/openshift) using Kustomize.

{% page-ref page="ibm-cloud-iks-with-helm.md" %}
{% page-ref page="ibm-cloud-iks-with-kustomize.md" %}

Original file line number Diff line number Diff line change
@@ -1,23 +1,28 @@
# IBM Cloud Kubernetes Service \(IKS\) \(with Helm\)
# IBM Cloud Kubernetes Service and Red Hat OpenShift \(with Kustomize\)

## Overview

This guide installs Feast on an existing IBM Cloud Kubernetes cluster, and ensures the following services are running:
This guide installs Feast on an existing IBM Cloud Kubernetes cluster or Red Hat OpenShift on IBM Cloud , and ensures the following services are running:

* Feast Core
* Feast Online Serving
* Postgres
* Redis
* Kafka \(Optional\)
* Feast Jupyter \(Optional\)
* Prometheus \(Optional\)

## 1. Prerequisites

## 1. Requirements

1. [IBM Cloud Kubernetes Service](https://www.ibm.com/cloud/kubernetes-service)
2. Install [Kubectl](https://cloud.ibm.com/docs/containers?topic=containers-cs_cli_install#kubectl) that matches the major.minor versions of your IKS.
1. [IBM Cloud Kubernetes Service](https://www.ibm.com/cloud/kubernetes-service) or [Red Hat OpenShift on IBM Cloud](https://www.ibm.com/cloud/openshift)
2. Install [Kubectl](https://cloud.ibm.com/docs/containers?topic=containers-cs_cli_install#kubectl) that matches the major.minor versions of your IKS or Install the [OpenShift CLI](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift-cli#cli_oc) that matches your local operating system and OpenShift cluster version.
3. Install [Helm 3](https://helm.sh/)
4. Install [Kustomize](https://kubectl.docs.kubernetes.io/installation/kustomize/)

## 2. Preparation
### IBM Cloud Block Storage Setup (IKS only)

## 2. IBM Cloud Block Storage Setup
:warning: If you have Red Hat OpenShift Cluster on IBM Cloud skip to this [section](#Security-Context-Constraint-Setup).

By default, IBM Cloud Kubernetes cluster uses [IBM Cloud File Storage](https://www.ibm.com/cloud/file-storage) based on NFS as the default storage class, and non-root users do not have write permission on the volume mount path for NFS-backed storage. Some common container images in Feast, such as Redis, Postgres, and Kafka specify a non-root user to access the mount path in the images. When containers are deployed using these images, the containers fail to start due to insufficient permissions of the non-root user creating folders on the mount path.

Expand Down Expand Up @@ -79,39 +84,43 @@ Therefore, to deploy Feast we need to set up [IBM Cloud Block Storage](https://c
```text
ibmc-block-gold (default) ibm.io/ibmc-block 65s
```
### Security Context Constraint Setup
tedhtchang marked this conversation as resolved.
Show resolved Hide resolved

**3. Preparation**

Add the Feast Helm repository and download the latest charts:
By default, in OpenShift, all pods or containers will use the [Restricted SCC](https://docs.openshift.com/container-platform/4.6/authentication/managing-security-context-constraints.html) which limits the UIDs pods can run with, causing the Feast installation to fail. To overcome this, you can allow Feast pods to run with any UID by executing the following:

```text
helm repo add feast-charts https://feast-helm-charts.storage.googleapis.com
helm repo update
oc adm policy add-scc-to-user anyuid -z default,kf-feast-kafka -n feast
```
## 3. Installation

Feast includes a Helm chart that installs all necessary components to run Feast Core, Feast Online Serving, and an example Jupyter notebook.

Feast Core requires Postgres to run, which requires a secret to be set on Kubernetes:
Install Feast using kustomize. The pods may take a few minutes to initialize.

```bash
kubectl create secret generic feast-postgresql --from-literal=postgresql-password=password
git clone https://github.com/kubeflow/manifests
cd manifests/contrib/feast/
kustomize build feast/base | kubectl apply -n feast -f -
```
### Optional: Enable Feast Jupyter and Kafka

## 4. Installation

Install Feast using Helm. The pods may take a few minutes to initialize.
You may optionally enable the Feast Jupyter component which contains code examples to demonstrate Feast. Some examples require Kafka to stream real time features to the Feast online serving. To enable, edit the following properties in the `values.yaml` under the `manifests/contrib/feast` folder:
```
kafka.enabled: true
feast-jupyter.enabled: true
```

```bash
helm install feast-release feast-charts/feast
Then regenerate the resource manifests and deploy:
```
make feast/base
kustomize build feast/base | kubectl apply -n feast -f -
```

## 5. Use Jupyter to connect to Feast
## 4. Use Feast Jupyter to connect to Feast
tedhtchang marked this conversation as resolved.
Show resolved Hide resolved

After all the pods are in a `RUNNING` state, port-forward to the Jupyter Notebook Server in the cluster:

```bash
kubectl port-forward \
$(kubectl get pod -l app=feast-jupyter -o custom-columns=:metadata.name) 8888:8888
$(kubectl get pod -l app=feast-jupyter -o custom-columns=:metadata.name) 8888:8888 -n feast
```

```text
Expand All @@ -123,6 +132,10 @@ You can now connect to the bundled Jupyter Notebook Server at `localhost:8888` a

{% embed url="http://localhost:8888/tree?" caption="" %}

## 5. Uninstall Feast
```text
kustomize build feast/base | kubectl delete -n feast -f -
```
## 6. Troubleshooting

When running the minimal\_ride\_hailing\_example Jupyter Notebook example the following errors may occur:
Expand Down
7 changes: 3 additions & 4 deletions docs/getting-started/install-feast/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,8 @@ This guide installs Feast into an Azure environment using Terraform. The Terrafo

This guide installs Feast into a Google Cloud environment using Terraform. The Terraform script is opinionated and intended to allow you to start quickly.

## IBM Cloud Kubernetes Service \(IKS\) \(using Helm\)
## IBM Cloud Kubernetes Service \(IKS\) and Red Hat OpenShift \(using Kustomize\)

This guide installs Feast into an existing [IBM Cloud Kubernetes Service](https://www.ibm.com/cloud/kubernetes-service) using Helm.

{% page-ref page="ibm-cloud-iks-with-helm.md" %}
This guide installs Feast into an existing [IBM Cloud Kubernetes Service](https://www.ibm.com/cloud/kubernetes-service) or [Red Hat OpenShift on IBM Cloud](https://www.ibm.com/cloud/openshift) using Kustomize.

{% page-ref page="ibm-cloud-iks-with-kustomize.md" %}
Original file line number Diff line number Diff line change
@@ -1,23 +1,28 @@
# IBM Cloud Kubernetes Service \(IKS\) \(with Helm\)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this duplicate installation file? Is this needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one will be replaced by the ibm-cloud-iks-with-kustomize.md.

# IBM Cloud Kubernetes Service and Red Hat OpenShift \(with Kustomize\)

## Overview

This guide installs Feast on an existing IBM Cloud Kubernetes cluster, and ensures the following services are running:
This guide installs Feast on an existing IBM Cloud Kubernetes cluster or Red Hat OpenShift on IBM Cloud , and ensures the following services are running:

* Feast Core
* Feast Online Serving
* Postgres
* Redis
* Kafka \(Optional\)
* Feast Jupyter \(Optional\)
* Prometheus \(Optional\)

## 1. Prerequisites

## 1. Requirements

1. [IBM Cloud Kubernetes Service](https://www.ibm.com/cloud/kubernetes-service)
2. Install [Kubectl](https://cloud.ibm.com/docs/containers?topic=containers-cs_cli_install#kubectl) that matches the major.minor versions of your IKS.
1. [IBM Cloud Kubernetes Service](https://www.ibm.com/cloud/kubernetes-service) or [Red Hat OpenShift on IBM Cloud](https://www.ibm.com/cloud/openshift)
2. Install [Kubectl](https://cloud.ibm.com/docs/containers?topic=containers-cs_cli_install#kubectl) that matches the major.minor versions of your IKS or Install the [OpenShift CLI](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift-cli#cli_oc) that matches your local operating system and OpenShift cluster version.
3. Install [Helm 3](https://helm.sh/)
4. Install [Kustomize](https://kubectl.docs.kubernetes.io/installation/kustomize/)

## 2. Preparation
### IBM Cloud Block Storage Setup (IKS only)

## 2. IBM Cloud Block Storage Setup
:warning: If you have Red Hat OpenShift Cluster on IBM Cloud skip to this [section](#Security-Context-Constraint-Setup).

By default, IBM Cloud Kubernetes cluster uses [IBM Cloud File Storage](https://www.ibm.com/cloud/file-storage) based on NFS as the default storage class, and non-root users do not have write permission on the volume mount path for NFS-backed storage. Some common container images in Feast, such as Redis, Postgres, and Kafka specify a non-root user to access the mount path in the images. When containers are deployed using these images, the containers fail to start due to insufficient permissions of the non-root user creating folders on the mount path.

Expand Down Expand Up @@ -79,39 +84,43 @@ Therefore, to deploy Feast we need to set up [IBM Cloud Block Storage](https://c
```text
ibmc-block-gold (default) ibm.io/ibmc-block 65s
```
### Security Context Constraint Setup

**3. Preparation**

Add the Feast Helm repository and download the latest charts:
By default, in OpenShift, all pods or containers will use the [Restricted SCC](https://docs.openshift.com/container-platform/4.6/authentication/managing-security-context-constraints.html) which limits the UIDs pods can run with, causing the Feast installation to fail. To overcome this, you can allow Feast pods to run with any UID by executing the following:

```text
helm repo add feast-charts https://feast-helm-charts.storage.googleapis.com
helm repo update
oc adm policy add-scc-to-user anyuid -z default,kf-feast-kafka -n feast
```
## 3. Installation

Feast includes a Helm chart that installs all necessary components to run Feast Core, Feast Online Serving, and an example Jupyter notebook.

Feast Core requires Postgres to run, which requires a secret to be set on Kubernetes:
Install Feast using kustomize. The pods may take a few minutes to initialize.

```bash
kubectl create secret generic feast-postgresql --from-literal=postgresql-password=password
git clone https://github.com/kubeflow/manifests
cd manifests/contrib/feast/
kustomize build feast/base | kubectl apply -n feast -f -
```
### Optional: Enable Feast Jupyter and Kafka

## 4. Installation

Install Feast using Helm. The pods may take a few minutes to initialize.
You may optionally enable the Feast Jupyter component which contains code examples to demonstrate Feast. Some examples require Kafka to stream real time features to the Feast online serving. To enable, edit the following properties in the `values.yaml` under the `manifests/contrib/feast` folder:
```
kafka.enabled: true
feast-jupyter.enabled: true
```

```bash
helm install feast-release feast-charts/feast
Then regenerate the resource manifests and deploy:
```
make feast/base
kustomize build feast/base | kubectl apply -n feast -f -
```

## 5. Use Jupyter to connect to Feast
## 4. Use Feast Jupyter to connect to Feast

After all the pods are in a `RUNNING` state, port-forward to the Jupyter Notebook Server in the cluster:

```bash
kubectl port-forward \
$(kubectl get pod -l app=feast-jupyter -o custom-columns=:metadata.name) 8888:8888
$(kubectl get pod -l app=feast-jupyter -o custom-columns=:metadata.name) 8888:8888 -n feast
```

```text
Expand All @@ -123,6 +132,10 @@ You can now connect to the bundled Jupyter Notebook Server at `localhost:8888` a

{% embed url="http://localhost:8888/tree?" caption="" %}

## 5. Uninstall Feast
```text
kustomize build feast/base | kubectl delete -n feast -f -
```
## 6. Troubleshooting

When running the minimal\_ride\_hailing\_example Jupyter Notebook example the following errors may occur:
Expand Down