diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 2c9597e380..0e821d202e 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -121,4 +121,4 @@ See the deployment guide of the respective cloud providers for how to work with - [Terraform Deployment on Amazon EKS](https://docs.feast.dev/feast-on-kubernetes/getting-started/install-feast/kubernetes-amazon-eks-with-terraform) - [Terraform Deployment on Azure AKS](https://docs.feast.dev/feast-on-kubernetes/getting-started/install-feast/kubernetes-azure-aks-with-terraform) - [Terraform Deployment on Google Cloud GKE](https://docs.feast.dev/feast-on-kubernetes/getting-started/install-feast/google-cloud-gke-with-terraform) -- [Helm Deployment on IBM Cloud IKS](https://docs.feast.dev/feast-on-kubernetes/getting-started/install-feast/ibm-cloud-iks-with-helm) +- [Kustomize Deployment on IBM Cloud IKS or OpenShift](https://docs.feast.dev/feast-on-kubernetes/getting-started/install-feast/ibm-cloud-iks-with-kustomize) diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md index 75501820ff..5344a2fae0 100644 --- a/docs/SUMMARY.md +++ b/docs/SUMMARY.md @@ -38,7 +38,7 @@ * [Azure AKS \(with Helm\)](feast-on-kubernetes/getting-started/install-feast/kubernetes-azure-aks-with-helm.md) * [Azure AKS \(with Terraform\)](feast-on-kubernetes/getting-started/install-feast/kubernetes-azure-aks-with-terraform.md) * [Google Cloud GKE \(with Terraform\)](feast-on-kubernetes/getting-started/install-feast/google-cloud-gke-with-terraform.md) - * [IBM Cloud Kubernetes Service \(IKS\) \(with Helm\)](feast-on-kubernetes/getting-started/install-feast/ibm-cloud-iks-with-helm.md) + * [IBM Cloud Kubernetes Service \(IKS\) and Red Hat OpenShift \(with Kustomize\)](feast-on-kubernetes/getting-started/install-feast/ibm-cloud-iks-with-kustomize.md) * [Connect to Feast](feast-on-kubernetes/getting-started/connect-to-feast/README.md) * [Python SDK](feast-on-kubernetes/getting-started/connect-to-feast/python-sdk.md) * [Feast CLI](feast-on-kubernetes/getting-started/connect-to-feast/feast-cli.md) diff --git a/docs/feast-on-kubernetes/getting-started/install-feast/README.md b/docs/feast-on-kubernetes/getting-started/install-feast/README.md index ffc9d868b5..0b77ab431a 100644 --- a/docs/feast-on-kubernetes/getting-started/install-feast/README.md +++ b/docs/feast-on-kubernetes/getting-started/install-feast/README.md @@ -32,9 +32,9 @@ This guide installs Feast into a Google Cloud environment using Terraform. The T {% page-ref page="google-cloud-gke-with-terraform.md" %} -## IBM Cloud Kubernetes Service \(IKS\) \(using Helm\) +## IBM Cloud Kubernetes Service \(IKS\) and Red Hat OpenShift \(using Kustomize\) -This guide installs Feast into an existing [IBM Cloud Kubernetes Service](https://www.ibm.com/cloud/kubernetes-service) using Helm. +This guide installs Feast into an existing [IBM Cloud Kubernetes Service](https://www.ibm.com/cloud/kubernetes-service) or [Red Hat OpenShift on IBM Cloud](https://www.ibm.com/cloud/openshift) using Kustomize. -{% page-ref page="ibm-cloud-iks-with-helm.md" %} +{% page-ref page="ibm-cloud-iks-with-kustomize.md" %} diff --git a/docs/getting-started/install-feast/ibm-cloud-iks-with-helm.md b/docs/feast-on-kubernetes/getting-started/install-feast/ibm-cloud-iks-with-kustomize.md similarity index 68% rename from docs/getting-started/install-feast/ibm-cloud-iks-with-helm.md rename to docs/feast-on-kubernetes/getting-started/install-feast/ibm-cloud-iks-with-kustomize.md index 6bcfc5ad5b..d6c86a40a7 100644 --- a/docs/getting-started/install-feast/ibm-cloud-iks-with-helm.md +++ b/docs/feast-on-kubernetes/getting-started/install-feast/ibm-cloud-iks-with-kustomize.md @@ -1,23 +1,28 @@ -# IBM Cloud Kubernetes Service \(IKS\) \(with Helm\) +# IBM Cloud Kubernetes Service and Red Hat OpenShift \(with Kustomize\) ## Overview -This guide installs Feast on an existing IBM Cloud Kubernetes cluster, and ensures the following services are running: +This guide installs Feast on an existing IBM Cloud Kubernetes cluster or Red Hat OpenShift on IBM Cloud , and ensures the following services are running: * Feast Core * Feast Online Serving * Postgres * Redis +* Kafka \(Optional\) * Feast Jupyter \(Optional\) * Prometheus \(Optional\) + +## 1. Prerequisites -## 1. Requirements - -1. [IBM Cloud Kubernetes Service](https://www.ibm.com/cloud/kubernetes-service) -2. Install [Kubectl](https://cloud.ibm.com/docs/containers?topic=containers-cs_cli_install#kubectl) that matches the major.minor versions of your IKS. +1. [IBM Cloud Kubernetes Service](https://www.ibm.com/cloud/kubernetes-service) or [Red Hat OpenShift on IBM Cloud](https://www.ibm.com/cloud/openshift) +2. Install [Kubectl](https://cloud.ibm.com/docs/containers?topic=containers-cs_cli_install#kubectl) that matches the major.minor versions of your IKS or Install the [OpenShift CLI](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift-cli#cli_oc) that matches your local operating system and OpenShift cluster version. 3. Install [Helm 3](https://helm.sh/) +4. Install [Kustomize](https://kubectl.docs.kubernetes.io/installation/kustomize/) + +## 2. Preparation +### IBM Cloud Block Storage Setup (IKS only) -## 2. IBM Cloud Block Storage Setup +:warning: If you have Red Hat OpenShift Cluster on IBM Cloud skip to this [section](#Security-Context-Constraint-Setup). By default, IBM Cloud Kubernetes cluster uses [IBM Cloud File Storage](https://www.ibm.com/cloud/file-storage) based on NFS as the default storage class, and non-root users do not have write permission on the volume mount path for NFS-backed storage. Some common container images in Feast, such as Redis, Postgres, and Kafka specify a non-root user to access the mount path in the images. When containers are deployed using these images, the containers fail to start due to insufficient permissions of the non-root user creating folders on the mount path. @@ -79,39 +84,43 @@ Therefore, to deploy Feast we need to set up [IBM Cloud Block Storage](https://c ```text ibmc-block-gold (default) ibm.io/ibmc-block 65s ``` +### Security Context Constraint Setup (OpenShift only) - **3. Preparation** - -Add the Feast Helm repository and download the latest charts: +By default, in OpenShift, all pods or containers will use the [Restricted SCC](https://docs.openshift.com/container-platform/4.6/authentication/managing-security-context-constraints.html) which limits the UIDs pods can run with, causing the Feast installation to fail. To overcome this, you can allow Feast pods to run with any UID by executing the following: ```text -helm repo add feast-charts https://feast-helm-charts.storage.googleapis.com -helm repo update +oc adm policy add-scc-to-user anyuid -z default,kf-feast-kafka -n feast ``` +## 3. Installation -Feast includes a Helm chart that installs all necessary components to run Feast Core, Feast Online Serving, and an example Jupyter notebook. - -Feast Core requires Postgres to run, which requires a secret to be set on Kubernetes: +Install Feast using kustomize. The pods may take a few minutes to initialize. ```bash -kubectl create secret generic feast-postgresql --from-literal=postgresql-password=password +git clone https://github.com/kubeflow/manifests +cd manifests/contrib/feast/ +kustomize build feast/base | kubectl apply -n feast -f - ``` +### Optional: Enable Feast Jupyter and Kafka -## 4. Installation - -Install Feast using Helm. The pods may take a few minutes to initialize. +You may optionally enable the Feast Jupyter component which contains code examples to demonstrate Feast. Some examples require Kafka to stream real time features to the Feast online serving. To enable, edit the following properties in the `values.yaml` under the `manifests/contrib/feast` folder: +``` +kafka.enabled: true +feast-jupyter.enabled: true +``` -```bash -helm install feast-release feast-charts/feast +Then regenerate the resource manifests and deploy: +``` +make feast/base +kustomize build feast/base | kubectl apply -n feast -f - ``` -## 5. Use Jupyter to connect to Feast +## 4. Use Feast Jupyter Notebook Server to connect to Feast After all the pods are in a `RUNNING` state, port-forward to the Jupyter Notebook Server in the cluster: ```bash kubectl port-forward \ -$(kubectl get pod -l app=feast-jupyter -o custom-columns=:metadata.name) 8888:8888 +$(kubectl get pod -l app=feast-jupyter -o custom-columns=:metadata.name) 8888:8888 -n feast ``` ```text @@ -123,6 +132,10 @@ You can now connect to the bundled Jupyter Notebook Server at `localhost:8888` a {% embed url="http://localhost:8888/tree?" caption="" %} +## 5. Uninstall Feast +```text +kustomize build feast/base | kubectl delete -n feast -f - +``` ## 6. Troubleshooting When running the minimal\_ride\_hailing\_example Jupyter Notebook example the following errors may occur: @@ -169,4 +182,3 @@ When running the minimal\_ride\_hailing\_example Jupyter Notebook example the fo ```text os.environ["DEMO_KAFKA_BROKERS"] = "feast-release-kafka:9092" ``` - diff --git a/docs/getting-started/install-feast/README.md b/docs/getting-started/install-feast/README.md index 2d7718d26d..6c1dd80134 100644 --- a/docs/getting-started/install-feast/README.md +++ b/docs/getting-started/install-feast/README.md @@ -26,9 +26,8 @@ This guide installs Feast into an Azure environment using Terraform. The Terrafo This guide installs Feast into a Google Cloud environment using Terraform. The Terraform script is opinionated and intended to allow you to start quickly. -## IBM Cloud Kubernetes Service \(IKS\) \(using Helm\) +## IBM Cloud Kubernetes Service \(IKS\) and Red Hat OpenShift \(using Kustomize\) -This guide installs Feast into an existing [IBM Cloud Kubernetes Service](https://www.ibm.com/cloud/kubernetes-service) using Helm. - -{% page-ref page="ibm-cloud-iks-with-helm.md" %} +This guide installs Feast into an existing [IBM Cloud Kubernetes Service](https://www.ibm.com/cloud/kubernetes-service) or [Red Hat OpenShift on IBM Cloud](https://www.ibm.com/cloud/openshift) using Kustomize. +{% page-ref page="ibm-cloud-iks-with-kustomize.md" %} diff --git a/docs/feast-on-kubernetes/getting-started/install-feast/ibm-cloud-iks-with-helm.md b/docs/getting-started/install-feast/ibm-cloud-iks-with-kustomize.md similarity index 68% rename from docs/feast-on-kubernetes/getting-started/install-feast/ibm-cloud-iks-with-helm.md rename to docs/getting-started/install-feast/ibm-cloud-iks-with-kustomize.md index 6bcfc5ad5b..817d4dbe14 100644 --- a/docs/feast-on-kubernetes/getting-started/install-feast/ibm-cloud-iks-with-helm.md +++ b/docs/getting-started/install-feast/ibm-cloud-iks-with-kustomize.md @@ -1,23 +1,28 @@ -# IBM Cloud Kubernetes Service \(IKS\) \(with Helm\) +# IBM Cloud Kubernetes Service and Red Hat OpenShift \(with Kustomize\) ## Overview -This guide installs Feast on an existing IBM Cloud Kubernetes cluster, and ensures the following services are running: +This guide installs Feast on an existing IBM Cloud Kubernetes cluster or Red Hat OpenShift on IBM Cloud , and ensures the following services are running: * Feast Core * Feast Online Serving * Postgres * Redis +* Kafka \(Optional\) * Feast Jupyter \(Optional\) * Prometheus \(Optional\) + +## 1. Prerequisites -## 1. Requirements - -1. [IBM Cloud Kubernetes Service](https://www.ibm.com/cloud/kubernetes-service) -2. Install [Kubectl](https://cloud.ibm.com/docs/containers?topic=containers-cs_cli_install#kubectl) that matches the major.minor versions of your IKS. +1. [IBM Cloud Kubernetes Service](https://www.ibm.com/cloud/kubernetes-service) or [Red Hat OpenShift on IBM Cloud](https://www.ibm.com/cloud/openshift) +2. Install [Kubectl](https://cloud.ibm.com/docs/containers?topic=containers-cs_cli_install#kubectl) that matches the major.minor versions of your IKS or Install the [OpenShift CLI](https://cloud.ibm.com/docs/openshift?topic=openshift-openshift-cli#cli_oc) that matches your local operating system and OpenShift cluster version. 3. Install [Helm 3](https://helm.sh/) +4. Install [Kustomize](https://kubectl.docs.kubernetes.io/installation/kustomize/) + +## 2. Preparation +### IBM Cloud Block Storage Setup (IKS only) -## 2. IBM Cloud Block Storage Setup +:warning: If you have Red Hat OpenShift Cluster on IBM Cloud skip to this [section](#Security-Context-Constraint-Setup). By default, IBM Cloud Kubernetes cluster uses [IBM Cloud File Storage](https://www.ibm.com/cloud/file-storage) based on NFS as the default storage class, and non-root users do not have write permission on the volume mount path for NFS-backed storage. Some common container images in Feast, such as Redis, Postgres, and Kafka specify a non-root user to access the mount path in the images. When containers are deployed using these images, the containers fail to start due to insufficient permissions of the non-root user creating folders on the mount path. @@ -79,39 +84,43 @@ Therefore, to deploy Feast we need to set up [IBM Cloud Block Storage](https://c ```text ibmc-block-gold (default) ibm.io/ibmc-block 65s ``` +### Security Context Constraint Setup - **3. Preparation** - -Add the Feast Helm repository and download the latest charts: +By default, in OpenShift, all pods or containers will use the [Restricted SCC](https://docs.openshift.com/container-platform/4.6/authentication/managing-security-context-constraints.html) which limits the UIDs pods can run with, causing the Feast installation to fail. To overcome this, you can allow Feast pods to run with any UID by executing the following: ```text -helm repo add feast-charts https://feast-helm-charts.storage.googleapis.com -helm repo update +oc adm policy add-scc-to-user anyuid -z default,kf-feast-kafka -n feast ``` +## 3. Installation -Feast includes a Helm chart that installs all necessary components to run Feast Core, Feast Online Serving, and an example Jupyter notebook. - -Feast Core requires Postgres to run, which requires a secret to be set on Kubernetes: +Install Feast using kustomize. The pods may take a few minutes to initialize. ```bash -kubectl create secret generic feast-postgresql --from-literal=postgresql-password=password +git clone https://github.com/kubeflow/manifests +cd manifests/contrib/feast/ +kustomize build feast/base | kubectl apply -n feast -f - ``` +### Optional: Enable Feast Jupyter and Kafka -## 4. Installation - -Install Feast using Helm. The pods may take a few minutes to initialize. +You may optionally enable the Feast Jupyter component which contains code examples to demonstrate Feast. Some examples require Kafka to stream real time features to the Feast online serving. To enable, edit the following properties in the `values.yaml` under the `manifests/contrib/feast` folder: +``` +kafka.enabled: true +feast-jupyter.enabled: true +``` -```bash -helm install feast-release feast-charts/feast +Then regenerate the resource manifests and deploy: +``` +make feast/base +kustomize build feast/base | kubectl apply -n feast -f - ``` -## 5. Use Jupyter to connect to Feast +## 4. Use Feast Jupyter to connect to Feast After all the pods are in a `RUNNING` state, port-forward to the Jupyter Notebook Server in the cluster: ```bash kubectl port-forward \ -$(kubectl get pod -l app=feast-jupyter -o custom-columns=:metadata.name) 8888:8888 +$(kubectl get pod -l app=feast-jupyter -o custom-columns=:metadata.name) 8888:8888 -n feast ``` ```text @@ -123,6 +132,10 @@ You can now connect to the bundled Jupyter Notebook Server at `localhost:8888` a {% embed url="http://localhost:8888/tree?" caption="" %} +## 5. Uninstall Feast +```text +kustomize build feast/base | kubectl delete -n feast -f - +``` ## 6. Troubleshooting When running the minimal\_ride\_hailing\_example Jupyter Notebook example the following errors may occur: