diff --git a/Makefile b/Makefile index c9c78a775f..a2d8bca5e4 100644 --- a/Makefile +++ b/Makefile @@ -363,6 +363,15 @@ build-feature-server-java-docker: -t $(REGISTRY)/feature-server-java:$(VERSION) \ -f java/infra/docker/feature-server/Dockerfile --load . +build-java-docker-dev: + make build-java-no-tests REVISION=dev + docker buildx build --build-arg VERSION=dev \ + -t feastdev/feature-transformation-server:dev \ + -f sdk/python/feast/infra/transformation_servers/Dockerfile --load . + docker buildx build --build-arg VERSION=dev \ + -t feastdev/feature-server-java:dev \ + -f java/infra/docker/feature-server/Dockerfile.dev --load . + # Documentation install-dependencies-proto-docs: diff --git a/examples/java-demo/README.md b/examples/java-demo/README.md index 2b1d7f75a5..ef4960fb4d 100644 --- a/examples/java-demo/README.md +++ b/examples/java-demo/README.md @@ -52,12 +52,12 @@ For this tutorial, we setup Feast with Redis, using the Feast CLI to register an ```bash minikube addons enable gcp-auth ``` -3. Add Feast's Java feature server chart repo +2. Add Feast's Java feature server chart repo ```bash helm repo add feast-charts https://feast-helm-charts.storage.googleapis.com helm repo update ``` -4. Modify the application-override.yaml file to have your credentials + bucket location: +3. Modify the application-override.yaml file to have your credentials + bucket location: ```yaml feature-server: application-override.yaml: @@ -77,16 +77,21 @@ For this tutorial, we setup Feast with Redis, using the Feast CLI to register an cache_ttl_seconds: 60 project: feast_java_demo ``` -5. Install the Feast helm chart: `helm install feast-release feast-charts/feast --values application-override.yaml` -6. (Optional): check logs of the server to make sure it’s working +4. Install the Feast helm chart: `helm install feast-release feast-charts/feast --values application-override.yaml` + > **Dev instructions**: if you're changing the java logic or chart, you can do + >1. `eval $(minikube docker-env)` + >2. `make build-java-docker-dev` + >3. In the `application-override.yaml`, uncomment the two `image: tag: dev` blocks + >4. `helm install feast-release ../../../infra/charts/feast --values application-override.yaml` +5. (Optional): check logs of the server to make sure it’s working ```bash kubectl logs svc/feast-release-feature-server ``` -7. Port forward to expose the grpc endpoint: +6. Port forward to expose the grpc endpoint: ```bash kubectl port-forward svc/feast-release-feature-server 6566:6566 ``` -8. Make a gRPC call: +7. Make a gRPC call: - Python example ```bash python test.py diff --git a/examples/java-demo/feature_repo/application-override.yaml b/examples/java-demo/feature_repo/application-override.yaml index 5a43d886dc..caaa5411e2 100644 --- a/examples/java-demo/feature_repo/application-override.yaml +++ b/examples/java-demo/feature_repo/application-override.yaml @@ -11,8 +11,17 @@ feature-server: port: 6379 password: [YOUR PASSWORD] entityKeySerializationVersion: 2 + # Uncomment below for dev +# image: +# tag: dev + +# Uncomment below for dev +#transformation-service: +# image: +# tag: dev + global: registry: - path: gs://[YOUR BUCKET]/demo-repo/registry.db + path: gs://[YOUR GCS BUCKET]/demo-repo/registry.db cache_ttl_seconds: 60 project: feast_java_demo diff --git a/examples/java-demo/feature_repo/feature_store.yaml b/examples/java-demo/feature_repo/feature_store.yaml index cfb0a89e8d..29fe3c97ce 100644 --- a/examples/java-demo/feature_repo/feature_store.yaml +++ b/examples/java-demo/feature_repo/feature_store.yaml @@ -1,4 +1,4 @@ -registry: gs://[YOUR BUCKET]/demo-repo/registry.db +registry: gs://[YOUR GCS BUCKET]/demo-repo/registry.db project: feast_java_demo provider: gcp online_store: diff --git a/infra/charts/feast-feature-server/README.md b/infra/charts/feast-feature-server/README.md index 6a811a9c11..c5c2b9d654 100644 --- a/infra/charts/feast-feature-server/README.md +++ b/infra/charts/feast-feature-server/README.md @@ -1,44 +1,9 @@ -# feast-feature-server - -![Version: 0.23.0](https://img.shields.io/badge/Version-0.23.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) - -Feast Feature Server in Go or Python - -**Homepage:** - -## Source Code - -* - -## Values - -| Key | Type | Default | Description | -| ---------------------------------- | ------ | ---------------- | ----------- | -| affinity | object | `{}` | | -| fullnameOverride | string | `""` | | -| image.pullPolicy | string | `"IfNotPresent"` | | -| image.repository | string | `""` | | -| image.tag | string | `""` | | -| imagePullSecrets | list | `[]` | | -| livenessProbe.initialDelaySeconds | int | `30` | | -| livenessProbe.periodSeconds | int | `30` | | -| nameOverride | string | `""` | | -| nodeSelector | object | `{}` | | -| podAnnotations | object | `{}` | | -| podSecurityContext | object | `{}` | | -| readinessProbe.initialDelaySeconds | int | `20` | | -| readinessProbe.periodSeconds | int | `10` | | -| replicaCount | int | `1` | | -| resources | object | `{}` | | -| securityContext | object | `{}` | | -| service.port | int | `80` | | -| service.type | string | `"ClusterIP"` | | -| tolerations | list | `[]` | | - ----------------------------------------------- -Autogenerated from chart metadata using [helm-docs v1.11.0](https://github.com/norwoodj/helm-docs/releases/v1.11.0) +# Feast Python / Go Feature Server Helm Charts +Current chart version is `0.23.0` +## Installation +### Python feature server Docker repository and tag are required. Helm install example: ``` helm install feast-feature-server . --set image.repository=REPO --set image.tag=TAG @@ -58,25 +23,51 @@ RUN pip install feast COPY feature_store.yaml /feature_store.yaml ``` +### Go feature server +> Warning: this is experimental, and only supports a local file registry + Redis + Furthermore, if you wish to use the Go feature server, then you must install the Apache Arrow C++ libraries, and your `feature_store.yaml` should include `go_feature_server: True`. For more details, see the [docs](https://docs.feast.dev/reference/feature-servers/go-feature-server). + The docker image might look like: ``` -FROM python:3.8 - -RUN apt update && \ - apt install -y jq - -RUN pip install pip --upgrade - -RUN pip install feast + FROM python:3.8 + + RUN apt update && \ + apt install -y jq + RUN pip install pip --upgrade + RUN pip install feast + RUN apt update + RUN apt install -y -V ca-certificates lsb-release wget + RUN wget https://apache.jfrog.io/artifactory/arrow/$(lsb_release --id --short | tr 'A-Z' 'a-z')/apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb + RUN apt install -y -V ./apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb + RUN apt update + RUN apt -y install libarrow-dev + + COPY feature_store.yaml /feature_store.yaml + ``` -RUN apt update -RUN apt install -y -V ca-certificates lsb-release wget -RUN wget https://apache.jfrog.io/artifactory/arrow/$(lsb_release --id --short | tr 'A-Z' 'a-z')/apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb -RUN apt install -y -V ./apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb -RUN apt update -RUN apt -y install libarrow-dev +## Values -COPY feature_store.yaml /feature_store.yaml -``` \ No newline at end of file +| Key | Type | Default | Description | +|-----|------|---------|-------------| +| affinity | object | `{}` | | +| fullnameOverride | string | `""` | | +| image.pullPolicy | string | `"IfNotPresent"` | | +| image.repository | string | `""` | [required] The repository for the Docker image | +| image.tag | string | `""` | [required] The Docker image tag | +| imagePullSecrets | list | `[]` | | +| livenessProbe.initialDelaySeconds | int | `30` | | +| livenessProbe.periodSeconds | int | `30` | | +| nameOverride | string | `""` | | +| nodeSelector | object | `{}` | | +| podAnnotations | object | `{}` | | +| podSecurityContext | object | `{}` | | +| readinessProbe.initialDelaySeconds | int | `20` | | +| readinessProbe.periodSeconds | int | `10` | | +| replicaCount | int | `1` | | +| resources | object | `{}` | | +| securityContext | object | `{}` | | +| service.port | int | `80` | | +| service.type | string | `"ClusterIP"` | | +| tolerations | list | `[]` | | \ No newline at end of file diff --git a/infra/charts/feast-feature-server/README.md.gotmpl b/infra/charts/feast-feature-server/README.md.gotmpl new file mode 100644 index 0000000000..79a681b3fc --- /dev/null +++ b/infra/charts/feast-feature-server/README.md.gotmpl @@ -0,0 +1,52 @@ +# Feast Python / Go Feature Server Helm Charts + +Current chart version is `{{ template "chart.version" . }}` + +## Installation +### Python feature server +Docker repository and tag are required. Helm install example: +``` +helm install feast-feature-server . --set image.repository=REPO --set image.tag=TAG +``` + +Deployment assumes that `feature_store.yaml` exists on docker image. Example docker image: +``` +FROM python:3.8 + +RUN apt update && \ + apt install -y jq + +RUN pip install pip --upgrade + +RUN pip install feast + +COPY feature_store.yaml /feature_store.yaml +``` + +### Go feature server +> Warning: this is experimental, and only supports a local file registry + Redis + +Furthermore, if you wish to use the Go feature server, then you must install the Apache Arrow C++ libraries, and your `feature_store.yaml` should include `go_feature_server: True`. +For more details, see the [docs](https://docs.feast.dev/reference/feature-servers/go-feature-server). + +The docker image might look like: +``` + FROM python:3.8 + + RUN apt update && \ + apt install -y jq + RUN pip install pip --upgrade + RUN pip install feast + RUN apt update + RUN apt install -y -V ca-certificates lsb-release wget + RUN wget https://apache.jfrog.io/artifactory/arrow/$(lsb_release --id --short | tr 'A-Z' 'a-z')/apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb + RUN apt install -y -V ./apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb + RUN apt update + RUN apt -y install libarrow-dev + + COPY feature_store.yaml /feature_store.yaml + ``` + +{{ template "chart.requirementsSection" . }} + +{{ template "chart.valuesSection" . }} \ No newline at end of file diff --git a/infra/charts/feast-feature-server/values.yaml b/infra/charts/feast-feature-server/values.yaml index f62f95a757..6d0ab9c0ae 100644 --- a/infra/charts/feast-feature-server/values.yaml +++ b/infra/charts/feast-feature-server/values.yaml @@ -5,8 +5,10 @@ replicaCount: 1 image: + # image.repository -- [required] The repository for the Docker image repository: "" pullPolicy: IfNotPresent + # image.tag -- [required] The Docker image tag tag: "" imagePullSecrets: [] diff --git a/infra/charts/feast-python-server/README.md b/infra/charts/feast-python-server/README.md index e3da9b1d29..2c8264c35c 100644 --- a/infra/charts/feast-python-server/README.md +++ b/infra/charts/feast-python-server/README.md @@ -1,14 +1,26 @@ -# feast-python-server +# Feast Python Feature Server Helm Charts -![Version: 0.23.0](https://img.shields.io/badge/Version-0.23.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) +Current chart version is `0.23.0` -Feast Feature Server in Python +## Installation +Docker repository and tag are required. Helm install example: +``` +helm install feast-python-server . --set image.repository=REPO --set image.tag=TAG +``` + +Deployment assumes that `feature_store.yaml` exists on docker image. Example docker image: +``` +FROM python:3.8 + +RUN apt update && \ + apt install -y jq -**Homepage:** +RUN pip install pip --upgrade -## Source Code +RUN pip install feast -* +COPY feature_store.yaml /feature_store.yaml +``` ## Values @@ -33,27 +45,4 @@ Feast Feature Server in Python | securityContext | object | `{}` | | | service.port | int | `80` | | | service.type | string | `"ClusterIP"` | | -| tolerations | list | `[]` | | - ----------------------------------------------- -Autogenerated from chart metadata using [helm-docs v1.5.0](https://github.com/norwoodj/helm-docs/releases/v1.5.0) - - -Docker repository and tag are required. Helm install example: -``` -helm install feast-python-server . --set image.repository=REPO --set image.tag=TAG -``` - -Deployment assumes that `feature_store.yaml` exists on docker image. Example docker image: -``` -FROM python:3.8 - -RUN apt update && \ - apt install -y jq - -RUN pip install pip --upgrade - -RUN pip install feast - -COPY feature_store.yaml /feature_store.yaml -``` \ No newline at end of file +| tolerations | list | `[]` | | \ No newline at end of file diff --git a/infra/charts/feast-python-server/README.md.gotmpl b/infra/charts/feast-python-server/README.md.gotmpl new file mode 100644 index 0000000000..b9c88d11b4 --- /dev/null +++ b/infra/charts/feast-python-server/README.md.gotmpl @@ -0,0 +1,27 @@ +# Feast Python Feature Server Helm Charts + +Current chart version is `{{ template "chart.version" . }}` + +## Installation +Docker repository and tag are required. Helm install example: +``` +helm install feast-python-server . --set image.repository=REPO --set image.tag=TAG +``` + +Deployment assumes that `feature_store.yaml` exists on docker image. Example docker image: +``` +FROM python:3.8 + +RUN apt update && \ + apt install -y jq + +RUN pip install pip --upgrade + +RUN pip install feast + +COPY feature_store.yaml /feature_store.yaml +``` + +{{ template "chart.requirementsSection" . }} + +{{ template "chart.valuesSection" . }} \ No newline at end of file diff --git a/infra/charts/feast-python-server/values.yaml b/infra/charts/feast-python-server/values.yaml index f62f95a757..6d0ab9c0ae 100644 --- a/infra/charts/feast-python-server/values.yaml +++ b/infra/charts/feast-python-server/values.yaml @@ -5,8 +5,10 @@ replicaCount: 1 image: + # image.repository -- [required] The repository for the Docker image repository: "" pullPolicy: IfNotPresent + # image.tag -- [required] The Docker image tag tag: "" imagePullSecrets: [] diff --git a/infra/charts/feast/README.md b/infra/charts/feast/README.md index f71dcf6124..905fb4b072 100644 --- a/infra/charts/feast/README.md +++ b/infra/charts/feast/README.md @@ -54,9 +54,9 @@ For more details, please see: https://docs.feast.dev/how-to-guides/running-feast | Repository | Name | Version | |------------|------|---------| -| https://charts.helm.sh/stable | redis | 10.5.6 | -| https://feast-helm-charts.storage.googleapis.com | feature-server(feature-server) | 0.23.0 | -| https://feast-helm-charts.storage.googleapis.com | transformation-service(transformation-service) | 0.23.0 | +| https://charts.helm.sh/stable | redis | 10.5.6 | +| https://feast-helm-charts.storage.googleapis.com | feature-server(feature-server) | 0.23.0 | +| https://feast-helm-charts.storage.googleapis.com | transformation-service(transformation-service) | 0.23.0 | ## Values diff --git a/infra/charts/feast/README.md.gotmpl b/infra/charts/feast/README.md.gotmpl index acb4e830e7..e215858fe0 100644 --- a/infra/charts/feast/README.md.gotmpl +++ b/infra/charts/feast/README.md.gotmpl @@ -1,7 +1,5 @@ # Feast Helm Charts -> :warning: **Disclaimer**: Since Feast 0.10 our vision is to manage all infrastructure for feature store from one place - Feast SDK. But while this new paradigm is still in development, we are planning to support the installation of some Feast components (like Java feature server) through Helm chart presented in this repository. However, we do not expect helm chart to become a long-term solution for deploying Feast components to production, and some frictions still might exist. For example, you will need to manually sync some configurations from [feature_store.yaml](https://docs.feast.dev/reference/feature-repository/feature-store-yaml) into the chart context (like path to the registry file or project name). - This repo contains Helm charts for Feast components that are being installed on Kubernetes: * Feast (root chart): The complete Helm chart containing all Feast components and dependencies. Most users will use this chart, but can selectively enable/disable subcharts using the values.yaml file. * [Feature Server](charts/feature-server): High performant JVM-based implementation of feature server. diff --git a/infra/charts/feast/charts/feature-server/README.md b/infra/charts/feast/charts/feature-server/README.md index 4717cfff3a..92ee102a07 100644 --- a/infra/charts/feast/charts/feature-server/README.md +++ b/infra/charts/feast/charts/feature-server/README.md @@ -8,63 +8,60 @@ Feast Feature Server: Online feature serving service for Feast ## Values -| Key | Type | Default | Description | -|-----|------|-------------------------------------------------------|-------------| -| "application-generated.yaml".enabled | bool | `true` | Flag to include Helm generated configuration. Please set `application-override.yaml` to override this configuration. | -| "application-override.yaml" | object | `{"enabled":true}` | Configuration to override the default [application.yaml](https://github.com/feast-dev/feast/blob/master/java/serving/src/main/resources/application.yml). Will be created as a ConfigMap. `application-override.yaml` has a higher precedence than `application-secret.yaml` | -| "application-secret.yaml" | object | `{"enabled":true}` | Configuration to override the default [application.yaml](https://github.com/feast-dev/feast/blob/master/java/serving/src/main/resources/application.yml). Will be created as a Secret. `application-override.yaml` has a higher precedence than `application-secret.yaml`. It is recommended to either set `application-override.yaml` or `application-secret.yaml` only to simplify config management. | -| "application.yaml".enabled | bool | `true` | Flag to include the default [configuration](https://github.com/feast-dev/feast/blob/master/java/serving/src/main/resources/application.yml). Please set `application-override.yaml` to override this configuration. | -| envOverrides | object | `{}` | Extra environment variables to set | -| image.pullPolicy | string | `"IfNotPresent"` | Image pull policy | -| image.repository | string | `"feastdev/feature-server-java"` | Docker image for Feature Server repository | -| image.tag | string | `"0.23.0"` | Image tag | -| ingress.grpc.annotations | object | `{}` | Extra annotations for the ingress | -| ingress.grpc.auth.enabled | bool | `false` | Flag to enable auth | -| ingress.grpc.class | string | `"nginx"` | Which ingress controller to use | -| ingress.grpc.enabled | bool | `false` | Flag to create an ingress resource for the service | -| ingress.grpc.hosts | list | `[]` | List of hostnames to match when routing requests | -| ingress.grpc.https.enabled | bool | `true` | Flag to enable HTTPS | -| ingress.grpc.https.secretNames | object | `{}` | Map of hostname to TLS secret name | -| ingress.grpc.whitelist | string | `""` | Allowed client IP source ranges | -| ingress.http.annotations | object | `{}` | Extra annotations for the ingress | +| Key | Type | Default | Description | +|-----|------|---------|-------------| +| "application-generated.yaml".enabled | bool | `true` | Flag to include Helm generated configuration. Please set `application-override.yaml` to override this configuration. | +| "application-override.yaml" | object | `{"enabled":true}` | Configuration to override the default [application.yaml](https://github.com/feast-dev/feast/blob/master/java/serving/src/main/resources/application.yml). Will be created as a ConfigMap. `application-override.yaml` has a higher precedence than `application-secret.yaml` | +| "application-secret.yaml" | object | `{"enabled":false}` | Configuration to override the default [application.yaml](https://github.com/feast-dev/feast/blob/master/java/serving/src/main/resources/application.yml). Will be created as a Secret. `application-override.yaml` has a higher precedence than `application-secret.yaml`. It is recommended to either set `application-override.yaml` or `application-secret.yaml` only to simplify config management. | +| "application.yaml".enabled | bool | `true` | Flag to include the default [configuration](https://github.com/feast-dev/feast/blob/master/java/serving/src/main/resources/application.yml). Please set `application-override.yaml` to override this configuration. | +| envOverrides | object | `{}` | Extra environment variables to set | +| image.pullPolicy | string | `"IfNotPresent"` | Image pull policy | +| image.repository | string | `"feastdev/feature-server-java"` | Docker image for Feature Server repository | +| image.tag | string | `"0.23.0"` | Image tag | +| ingress.grpc.annotations | object | `{}` | Extra annotations for the ingress | +| ingress.grpc.auth.enabled | bool | `false` | Flag to enable auth | +| ingress.grpc.class | string | `"nginx"` | Which ingress controller to use | +| ingress.grpc.enabled | bool | `false` | Flag to create an ingress resource for the service | +| ingress.grpc.hosts | list | `[]` | List of hostnames to match when routing requests | +| ingress.grpc.https.enabled | bool | `true` | Flag to enable HTTPS | +| ingress.grpc.https.secretNames | object | `{}` | Map of hostname to TLS secret name | +| ingress.grpc.whitelist | string | `""` | Allowed client IP source ranges | +| ingress.http.annotations | object | `{}` | Extra annotations for the ingress | | ingress.http.auth.authUrl | string | `"http://auth-server.auth-ns.svc.cluster.local/auth"` | URL to an existing authentication service | -| ingress.http.auth.enabled | bool | `false` | Flag to enable auth | -| ingress.http.class | string | `"nginx"` | Which ingress controller to use | -| ingress.http.enabled | bool | `false` | Flag to create an ingress resource for the service | -| ingress.http.hosts | list | `[]` | List of hostnames to match when routing requests | -| ingress.http.https.enabled | bool | `true` | Flag to enable HTTPS | -| ingress.http.https.secretNames | object | `{}` | Map of hostname to TLS secret name | -| ingress.http.whitelist | string | `""` | Allowed client IP source ranges | -| javaOpts | string | `nil` | [JVM options](https://docs.oracle.com/cd/E22289_01/html/821-1274/configuring-the-default-jvm-and-java-arguments.html). For better performance, it is advised to set the min and max heap:
`-Xms2048m -Xmx2048m` | -| livenessProbe.enabled | bool | `true` | Flag to enabled the probe | -| livenessProbe.failureThreshold | int | `5` | Min consecutive failures for the probe to be considered failed | -| livenessProbe.initialDelaySeconds | int | `60` | Delay before the probe is initiated | -| livenessProbe.periodSeconds | int | `10` | How often to perform the probe | -| livenessProbe.successThreshold | int | `1` | Min consecutive success for the probe to be considered successful | -| livenessProbe.timeoutSeconds | int | `5` | When the probe times out | -| logLevel | string | `"WARN"` | Default log level, use either one of `DEBUG`, `INFO`, `WARN` or `ERROR` | -| logType | string | `"Console"` | Log format, either `JSON` or `Console` | -| nodeSelector | object | `{}` | Node labels for pod assignment | -| podAnnotations | object | `{}` | Annotations to be added to Feast Serving pods | -| podLabels | object | `{}` | Labels to be added to Feast Serving pods | -| readinessProbe.enabled | bool | `true` | Flag to enabled the probe | -| readinessProbe.failureThreshold | int | `5` | Min consecutive failures for the probe to be considered failed | -| readinessProbe.initialDelaySeconds | int | `15` | Delay before the probe is initiated | -| readinessProbe.periodSeconds | int | `10` | How often to perform the probe | -| readinessProbe.successThreshold | int | `1` | Min consecutive success for the probe to be considered successful | -| readinessProbe.timeoutSeconds | int | `10` | When the probe times out | -| replicaCount | int | `1` | Number of pods that will be created | -| resources | object | `{}` | CPU/memory [resource requests/limit](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#resource-requests-and-limits-of-pod-and-container) | -| secrets | list | `[]` | List of Kubernetes secrets to be mounted. These secrets will be mounted on /etc/secrets/. | -| service.grpc.nodePort | string | `nil` | Port number that each cluster node will listen to | -| service.grpc.port | int | `6566` | Service port for GRPC requests | -| service.grpc.targetPort | int | `6566` | Container port serving GRPC requests | -| service.http.nodePort | string | `nil` | Port number that each cluster node will listen to | -| service.http.port | int | `80` | Service port for HTTP requests | -| service.http.targetPort | int | `8080` | Container port serving HTTP requests and Prometheus metrics | -| service.type | string | `"ClusterIP"` | Kubernetes service type | -| transformationService.host | string | `""` | | -| transformationService.port | int | `6566` | | +| ingress.http.auth.enabled | bool | `false` | Flag to enable auth | +| ingress.http.class | string | `"nginx"` | Which ingress controller to use | +| ingress.http.enabled | bool | `false` | Flag to create an ingress resource for the service | +| ingress.http.hosts | list | `[]` | List of hostnames to match when routing requests | +| ingress.http.https.enabled | bool | `true` | Flag to enable HTTPS | +| ingress.http.https.secretNames | object | `{}` | Map of hostname to TLS secret name | +| ingress.http.whitelist | string | `""` | Allowed client IP source ranges | +| javaOpts | string | `nil` | [JVM options](https://docs.oracle.com/cd/E22289_01/html/821-1274/configuring-the-default-jvm-and-java-arguments.html). For better performance, it is advised to set the min and max heap:
`-Xms2048m -Xmx2048m` | +| livenessProbe.enabled | bool | `true` | Flag to enabled the probe | +| livenessProbe.failureThreshold | int | `5` | Min consecutive failures for the probe to be considered failed | +| livenessProbe.initialDelaySeconds | int | `60` | Delay before the probe is initiated | +| livenessProbe.periodSeconds | int | `10` | How often to perform the probe | +| livenessProbe.successThreshold | int | `1` | Min consecutive success for the probe to be considered successful | +| livenessProbe.timeoutSeconds | int | `5` | When the probe times out | +| logLevel | string | `"WARN"` | Default log level, use either one of `DEBUG`, `INFO`, `WARN` or `ERROR` | +| logType | string | `"Console"` | Log format, either `JSON` or `Console` | +| nodeSelector | object | `{}` | Node labels for pod assignment | +| podAnnotations | object | `{}` | Annotations to be added to Feast Serving pods | +| podLabels | object | `{}` | Labels to be added to Feast Serving pods | +| readinessProbe.enabled | bool | `true` | Flag to enabled the probe | +| readinessProbe.failureThreshold | int | `5` | Min consecutive failures for the probe to be considered failed | +| readinessProbe.initialDelaySeconds | int | `15` | Delay before the probe is initiated | +| readinessProbe.periodSeconds | int | `10` | How often to perform the probe | +| readinessProbe.successThreshold | int | `1` | Min consecutive success for the probe to be considered successful | +| readinessProbe.timeoutSeconds | int | `10` | When the probe times out | +| replicaCount | int | `1` | Number of pods that will be created | +| resources | object | `{}` | CPU/memory [resource requests/limit](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#resource-requests-and-limits-of-pod-and-container) | +| secrets | list | `[]` | List of Kubernetes secrets to be mounted. These secrets will be mounted on /etc/secrets/. | +| service.grpc.nodePort | string | `nil` | Port number that each cluster node will listen to | +| service.grpc.port | int | `6566` | Service port for GRPC requests | +| service.grpc.targetPort | int | `6566` | Container port serving GRPC requests | +| service.type | string | `"ClusterIP"` | Kubernetes service type | +| transformationService.host | string | `""` | | +| transformationService.port | int | `6566` | | ---------------------------------------------- -Autogenerated from chart metadata using [helm-docs v1.5.0](https://github.com/norwoodj/helm-docs/releases/v1.5.0) +Autogenerated from chart metadata using [helm-docs v1.11.0](https://github.com/norwoodj/helm-docs/releases/v1.11.0) diff --git a/infra/charts/feast/charts/feature-server/templates/configmap.yaml b/infra/charts/feast/charts/feature-server/templates/configmap.yaml index fbf2633e8e..c172e9e288 100644 --- a/infra/charts/feast/charts/feature-server/templates/configmap.yaml +++ b/infra/charts/feast/charts/feature-server/templates/configmap.yaml @@ -28,9 +28,6 @@ data: config: host: {{ .Release.Name }}-redis-master port: 6379 - rest: - server: - port: {{ .Values.service.http.targetPort }} grpc: server: port: {{ .Values.service.grpc.targetPort }} diff --git a/infra/charts/feast/charts/feature-server/templates/deployment.yaml b/infra/charts/feast/charts/feature-server/templates/deployment.yaml index 1d1bc40029..ad0a12b3fc 100644 --- a/infra/charts/feast/charts/feature-server/templates/deployment.yaml +++ b/infra/charts/feast/charts/feature-server/templates/deployment.yaml @@ -106,8 +106,6 @@ spec: {{- end }} ports: - - name: http - containerPort: {{ .Values.service.http.targetPort }} - name: grpc containerPort: {{ .Values.service.grpc.targetPort }} diff --git a/infra/charts/feast/charts/feature-server/templates/service.yaml b/infra/charts/feast/charts/feature-server/templates/service.yaml index 037fe03870..c2455bd9f7 100644 --- a/infra/charts/feast/charts/feature-server/templates/service.yaml +++ b/infra/charts/feast/charts/feature-server/templates/service.yaml @@ -22,12 +22,6 @@ spec: {{ toYaml .Values.service.loadBalancerSourceRanges | indent 2 }} {{- end }} ports: - - name: http - port: {{ .Values.service.http.port }} - targetPort: {{ .Values.service.http.targetPort }} - {{- if .Values.service.http.nodePort }} - nodePort: {{ .Values.service.http.nodePort }} - {{- end }} - name: grpc port: {{ .Values.service.grpc.port }} targetPort: {{ .Values.service.grpc.targetPort }} diff --git a/infra/charts/feast/charts/feature-server/values.yaml b/infra/charts/feast/charts/feature-server/values.yaml index 011ce9dc33..eddec54cbe 100644 --- a/infra/charts/feast/charts/feature-server/values.yaml +++ b/infra/charts/feast/charts/feature-server/values.yaml @@ -71,13 +71,6 @@ readinessProbe: service: # service.type -- Kubernetes service type type: ClusterIP - http: - # service.http.port -- Service port for HTTP requests - port: 80 - # service.http.targetPort -- Container port serving HTTP requests and Prometheus metrics - targetPort: 8080 - # service.http.nodePort -- Port number that each cluster node will listen to - nodePort: grpc: # service.grpc.port -- Service port for GRPC requests port: 6566 diff --git a/infra/charts/feast/charts/transformation-service/README.md b/infra/charts/feast/charts/transformation-service/README.md index 9bc7a1e5d6..c7322dbef2 100644 --- a/infra/charts/feast/charts/transformation-service/README.md +++ b/infra/charts/feast/charts/transformation-service/README.md @@ -8,20 +8,21 @@ Transformation service: to compute on-demand features ## Values -| Key | Type | Default | Description | -|-----|------|--------------------------------------------|-------------| -| envOverrides | object | `{}` | Extra environment variables to set | -| image.pullPolicy | string | `"IfNotPresent"` | Image pull policy | +| Key | Type | Default | Description | +|-----|------|---------|-------------| +| envOverrides | object | `{}` | Extra environment variables to set | +| image.pullPolicy | string | `"IfNotPresent"` | Image pull policy | | image.repository | string | `"feastdev/feature-transformation-server"` | Docker image for Transformation Server repository | -| image.tag | string | `"0.23.0"` | Image tag | -| nodeSelector | object | `{}` | Node labels for pod assignment | -| podLabels | object | `{}` | Labels to be added to Feast Serving pods | -| replicaCount | int | `1` | Number of pods that will be created | -| resources | object | `{}` | CPU/memory [resource requests/limit](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#resource-requests-and-limits-of-pod-and-container) | -| service.grpc.nodePort | string | `nil` | Port number that each cluster node will listen to | -| service.grpc.port | int | `6566` | Service port for GRPC requests | -| service.grpc.targetPort | int | `6566` | Container port serving GRPC requests | -| service.type | string | `"ClusterIP"` | Kubernetes service type | +| image.tag | string | `"0.23.0"` | Image tag | +| nodeSelector | object | `{}` | Node labels for pod assignment | +| podLabels | object | `{}` | Labels to be added to Feast Serving pods | +| replicaCount | int | `1` | Number of pods that will be created | +| resources | object | `{}` | CPU/memory [resource requests/limit](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#resource-requests-and-limits-of-pod-and-container) | +| secrets | list | `[]` | List of Kubernetes secrets to be mounted. These secrets will be mounted on /etc/secrets/. | +| service.grpc.nodePort | string | `nil` | Port number that each cluster node will listen to | +| service.grpc.port | int | `6566` | Service port for GRPC requests | +| service.grpc.targetPort | int | `6566` | Container port serving GRPC requests | +| service.type | string | `"ClusterIP"` | Kubernetes service type | ---------------------------------------------- -Autogenerated from chart metadata using [helm-docs v1.5.0](https://github.com/norwoodj/helm-docs/releases/v1.5.0) +Autogenerated from chart metadata using [helm-docs v1.11.0](https://github.com/norwoodj/helm-docs/releases/v1.11.0) diff --git a/java/infra/docker/feature-server/Dockerfile.dev b/java/infra/docker/feature-server/Dockerfile.dev index 93bbbbb718..4eaec41ae3 100644 --- a/java/infra/docker/feature-server/Dockerfile.dev +++ b/java/infra/docker/feature-server/Dockerfile.dev @@ -7,7 +7,7 @@ ARG REVISION=dev RUN wget -q https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/v0.3.1/grpc_health_probe-linux-amd64 \ -O /usr/bin/grpc-health-probe && \ chmod +x /usr/bin/grpc-health-probe -ADD $PWD/serving/target/feast-serving-$REVISION-exec.jar /opt/feast/feast-serving.jar +ADD $PWD/java/serving/target/feast-serving-$REVISION-jar-with-dependencies.jar /opt/feast/feast-serving.jar CMD ["java",\ "-Xms1024m",\ "-Xmx1024m",\ diff --git a/java/serving/README.md b/java/serving/README.md index 0a153ceab8..dc23702d0f 100644 --- a/java/serving/README.md +++ b/java/serving/README.md @@ -3,14 +3,18 @@ ### Overview This guide is targeted at developers looking to contribute to Feast Serving: - [Building and running Feast Serving locally](#building-and-running-feast-serving-locally) +- [Unit / Integration Tests](#unit-/-integration-tests) +- [Developing against Feast Helm charts](#developing-against-feast-helm-charts) -### Pre-requisites: +### Building and running Feast Serving locally: + +#### Pre-requisites - [Maven](https://maven.apache.org/install.html) build tool version 3.6.x - A Feast feature repo (e.g. https://github.com/feast-dev/feast-demo) - A running Store instance e.g. local Redis instance with `redis-server` -### Building and running Feast Serving locally: +#### Steps From the Feast GitHub root, run: 1. `mvn -f java/pom.xml install -Dmaven.test.skip=true` @@ -46,7 +50,7 @@ From the Feast GitHub root, run: ``` 5. Now you have a Feast Serving gRPC service running on port 6566 locally! -### Running test queries +#### Running test queries If you have [grpc_cli](https://github.com/grpc/grpc/blob/master/doc/command_line_tool.md) installed, you can check that Feast Serving is running ``` grpc_cli ls localhost:6566 @@ -116,7 +120,7 @@ results { Rpc succeeded with OK status ``` -### Debugging Feast Serving +#### Debugging Feast Serving You can debug this like any other Java executable. Swap the java command above with: ``` java \ @@ -138,4 +142,8 @@ mvn test -pl serving --also-make mvn verify -pl serving --also-make # run integration tests with debugger mvn -Dmaven.failsafe.debug verify -pl serving --also-make -``` \ No newline at end of file +``` + +### Developing against Feast Helm charts +Look at [java-demo](../../examples/java-demo) for steps on how to update the helm chart or java logic and test their +interactions. \ No newline at end of file diff --git a/java/serving/src/main/resources/application.yml b/java/serving/src/main/resources/application.yml index b0c3bf7636..62ace5018e 100644 --- a/java/serving/src/main/resources/application.yml +++ b/java/serving/src/main/resources/application.yml @@ -40,10 +40,3 @@ grpc: # It is set default to 6566 so it does not conflict with the GRPC server on Feast Core # which defaults to port 6565 port: 6566 - -rest: - server: - # The port number on which the Tomcat webserver that serves REST API endpoints should listen - # It is set by default to 8081 so it does not conflict with Tomcat webserver on Feast Core - # if both Feast Core and Serving are running on the same machine - port: 8081 diff --git a/sdk/python/feast/infra/transformation_servers/Dockerfile b/sdk/python/feast/infra/transformation_servers/Dockerfile index 5e77144757..c072ed0160 100644 --- a/sdk/python/feast/infra/transformation_servers/Dockerfile +++ b/sdk/python/feast/infra/transformation_servers/Dockerfile @@ -15,7 +15,7 @@ COPY README.md README.md # Install dependencies -RUN --mount=source=.git,target=.git,type=bind pip3 install --no-cache-dir -e '.' +RUN --mount=source=.git,target=.git,type=bind pip3 install --no-cache-dir -e '.[gcp,aws]' # Start feature transformation server CMD [ "python", "app.py" ]