Skip to content

Commit

Permalink
remove sklearn example, fix some typos
Browse files Browse the repository at this point in the history
  • Loading branch information
ssiegel95 committed Aug 23, 2024
1 parent f707605 commit 1e29f3d
Showing 1 changed file with 21 additions and 62 deletions.
83 changes: 21 additions & 62 deletions services/inference/README.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,27 @@
# TSFM Services
# TSFM Inference Services

This component provides RESTful services for the tsfm-granite class of
timeseries foundation models. At present it can serve the following models:
The TSFM Inference Services component provides a runtime for inference related tasks for the tsfm-granite class of
timeseries foundation models. At present it provide inference endpoints for the following models:

* https://huggingface.co/ibm-granite/granite-timeseries-ttm-v1
* https://huggingface.co/ibm-granite/granite-timeseries-patchtst
* https://huggingface.co/ibm-granite/granite-timeseries-patchtsmixer

## Limitations

* At present the API includes only a forecasting endpoint. Other task-based endpoints such as regression and classification are in the works.
* The primary target environment is x86_64 Linux.
You may encounter hiccups if you try to use this on a different environment. If so, please file an issue. Many of our developers do use a Mac and run all tests not involving building containers locally so you're likely to find a quick resolution. None of our developers use native Windows, however, and we do not plan on supporting that environment. Windows users are advised to use Microsoft's excellent WSL2 implementation which provides a native Linux environment running under Windows without the overheads of virtualization.


## Prerequisites:

* GNU make
* python >=3.10, <3.13
* poetry (`pip install poetry`)
* zsh or bash

_Note that our primary target environment for services deployment is x86_64 Linux.
You may encounter hiccups if you try to use this on a different environment.
If so, please file an issue. Some of our developers do use a Mac so you're
likely to find a quick resolution. None of our developers use native Windows,
however._
* (optional) docker or podman
* (optional) kubectl if you plan on testing kubernetes-based deployments

## Known issues:

Expand All @@ -40,17 +42,21 @@ pip install poetry && poetry install --with dev
make test_local
```

### Creating an image
### Building an image

You must have either docker or podman installed on your system for this to
work. You must also have proper permissions on your system to build images. In this
README we assume you have a working docker command which can be docker itself
or `podman` that has been aliased twith dev
work. You must also have proper permissions on your system to build images. We assume you have a working docker command which can be docker itself
or `podman` that has been aliased as `docker` or has been installed with the podman-docker package that will do this for you.

```bash
make image
```

After a successful build you should have a local image named
`tsfminference:latest`

```sh
(py311) ➜ tsfm-services git:(revised-build-system) ✗ docker images | grep tsfminference | head -n 1
docker images | grep tsfminference | head -n 1
tsfminference latest df592dcb0533 46 seconds ago 1.49GB
# some of the numeric and hash values on your machine could be different
```
Expand Down Expand Up @@ -124,7 +130,7 @@ Have a nice day! 👋
configmap/local-registry-hosting created
```

### Upload our tsfm service image to the kind local registry:
### Upload the tsfm service image to the kind local registry:

```bash
# don't forget to run "make image" first
Expand Down Expand Up @@ -157,53 +163,6 @@ or more tries):
clusterstoragecontainer.serving.kserve.io/default created
😀 Successfully installed KServe
```

### Confirm that you can perform a sklearn inference run (optional)

This optional section is to help you confirm that your kserve setup is correct
to isolate any k8s issues from TSFM-specific ones. After following the steps up to
[here](https://kserve.github.io/website/0.13/modelserving/v1beta1/sklearn/v2/#test-the-deployed-model), you should be able to run the following inference request
(note that we're going to use port-forwarding instead of an ingress port for simplicity)

First check that the sklearn service is running (you have to apply the sklearn.yaml content given in [this section](https://kserve.github.io/website/0.13/modelserving/v1beta1/sklearn/v2/#deploy-the-model-with-rest-endpoint-through-inferenceservice).

```bash
kubectl get inferenceservices.serving.kserve.io sklearn-v2-iris
NAME URL READY PREV LATEST PREVROLLEDOUTREVISION LATESTREADYREVISION AGE
sklearn-v2-iris http://sklearn-v2-iris.default.example.com True 100 sklearn-v2-iris-predictor-00001 48s
```

Start a k8s port forward

```bash
INGRESS_GATEWAY_SERVICE=$(kubectl get svc --namespace istio-system --selector="app=istio-ingressgateway" --output jsonpath='{.items[0].metadata.name}')
kubectl port-forward --namespace istio-system svc/${INGRESS_GATEWAY_SERVICE} 8080:80
```

In a separate terminal, run an inference (don't forget to create the `iris-input-v2.json`
file using the payload content shown [here](iris-input-v2.json) ):

```bash
curl \
-H "Host: ${SERVICE_HOSTNAME}" \
-H "Content-Type: application/json" \
-d @./iris-input-v2.json \
http://localhost:8080/v2/models/sklearn-v2-iris/infer
```

You should get an output that resembles:

```json
{"model_name":"sklearn-v2-iris",
"model_version":null,"id":"1b479ed4-14ca-4b71-8a33-47a7d5c40134",
"parameters":null,
"outputs":[{"name":"output-0",
"shape":[2],
"datatype":"INT32",
"parameters":null,
"data":[1,1]}]}
```

### Deploy the tsfm kserve service

Save the folling yaml snippet to a file called tsfm.yaml.
Expand Down

0 comments on commit 1e29f3d

Please sign in to comment.