tgis-standalone/caikit-standalone manifests #186
Conversation
|
Skipping CI for Draft Pull Request. |
| } | ||
| .... | ||
| ``` | ||
| - http only. Perform inference with HTTP. This example uses cURL. |
There was a problem hiding this comment.
Apart from this line, everything in the diff block below below hasn't really changed, only the level of indentation has.
| - **[Text Generation Inference Server (TGIS)](https://github.com/opendatahub-io/text-generation-inference)**: Runtime that loads the models and provides the inference engine. | ||
| - **[KServe](https://github.com/opendatahub-io/kserve)**: A Kubernetes Custom Resource Definition that orchestrates model serving for all types of models. It includes serving runtimes that implement the loading of given types of model servers. KServe handles the lifecycle of the deployment object, storage access, and networking setup. | ||
| - **[Service Mesh](https://docs.openshift.com/container-platform/4.13/service_mesh/v2x/ossm-architecture.html)** (istio): The service mesh networking layer that manages traffic flows and enforces access policies. | ||
| - **[Serverless](https://docs.openshift.com/serverless/1.29/about/about-serverless.html)** (knative): A cloud-native development model that allows for serverless deployments of data models. | ||
|
|
||
| ## Architecture of the stack | ||
|
|
||
|  |
There was a problem hiding this comment.
@heyselbi I see you're the last person to have updated this diagram. Do you think we should update it once again to reflect the new architecture?
| name: caikit | ||
| containers: | ||
| - name: kserve-container | ||
| image: quay.io/opendatahub/caikit-nlp:stable |
There was a problem hiding this comment.
This image isn't available (yet). Waiting on openshift/release#46641 for the workflows required to push the image on quay and the creation of a release branch on https://github.com/opendatahub-io/caikit-nlp
dtrifiro
force-pushed
the
caikit-standalone-manifests
branch
from
37c3bc8
to
2061a7e
Compare
|
Overall lgtm, once openshift/release#46641 merges then will be good to merge |
This comment was marked as outdated.
This comment was marked as outdated.
dtrifiro
force-pushed
the
caikit-standalone-manifests
branch
from
2061a7e
to
10a10ee
Compare
dtrifiro
changed the title
| args: | ||
| - "--model-name=/mnt/models/" | ||
| args: | ||
| - "--model-name=/mnt/models/<model>" |
There was a problem hiding this comment.
I think we should remove <model>. I tried on my side, and it fails with it
dtrifiro
force-pushed
the
caikit-standalone-manifests
branch
2 times, most recently
from
db52ea8
to
0ec7d54
Compare
|
/retest |
dtrifiro
force-pushed
the
caikit-standalone-manifests
branch
from
0ec7d54
to
5b13980
Compare
|
/lgtm |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: dtrifiro, Xaenalt The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
demo/kserve/custom-manifests/caikit/caikit-standalone(cakit only)demo/kserve/custom-manifests/caikit/caikit-tgis(caikit frontend with tgis backend)demo/kserve/custom-manifests/tgis(tgis only)