Change service key to allow container services to always match correc…

…tly (#4043) * Change service key to allow container services to always match correctly * add combiner example * ad missing graph examples linj * recreate deployments and services whos selectors change * update licences
SeldonIO · Sep 6, 2022 · f5c3a29 · f5c3a29
1 parent 472d5dc
commit f5c3a29
Show file tree

Hide file tree

Showing 12 changed files with 500 additions and 33,925 deletions.
diff --git a/doc/source/examples/graph-examples.nblink b/doc/source/examples/graph-examples.nblink
@@ -0,0 +1,3 @@
+{
+  "path": "../../../examples/graph_examples/graph-examples.ipynb"
+}
diff --git a/doc/source/examples/notebooks.rst b/doc/source/examples/notebooks.rst
@@ -156,6 +156,7 @@ Complex Graph Examples
 
    Chainer MNIST <chainer_mnist>
    Custom pre-processors with the V2 Protocol <transformers-v2-protocol>
+   graph-examples <graph-examples>
 
 Ingress
 -------

diff --git a/doc/source/reference/upgrading.md b/doc/source/reference/upgrading.md
@@ -12,6 +12,10 @@ Make sure you also [read the CHANGELOG](./changelog.html) to see the detailed fe
 
 With this release of Seldon Core we upgrade Flask dependency to 2.x line. This includes Python wrapper and most pre-packaged model servers. In an unlikely scenario that this causes issues one can fallback to `1.14.1` images.
 
+## Services and Deployments Update
+
+The label on services pointing to containers for each node in your inference graph has been changed to stop clashes in key-values for label selectors when you have multiple nodes in your inference graph. This will cause all your existing seldon deployments to be recreated as their service and deployment specs will have changed in the upgrade. **This will cause some downtime in availability**.
+
 ## Upgrading to 1.14.1
 
 ### OpenShift default storage initializer

diff --git a/examples/graph_examples/combiner.yaml b/examples/graph_examples/combiner.yaml
@@ -0,0 +1,34 @@
+apiVersion: machinelearning.seldon.io/v1
+kind: SeldonDeployment
+metadata:
+  name: test
+spec:
+  name: test
+  predictors:
+  - componentSpecs:
+    - spec:
+        containers:
+        - name: combiner
+          image: seldonio/metadata-generic-node:1.14.0-dev
+        - name: model-1
+          image: seldonio/metadata-generic-node:1.14.0-dev
+        - name: model-2
+          image: seldonio/metadata-generic-node:1.14.0-dev
+    graph:
+      children:
+      - children: []
+        name: model-1
+        type: MODEL
+        endpoint:
+          type: REST
+      - children: []
+        name: model-2
+        type: MODEL
+        endpoint:
+          type: REST
+      endpoint:
+        type: REST
+      name: combiner
+      type: COMBINER
+    name: combiner
+    replicas: 1
diff --git a/examples/graph_examples/graph-examples.ipynb b/examples/graph_examples/graph-examples.ipynb
@@ -0,0 +1,306 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "945f9db1",
+   "metadata": {},
+   "source": [
+    "# Graph Examples\n",
+    "\n",
+    "Port-forward to that ingress on localhost:8003 in a separate terminal either with:\n",
+    "\n",
+    "  * Ambassador: `kubectl port-forward $(kubectl get pods -n seldon -l app.kubernetes.io/name=ambassador -o jsonpath='{.items[0].metadata.name}') -n seldon 8003:8080`\n",
+    "  * Istio: `kubectl port-forward $(kubectl get pods -l istio=ingressgateway -n istio-system -o jsonpath='{.items[0].metadata.name}') -n istio-system 8003:8080`\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "0bcfe613",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!kubectl create namespace seldon"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "c6d2452b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!kubectl config set-context $(kubectl config current-context) --namespace=seldon"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "33e483c7",
+   "metadata": {},
+   "source": [
+    "## Input and output transformer with model in same pod"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "d6b9dca0",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!cat tin-model-tout.yaml"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "c6f39a99",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!kubectl create -f tin-model-tout.yaml"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "ba27098e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!kubectl wait --for condition=ready --timeout=300s sdep --all -n seldon"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "6f02a38f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!curl -s -d '{\"data\": {\"ndarray\":[[1.0, 2.0, 5.0]]}}' \\\n",
+    "   -X POST http://localhost:8003/seldon/seldon/test/api/v1.0/predictions \\\n",
+    "   -H \"Content-Type: application/json\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "a9a8aa88",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!kubectl delete -f tin-model-tout.yaml"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "5ab6d167",
+   "metadata": {},
+   "source": [
+    "## Input and output transformer with model in separate pods"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "d86ed5c7",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!cat tin-model-tout-sep-pods.yaml"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "f7c5178c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!kubectl create -f tin-model-tout-sep-pods.yaml"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "5a83a428",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!kubectl wait --for condition=ready --timeout=300s sdep --all -n seldon"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "2d8ba680",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!curl -s -d '{\"data\": {\"ndarray\":[[1.0, 2.0, 5.0]]}}' \\\n",
+    "   -X POST http://localhost:8003/seldon/seldon/test/api/v1.0/predictions \\\n",
+    "   -H \"Content-Type: application/json\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "7febb431",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!kubectl delete -f tin-model-tout-sep-pods.yaml"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a95de935",
+   "metadata": {},
+   "source": [
+    "## Input and output transformer with svcOrch in separate pod"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "f89eaa5d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!cat tin-model-tout-sep-svcorch.yaml"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "9b073c46",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!kubectl create -f tin-model-tout-sep-svcorch.yaml"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "ef70b350",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!kubectl wait --for condition=ready --timeout=300s sdep --all -n seldon"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "08892e09",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!curl -s -d '{\"data\": {\"ndarray\":[[1.0, 2.0, 5.0]]}}' \\\n",
+    "   -X POST http://localhost:8003/seldon/seldon/test/api/v1.0/predictions \\\n",
+    "   -H \"Content-Type: application/json\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "3497c521",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!kubectl delete -f tin-model-tout-sep-svcorch.yaml"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1ec22240",
+   "metadata": {},
+   "source": [
+    "## Combiner"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "5c08af72",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!cat combiner.yaml"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "dc5fdbab",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!kubectl create -f combiner.yaml"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "b4691f11",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!kubectl wait --for condition=ready --timeout=300s sdep --all -n seldon"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "3268c165",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!curl -s -d '{\"data\": {\"ndarray\":[[1.0, 2.0, 5.0]]}}' \\\n",
+    "   -X POST http://localhost:8003/seldon/seldon/test/api/v1.0/predictions \\\n",
+    "   -H \"Content-Type: application/json\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "5895393c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!kubectl delete -f combiner.yaml"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "7b12320a",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.10"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
-Original file line number
+Diff line change
@@ Expand Up / @@ -156,6 +156,7 @@ Complex Graph Examples @@
        Chainer MNIST <chainer_mnist>
        Custom pre-processors with the V2 Protocol <transformers-v2-protocol>
+       graph-examples <graph-examples>
     Ingress
     -------
@@ Expand Down @@