Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add explainer replicas and handle scale to zero #3246

Merged
merged 4 commits into from
Jun 5, 2021
Merged

Add explainer replicas and handle scale to zero #3246

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
02479ff
Add explainer replicas and handle scale to zero
ukclivecox May 28, 2021
1da64ac
Add test for zero replicas
ukclivecox Jun 1, 2021
5a13a92
Merge branch 'master' into 3241_explainer_replicas
ukclivecox Jun 3, 2021
a109caa
rebase
ukclivecox Jun 4, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions ...rator/templates/customresourcedefinition_seldondeployments.machinelearning.seldon.io.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4016,6 +4016,9 @@ spec:
type: string
modelUri:
type: string
replicas:
format: int32
type: integer
serviceAccountName:
type: string
storageInitializerImage:
Expand Down
9 changes: 9 additions & 0 deletions ...or/templates/customresourcedefinition_v1_seldondeployments.machinelearning.seldon.io.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4879,6 +4879,9 @@ spec:
type: string
modelUri:
type: string
replicas:
format: int32
type: integer
serviceAccountName:
type: string
storageInitializerImage:
Expand Down Expand Up @@ -10671,6 +10674,9 @@ spec:
type: string
modelUri:
type: string
replicas:
format: int32
type: integer
serviceAccountName:
type: string
storageInitializerImage:
Expand Down Expand Up @@ -16463,6 +16469,9 @@ spec:
type: string
modelUri:
type: string
replicas:
format: int32
type: integer
serviceAccountName:
type: string
storageInitializerImage:
Expand Down
121 changes: 101 additions & 20 deletions notebooks/explainer_examples.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@
"name": "stdout",
"output_type": "stream",
"text": [
"Context \"kind-ansible\" modified.\r\n"
"Context \"gke_seldon-demos_europe-west2-a_deploy-test\" modified.\r\n"
]
}
],
Expand All @@ -129,14 +129,24 @@
"## Income Prediction Model with Anchors Explainer\n",
"The model and explainer used here can be trained yourself following the full example in the [Alibi Anchor Explanations for Income Notebook](https://docs.seldon.io/projects/alibi/en/latest/examples/anchor_tabular_adult.html) in the Alibi project documentation.\n",
"\n",
"This example also shows you can specify the replicas for your explainer.\n",
"\n",
"Note we used a python3.6 and version 0.5.2 of Alibi."
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 19,
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Overwriting resources/income_explainer.yaml\n"
]
}
],
"source": [
"%%writefile resources/income_explainer.yaml\n",
"apiVersion: machinelearning.seldon.io/v1\n",
Expand All @@ -156,44 +166,69 @@
" explainer:\n",
" type: AnchorTabular\n",
" modelUri: gs://seldon-models/sklearn/income/explainer-py36-0.5.2\n",
" replicas: 2\n",
" name: default\n",
" replicas: 1"
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 20,
"metadata": {
"scrolled": true
},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"seldondeployment.machinelearning.seldon.io/income configured\r\n"
]
}
],
"source": [
"!kubectl apply -f resources/income_explainer.yaml"
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 21,
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"deployment \"income-default-0-classifier\" successfully rolled out\r\n"
]
}
],
"source": [
"!kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=income -o jsonpath='{.items[0].metadata.name}')"
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 22,
"metadata": {
"scrolled": true
},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"deployment \"income-default-explainer\" successfully rolled out\r\n"
]
}
],
"source": [
"!kubectl rollout status deploy/income-default-explainer"
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 23,
"metadata": {
"scrolled": true
},
Expand All @@ -220,11 +255,19 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 24,
"metadata": {
"scrolled": true
},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'data': {'names': ['t:0', 't:1'], 'tensor': {'shape': [1, 2], 'values': [0.8585304277244477, 0.14146957227555243]}}, 'meta': {'requestPath': {'classifier': 'seldonio/sklearnserver:1.9.0-dev'}}}\n"
]
}
],
"source": [
"data = np.array([[39, 7, 1, 1, 1, 1, 4, 1, 2174, 0, 40, 9]])\n",
"r = sc.predict(data=data)\n",
Expand All @@ -240,9 +283,17 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 25,
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{\"data\":{\"names\":[\"t:0\",\"t:1\"],\"ndarray\":[[0.8585304277244477,0.14146957227555243]]},\"meta\":{\"requestPath\":{\"classifier\":\"seldonio/sklearnserver:1.9.0-dev\"}}}\r\n"
]
}
],
"source": [
"!curl -d '{\"data\": {\"ndarray\":[[39, 7, 1, 1, 1, 1, 4, 1, 2174, 0, 40, 9]]}}' \\\n",
" -X POST http://localhost:8003/seldon/seldon/income/api/v1.0/predictions \\\n",
Expand All @@ -258,11 +309,19 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 31,
"metadata": {
"scrolled": true
},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"['Marital Status = Never-Married', 'Hours per week <= 40.00']\n"
]
}
],
"source": [
"data = np.array([[39, 7, 1, 1, 1, 1, 4, 1, 2174, 0, 40, 9]])\n",
"explanation = sc.explain(deployment_name=\"income\", predictor=\"default\", data=data)\n",
Expand All @@ -278,9 +337,23 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 29,
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" % Total % Received % Xferd Average Speed Time Time Time Current\n",
" Dload Upload Total Spent Left Speed\n",
"100 2319 100 2228 100 91 15690 640 --:--:-- --:--:-- --:--:-- 15801\n",
"\u001b[1;39m[\n",
" \u001b[0;32m\"Marital Status = Separated\"\u001b[0m\u001b[1;39m,\n",
" \u001b[0;32m\"Capital Loss <= 0.00\"\u001b[0m\u001b[1;39m\n",
"\u001b[1;39m]\u001b[0m\n"
]
}
],
"source": [
"!curl -X POST -H 'Content-Type: application/json' \\\n",
" -d '{\"data\": {\"names\": [\"text\"], \"ndarray\": [[52, 4, 0, 2, 8, 4, 2, 0, 0, 0, 60, 9]]}}' \\\n",
Expand All @@ -289,11 +362,19 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 7,
"metadata": {
"scrolled": true
},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"seldondeployment.machinelearning.seldon.io \"income\" deleted\r\n"
]
}
],
"source": [
"!kubectl delete -f resources/income_explainer.yaml"
]
Expand Down
1 change: 1 addition & 0 deletions operator/apis/machinelearning.seldon.io/v1/seldondeployment_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -289,6 +289,7 @@ type Explainer struct {
Endpoint *Endpoint `json:"endpoint,omitempty" protobuf:"bytes,6,opt,name=endpoint"`
EnvSecretRefName string `json:"envSecretRefName,omitempty" protobuf:"bytes,7,opt,name=envSecretRefName"`
StorageInitializerImage string `json:"storageInitializerImage,omitempty" protobuf:"bytes,8,opt,name=storageInitializerImage"`
Replicas *int32 `json:"replicas,omitempty" protobuf:"string,4,opt,name=replicas"`
}

// ObjectMeta is a copy of the "k8s.io/apimachinery/pkg/apis/meta/v1" ObjectMeta.
Expand Down
5 changes: 5 additions & 0 deletions operator/apis/machinelearning.seldon.io/v1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions operator/config/crd/bases/machinelearning.seldon.io_seldondeployments.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8337,6 +8337,9 @@ spec:
type: string
modelUri:
type: string
replicas:
format: int32
type: integer
serviceAccountName:
type: string
storageInitializerImage:
Expand Down
9 changes: 9 additions & 0 deletions operator/config/crd_v1/bases/machinelearning.seldon.io_seldondeployments.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8578,6 +8578,9 @@ spec:
type: string
modelUri:
type: string
replicas:
format: int32
type: integer
serviceAccountName:
type: string
storageInitializerImage:
Expand Down Expand Up @@ -17452,6 +17455,9 @@ spec:
type: string
modelUri:
type: string
replicas:
format: int32
type: integer
serviceAccountName:
type: string
storageInitializerImage:
Expand Down Expand Up @@ -26326,6 +26332,9 @@ spec:
type: string
modelUri:
type: string
replicas:
format: int32
type: integer
serviceAccountName:
type: string
storageInitializerImage:
Expand Down
9 changes: 9 additions & 0 deletions operator/controllers/seldondeployment_explainers.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -204,6 +204,15 @@ func (ei *ExplainerInitialiser) createExplainer(mlDep *machinelearningv1.SeldonD

deploy := createDeploymentWithoutEngine(depName, seldonId, &seldonPodSpec, p, mlDep, podSecurityContect)

// Set replicas to zero if main predictor or graph has zero replicas otherwise set to explainer replicas
if p.Replicas != nil && *p.Replicas == 0 {
deploy.Spec.Replicas = p.Replicas
} else if p.Replicas == nil && mlDep.Spec.Replicas != nil && *mlDep.Spec.Replicas == 0 {
deploy.Spec.Replicas = mlDep.Spec.Replicas
} else {
deploy.Spec.Replicas = p.Explainer.Replicas
}

if p.Explainer.ModelUri != "" {
var err error

Expand Down
Loading