Merge pull request #689 from cliveseldon/standalone_servers

Model servers

doc/source/examples/notebooks.rst

@@ -41,6 +41,7 @@ Notebooks
    SKLearn Iris Classifier <iris>
    SKLearn MNIST <sklearn_mnist>
    SKLearn SpaCy Reddit Text Classifier Example <sklearn_spacy_text_classifier_example>
+   Sklearn, Xgboost, Tensorflow Server Examples <server_examples>
    Tensorflow MNIST <deep_mnist>
    TFserving MNIST <tfserving_mnist>
 

doc/source/examples/server_examples.nblink

@@ -0,0 +1,3 @@
+{
+  "path": "../../../notebooks/server_examples.ipynb"
+}

doc/source/index.rst

@@ -36,6 +36,15 @@ Seldon Core is an open source platform for deploying machine learning models on
    Usage reporting <workflow/usage-reporting.md>
    Upgrading <reference/upgrading.md>
 
+.. toctree::
+   :maxdepth: 1
+   :caption: Servers
+
+   Inference Servers Overview <servers/overview.md>
+   SKLearn server <servers/sklearn.md>
+   XGBoost server <servers/xgboost.md>
+   Tensorflow Serving <servers/tensorflow.md>
+
 .. toctree::
    :maxdepth: 1
    :caption: Wrappers

doc/source/python/api/seldon_core.rst

@@ -80,6 +80,14 @@ seldon\_core.seldon\_methods module
     :undoc-members:
     :show-inheritance:
 
+seldon\_core.storage module
+---------------------------
+
+.. automodule:: seldon_core.storage
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
 seldon\_core.user\_model module
 -------------------------------
 

doc/source/servers/overview.md

@@ -0,0 +1,68 @@
+# Prepackaged Model Servers
+
+Seldon provides several prepacked servers you can use to deploy trained models:
+
+ * [SKLearn Server](./sklearn.html)
+ * [XGBoost Server](xgboost.html)
+ * [Tensorflow Serving](tensorflow.html)
+
+
+For these servers you only need the location of the saved model in a local filestore, Google bucket or S3 bucket. An example manifest with an sklearn server is shown below:
+
+```
+apiVersion: machinelearning.seldon.io/v1alpha2
+kind: SeldonDeployment
+metadata:
+  name: sklearn
+spec:
+  name: iris
+  predictors:
+  - graph:
+      children: []
+      implementation: SKLEARN_SERVER
+      modelUri: gs://seldon-models/sklearn/iris
+      name: classifier
+    name: default
+    replicas: 1
+```
+
+The `modelUri` specifies the bucket containing the saved model, in this case `gs://seldon-models/sklearn/iris`.
+
+If you want to customize the resources for the server you can add a skeleton `Container` with the same name to your podSpecs, e.g.
+
+```
+apiVersion: machinelearning.seldon.io/v1alpha2
+kind: SeldonDeployment
+metadata:
+  name: sklearn
+spec:
+  name: iris
+  predictors:
+  - componentSpecs:
+    - spec:
+        containers:
+        - name: classifier
+          resources:
+            requests:
+              memory: 50Mi
+    graph:
+      children: []
+      implementation: SKLEARN_SERVER
+      modelUri: gs://seldon-models/sklearn/iris
+      name: classifier
+    name: default
+    replicas: 1
+
+```
+
+The image name and other details will be added when this is deployed automatically.
+
+Next steps:
+
+   * [Worked notebook](../examples/server_examples.html)
+   * [SKLearn Server](./sklearn.html)
+   * [XGBoost Server](xgboost.html)
+   * [Tensorflow Serving](tensorflow.html)
+
+If your use case does not fall into the above standard servers then you can create your own component using our wrappers.
+

doc/source/servers/sklearn.md

@@ -0,0 +1,31 @@
+# SKLearn Server
+
+If you have a trained SKLearn model saved as a pickle you can deploy it simply using Seldon's prepackaged SKLearn server.
+
+Prequisites:
+
+  * The model pickle must be saved using joblib and presently be named `model.joblib`
+  * We presently use sklearn version 0.20.3. Your pickled model must be compatbible with this version
+
+An example for a saved Iris prediction model:
+
+```
+apiVersion: machinelearning.seldon.io/v1alpha2
+kind: SeldonDeployment
+metadata:
+  name: sklearn
+spec:
+  name: iris
+  predictors:
+  - graph:
+      children: []
+      implementation: SKLEARN_SERVER
+      modelUri: gs://seldon-models/sklearn/iris
+      name: classifier
+    name: default
+    replicas: 1
+
+```
+
+
+Try out a [worked notebook](../examples/server_examples.html)

doc/source/servers/tensorflow.md

@@ -0,0 +1,82 @@
+# Tensorflow Serving
+
+If you have a trained Tensorflow model you can deploy this directly via REST or gRPC servers.
+
+## MNIST Example
+
+### REST MNIST Example
+
+For REST you need to specify paramaters for:
+
+ * signature_name
+ * model_name
+
+```
+apiVersion: machinelearning.seldon.io/v1alpha2
+kind: SeldonDeployment
+metadata:
+  name: tfserving
+spec:
+  name: mnist
+  predictors:
+  - graph:
+      children: []
+      implementation: TENSORFLOW_SERVER
+      modelUri: gs://seldon-tfserving-store/mnist-model
+      name: mnist-model
+      parameters:
+        - name: signature_name
+          type: STRING
+          value: predict_images
+        - name: model_name
+          type: STRING
+          value: mnist-model
+    name: default
+    replicas: 1
+
+```
+
+### gRPC MNIST Example
+
+For gRPC you need to specify the following parameters:
+
+ * signature_name
+ * model_name
+ * model_input
+ * model_output
+
+```
+apiVersion: machinelearning.seldon.io/v1alpha2
+kind: SeldonDeployment
+metadata:
+  name: tfserving
+spec:
+  name: mnist
+  predictors:
+  - graph:
+      children: []
+      implementation: TENSORFLOW_SERVER
+      modelUri: gs://seldon-tfserving-store/mnist-model
+      name: mnist-model
+      endpoint:
+        type: GRPC
+      parameters:
+        - name: signature_name
+          type: STRING
+          value: predict_images
+        - name: model_name
+          type: STRING
+          value: mnist-model
+        - name: model_input
+          type: STRING
+          value: images
+        - name: model_output
+          type: STRING
+          value: scores          
+    name: default
+    replicas: 1
+
+```
+
+
+Try out a [worked notebook](../examples/server_examples.html)

doc/source/servers/xgboost.md

@@ -0,0 +1,30 @@
+# XGBoost Server
+
+If you have a trained XGBoost model saved you can deploy it simply using Seldon's prepackaged XGBoost server.
+
+Prequisites:
+
+  * The model pickle must be named `model.bst`
+
+An example for a saved Iris prediction model:
+
+```
+apiVersion: machinelearning.seldon.io/v1alpha2
+kind: SeldonDeployment
+metadata:
+  name: xgboost
+spec:
+  name: iris
+  predictors:
+  - graph:
+      children: []
+      implementation: XGBOOST_SERVER
+      modelUri: gs://seldon-models/xgboost/iris
+      name: classifier
+    name: default
+    replicas: 1
+
+```
+
+
+Try out a [worked notebook](../examples/server_examples.html)

doc/source/workflow/README.md

@@ -34,9 +34,17 @@ You can use ```kubectl``` to deploy your ML service like any other Kubernetes re
 
 ## Next Steps
 
-We have created a set of  [notebook step-by-step tutorials](../examples/helm_examples.html) which you can follow to get started with Seldon core.
+If you have a saved model for SKLearn, XGBoost or Tensorflow then you can use oneof our [prepackaged model servers](../servers/overview.html).
 
-### Recommended starter tutorials
+If you want to wrap your custom inference code then follow one of our stater tutorials below for the framework you are using.
+
+### Prepacked Model Servers
+
+ * [SKLearn](../servers/sklearn.html)
+ * [XGBoost](../servers/xgboost.html) 
+ * [Tensorflow](../servers/tensorflow.html)
+
+### Recommended starter tutorials for custom inference code
 
 * [Tensorflow Deep MNIST Tutorial](https://docs.seldon.io/projects/seldon-core/en/latest/examples/deep_mnist.html) (Try it also in [AWS](https://docs.seldon.io/projects/seldon-core/en/latest/examples/aws_eks_deep_mnist.html), [Azure](https://docs.seldon.io/projects/seldon-core/en/latest/examples/azure_aks_deep_mnist.html) and [GKE with GPU](https://github.com/SeldonIO/seldon-core/tree/master/examples/models/gpu_tensorflow_deep_mnist))
 * [SKlearn SpaCy Reddit Text Classification Tutorial](https://docs.seldon.io/projects/seldon-core/en/latest/examples/sklearn_spacy_text_classifier_example.html)

engine/src/main/java/io/seldon/engine/predictors/PredictorConfigBean.java

@@ -82,7 +82,7 @@ public PredictorConfigBean(
     }
 
     public boolean hasMethod(PredictiveUnitState state) {
-    	return (state.implementation == PredictiveUnitImplementation.UNKNOWN_IMPLEMENTATION);
+    	return (state.implementation == PredictiveUnitImplementation.UNKNOWN_IMPLEMENTATION || !nodeImplementationMap.containsKey(state.implementation));
     }
 
     public PredictiveUnitImpl getImplementation(PredictiveUnitState state){
@@ -93,7 +93,8 @@ public PredictiveUnitImpl getImplementation(PredictiveUnitState state){
     }
 
     public Boolean hasMethod(PredictiveUnitMethod method, PredictiveUnitState state){
-    	if (state.implementation != PredictiveUnitImplementation.UNKNOWN_IMPLEMENTATION){
+    	if (state.implementation != PredictiveUnitImplementation.UNKNOWN_IMPLEMENTATION &&
+    			nodeImplementationMap.containsKey(state.implementation)){
     		return false;
     	}
     	if (state.type == PredictiveUnitType.UNKNOWN_TYPE) {