Merge pull request #201 from cliveseldon/python_wrappers_versioning

Change wrappers to be versioned and update examples and docs
SeldonIO · Aug 7, 2018 · dd0ff81 · dd0ff81
2 parents c1f7262 + 965344c
commit dd0ff81
Show file tree

Hide file tree

Showing 22 changed files with 95 additions and 53 deletions.
diff --git a/.gitignore b/.gitignore
@@ -93,6 +93,19 @@ examples/models/h2o_mojo/src/main/resources/model.zip
 examples/models/deep_mnist/MNIST_data/
 examples/models/deep_mnist/model/
 examples/models/sklearn_iris/IrisClassifier.sav
+examples/istio/canary_update/MNIST_data/
+examples/istio/canary_update/tmp/
+examples/models/keras_mnist/MnistClassifier.h5
+examples/models/keras_mnist/data/
+examples/models/keras_mnist/tensorboardlogs_test/
+examples/models/r_iris/model.Rds
+examples/models/r_mnist/model.Rds
+examples/models/r_mnist/t10k-images-idx3-ubyte
+examples/models/r_mnist/t10k-labels-idx1-ubyte
+examples/models/r_mnist/train-images-idx3-ubyte
+examples/models/r_mnist/train-labels-idx1-ubyte
+examples/models/sk_mnist/MNIST_data/
+examples/models/sk_mnist/sk.pkl
 
 # api tester created proto folder
-util/api_tester/proto/
+util/api_tester/proto/
diff --git a/docs/wrappers/java.md b/docs/wrappers/java.md
@@ -17,7 +17,7 @@ If you are not familar with s2i you can read [general instructions on using s2i]
 To check everything is working you can run
 
 ```bash
-s2i usage seldonio/seldon-core-s2i-java-build
+s2i usage seldonio/seldon-core-s2i-java-build:0.1
 ```
 
 # Step 2 - Create your source code
@@ -149,14 +149,14 @@ Use ```s2i build``` to create your Docker image from source code. You will need
 Using s2i you can build directly from a git repo or from a local source folder. See the [s2i docs](https://github.com/openshift/source-to-image/blob/master/docs/cli.md#s2i-build) for further details. The general format is:
 
 ```bash
-s2i build <git-repo> seldonio/seldon-core-s2i-java-build <my-image-name> --runtime-image seldonio/seldon-core-s2i-java-runtime
-s2i build <src-folder> seldonio/seldon-core-s2i-java-build <my-image-name> --runtime-image seldonio/seldon-core-s2i-java-runtime 
+s2i build <git-repo> seldonio/seldon-core-s2i-java-build:0.1 <my-image-name> --runtime-image seldonio/seldon-core-s2i-java-runtime:0.1
+s2i build <src-folder> seldonio/seldon-core-s2i-java-build:0.1 <my-image-name> --runtime-image seldonio/seldon-core-s2i-java-runtime:0.1 
 ```
 
 An example invocation using the test template model inside seldon-core:
 
 ```bash
-s2i build https://github.com/seldonio/seldon-core.git --context-dir=wrappers/s2i/python/test/model-template-app seldonio/seldon-core-s2i-java-build h2o-test:0.1 --runtime-image seldonio/seldon-core-s2i-java-runtime
+s2i build https://github.com/seldonio/seldon-core.git --context-dir=wrappers/s2i/python/test/model-template-app seldonio/seldon-core-s2i-java-build:0.1 h2o-test:0.1 --runtime-image seldonio/seldon-core-s2i-java-runtime:0.1
 ```
 
 The above s2i build invocation:
@@ -172,13 +172,13 @@ For building from a local source folder, an example where we clone the seldon-co
 ```bash
 git clone https://github.com/seldonio/seldon-core.git
 cd seldon-core
-s2i build wrappers/s2i/R/test/model-template-app seldonio/seldon-core-s2i-java-build h2o-test:0.1 --runtime-image seldonio/seldon-core-s2i-java-runtime
+s2i build wrappers/s2i/R/test/model-template-app seldonio/seldon-core-s2i-java-build:0.1 h2o-test:0.1 --runtime-image seldonio/seldon-core-s2i-java-runtime:0.1
 ```
 
 For more help see:
 
 ```
-s2i usage seldonio/seldon-core-s2i-java-build
+s2i usage seldonio/seldon-core-s2i-java-build:0.1
 s2i build --help
 ```
 

diff --git a/docs/wrappers/python-docker.md b/docs/wrappers/python-docker.md
@@ -1,5 +1,9 @@
 # Packaging a python model for Seldon Core using Seldon Wrapper
-In this guide, we illustrate the steps needed to wrap your own python model in a docker image ready for deployment with Seldon Core, using the Seldon wrapper script. This script is designed to take your python model and turn it into a dockerised microservice that conforms to Seldon's internal API, thus avoiding the hassle to write your own dockerised microservice.
+In this guide, we illustrate the steps needed to wrap your own python model in a docker image ready for deployment with Seldon Core, using the Seldon wrapper script.
+
+We suggest you look at using the [S2I tool for python models](python.md) before choosing this method.
+
+This script is designed to take your python model and turn it into a dockerised microservice that conforms to Seldon's internal API, thus avoiding the hassle to write your own dockerised microservice.
 
 You can use these wrappers with any model that offers a python API. Some examples are:
 

diff --git a/docs/wrappers/python.md b/docs/wrappers/python.md
@@ -17,7 +17,7 @@ If you are not familar with s2i you can read [general instructions on using s2i]
 To check everything is working you can run
 
 ```bash
-s2i usage seldonio/seldon-core-s2i-python3
+s2i usage seldonio/seldon-core-s2i-python3:0.1
 ```
 
 
@@ -83,22 +83,22 @@ These values can also be provided or overriden on the command line when building
 # Step 3 - Build your image
 Use ```s2i build``` to create your Docker image from source code. You will need Docker installed on the machine and optionally git if your source code is in a public git repo. You can choose from two python builder images
 
- * Python 2 : seldonio/seldon-core-s2i-python2
- * Python 3 : seldonio/seldon-core-s2i-python3
+ * Python 2 : seldonio/seldon-core-s2i-python2:0.1
+ * Python 3 : seldonio/seldon-core-s2i-python3:0.1
 
 Using s2i you can build directly from a git repo or from a local source folder. See the [s2i docs](https://github.com/openshift/source-to-image/blob/master/docs/cli.md#s2i-build) for further details. The general format is:
 
 ```bash
-s2i build <git-repo> seldonio/seldon-core-s2i-python2 <my-image-name>
-s2i build <src-folder> seldonio/seldon-core-s2i-python2 <my-image-name>
+s2i build <git-repo> seldonio/seldon-core-s2i-python2:0.1 <my-image-name>
+s2i build <src-folder> seldonio/seldon-core-s2i-python2:0.1 <my-image-name>
 ```
 
 Change to seldonio/seldon-core-s2i-python3 if using python 3.
 
 An example invocation using the test template model inside seldon-core:
 
 ```bash
-s2i build https://github.com/seldonio/seldon-core.git --context-dir=wrappers/s2i/python/test/model-template-app seldonio/seldon-core-s2i-python2 seldon-core-template-model
+s2i build https://github.com/seldonio/seldon-core.git --context-dir=wrappers/s2i/python/test/model-template-app seldonio/seldon-core-s2i-python2:0.1 seldon-core-template-model
 ```
 
 The above s2i build invocation:
@@ -113,14 +113,14 @@ For building from a local source folder, an example where we clone the seldon-co
 ```bash
 git clone https://github.com/seldonio/seldon-core.git
 cd seldon-core
-s2i build wrappers/s2i/python/test/model-template-app seldonio/seldon-core-s2i-python2 seldon-core-template-model
+s2i build wrappers/s2i/python/test/model-template-app seldonio/seldon-core-s2i-python2:0.1 seldon-core-template-model
 ```
 
 For more help see:
 
 ```
-s2i usage seldonio/seldon-core-s2i-python2
-s2i usage seldonio/seldon-core-s2i-python3
+s2i usage seldonio/seldon-core-s2i-python2:0.1
+s2i usage seldonio/seldon-core-s2i-python3:0.1
 s2i build --help
 ```
 

diff --git a/docs/wrappers/r.md b/docs/wrappers/r.md
@@ -17,7 +17,7 @@ If you are not familar with s2i you can read [general instructions on using s2i]
 To check everything is working you can run
 
 ```bash
-s2i usage seldonio/seldon-core-s2i-r
+s2i usage seldonio/seldon-core-s2i-r:0.1
 ```
 
 # Step 2 - Create your source code
@@ -84,14 +84,14 @@ Use ```s2i build``` to create your Docker image from source code. You will need
 Using s2i you can build directly from a git repo or from a local source folder. See the [s2i docs](https://github.com/openshift/source-to-image/blob/master/docs/cli.md#s2i-build) for further details. The general format is:
 
 ```bash
-s2i build <git-repo> seldonio/seldon-core-s2i-r <my-image-name>
-s2i build <src-folder> seldonio/seldon-core-s2i-r <my-image-name>
+s2i build <git-repo> seldonio/seldon-core-s2i-r:0.1 <my-image-name>
+s2i build <src-folder> seldonio/seldon-core-s2i-r:0.1 <my-image-name>
 ```
 
 An example invocation using the test template model inside seldon-core:
 
 ```bash
-s2i build https://github.com/seldonio/seldon-core.git --context-dir=wrappers/s2i/R/test/model-template-app seldonio/seldon-core-s2i-r seldon-core-template-model
+s2i build https://github.com/seldonio/seldon-core.git --context-dir=wrappers/s2i/R/test/model-template-app seldonio/seldon-core-s2i-r:0.1 seldon-core-template-model
 ```
 
 The above s2i build invocation:
@@ -106,13 +106,13 @@ For building from a local source folder, an example where we clone the seldon-co
 ```bash
 git clone https://github.com/seldonio/seldon-core.git
 cd seldon-core
-s2i build wrappers/s2i/R/test/model-template-app seldonio/seldon-core-s2i-r seldon-core-template-model
+s2i build wrappers/s2i/R/test/model-template-app seldonio/seldon-core-s2i-r:0.1 seldon-core-template-model
 ```
 
 For more help see:
 
 ```
-s2i usage seldonio/seldon-core-s2i-r
+s2i usage seldonio/seldon-core-s2i-r:0.1
 s2i build --help
 ```
 

diff --git a/examples/models/deep_mnist/deep_mnist.ipynb b/examples/models/deep_mnist/deep_mnist.ipynb
@@ -83,7 +83,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!s2i build . seldonio/seldon-core-s2i-python2 deep-mnist:0.1"
+    "!s2i build . seldonio/seldon-core-s2i-python2:0.1 deep-mnist:0.1"
    ]
   },
   {
@@ -172,7 +172,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!eval $(minikube docker-env) && s2i build . seldonio/seldon-core-s2i-python2 deep-mnist:0.1"
+    "!eval $(minikube docker-env) && s2i build . seldonio/seldon-core-s2i-python2:0.1 deep-mnist:0.1"
    ]
   },
   {

diff --git a/examples/models/h2o_mojo/h2o_model.ipynb b/examples/models/h2o_mojo/h2o_model.ipynb
@@ -77,7 +77,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!s2i build . seldonio/seldon-core-s2i-java-build h2o-test:0.1 --runtime-image seldonio/seldon-core-s2i-java-runtime"
+    "!s2i build . seldonio/seldon-core-s2i-java-build:0.1 h2o-test:0.1 --runtime-image seldonio/seldon-core-s2i-java-runtime:0.1"
    ]
   },
   {
@@ -166,7 +166,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!eval $(minikube docker-env) && s2i build . seldonio/seldon-core-s2i-java-build h2o-test:0.1 --runtime-image seldonio/seldon-core-s2i-java-runtime"
+    "!eval $(minikube docker-env) && s2i build . seldonio/seldon-core-s2i-java-build:0.1 h2o-test:0.1 --runtime-image seldonio/seldon-core-s2i-java-runtime:0.1"
    ]
   },
   {

diff --git a/examples/models/keras_mnist/MnistClassifier.py b/examples/models/keras_mnist/MnistClassifier.py
@@ -4,6 +4,7 @@ class MnistClassifier(object):
 
     def __init__(self):
         self.model = load_model('MnistClassifier.h5')
+        self.model._make_predict_function()
 
     def predict(self,X,features_names):
         return self.model.predict(X)
diff --git a/examples/models/keras_mnist/keras_mnist.ipynb b/examples/models/keras_mnist/keras_mnist.ipynb
@@ -251,7 +251,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!s2i build . seldonio/seldon-core-s2i-python3 keras-mnist:0.1"
+    "!s2i build . seldonio/seldon-core-s2i-python3:0.1 keras-mnist:0.1"
    ]
   },
   {
@@ -340,7 +340,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!eval $(minikube docker-env) && s2i build . seldonio/seldon-core-s2i-python3 keras-mnist:0.1"
+    "!eval $(minikube docker-env) && s2i build . seldonio/seldon-core-s2i-python3:0.1 keras-mnist:0.1"
    ]
   },
   {

diff --git a/examples/models/r_iris/r_iris.ipynb b/examples/models/r_iris/r_iris.ipynb
@@ -48,7 +48,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!s2i build . seldonio/seldon-core-s2i-r r-iris:0.1"
+    "!s2i build . seldonio/seldon-core-s2i-r:0.1 r-iris:0.1"
    ]
   },
   {
@@ -137,7 +137,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!eval $(minikube docker-env) && s2i build . seldonio/seldon-core-s2i-r r-iris:0.1"
+    "!eval $(minikube docker-env) && s2i build . seldonio/seldon-core-s2i-r:0.1 r-iris:0.1"
    ]
   },
   {

diff --git a/examples/models/r_mnist/r_mnist.ipynb b/examples/models/r_mnist/r_mnist.ipynb
@@ -48,7 +48,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!s2i build . seldonio/seldon-core-s2i-r r-mnist:0.1"
+    "!s2i build . seldonio/seldon-core-s2i-r:0.1 r-mnist:0.1"
    ]
   },
   {
@@ -137,7 +137,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!eval $(minikube docker-env) && s2i build . seldonio/seldon-core-s2i-r r-mnist:0.1"
+    "!eval $(minikube docker-env) && s2i build . seldonio/seldon-core-s2i-r:0.1 r-mnist:0.1"
    ]
   },
   {

diff --git a/examples/models/sk_mnist/skmnist.ipynb b/examples/models/sk_mnist/skmnist.ipynb
@@ -83,7 +83,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!s2i build . seldonio/seldon-core-s2i-python3 sk-mnist:0.1"
+    "!s2i build . seldonio/seldon-core-s2i-python3:0.1 sk-mnist:0.1"
    ]
   },
   {
@@ -172,7 +172,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!eval $(minikube docker-env) && s2i build . seldonio/seldon-core-s2i-python3 sk-mnist:0.1"
+    "!eval $(minikube docker-env) && s2i build . seldonio/seldon-core-s2i-python3:0.1 sk-mnist:0.1"
    ]
   },
   {

diff --git a/examples/models/sklearn_iris/sklearn_iris.ipynb b/examples/models/sklearn_iris/sklearn_iris.ipynb
@@ -78,7 +78,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!s2i build . seldonio/seldon-core-s2i-python3 sklearn-iris:0.1"
+    "!s2i build . seldonio/seldon-core-s2i-python3:0.1 sklearn-iris:0.1"
    ]
   },
   {
@@ -137,7 +137,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!s2i build -E .s2i/environment_grpc . seldonio/seldon-core-s2i-python3 sklearn-iris:0.1"
+    "!s2i build -E .s2i/environment_grpc . seldonio/seldon-core-s2i-python3:0.1 sklearn-iris:0.1"
    ]
   },
   {
@@ -240,7 +240,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!eval $(minikube docker-env) && s2i build . seldonio/seldon-core-s2i-python3 sklearn-iris:0.1"
+    "!eval $(minikube docker-env) && s2i build . seldonio/seldon-core-s2i-python3:0.1 sklearn-iris:0.1"
    ]
   },
   {

diff --git a/readme.md b/readme.md
@@ -3,8 +3,10 @@
 | Branch      | Status |
 |-------------|-------|
 | master      | [![Build Status](https://travis-ci.org/SeldonIO/seldon-core.svg?branch=master)](https://travis-ci.org/SeldonIO/seldon-core) |
+| release-0.2 | [![Build Status](https://travis-ci.org/SeldonIO/seldon-core.svg?branch=release-0.2)](https://travis-ci.org/SeldonIO/seldon-core) |
 | release-0.1 | [![Build Status](https://travis-ci.org/SeldonIO/seldon-core.svg?branch=release-0.1)](https://travis-ci.org/SeldonIO/seldon-core) |
 
+
 Seldon Core is an open source platform for deploying machine learning models on Kubernetes.
 
 - [Goals](#goals)
@@ -148,3 +150,17 @@ Three steps:
 
  - [CHANGELOG](CHANGELOG.md)
  - [Developer Guide](./docs/developer/readme.md)
+
+## Latest Seldon Images
+
+|Description | Image URL | Version | 
+|------------|-----------|---------|
+| Seldon Operator | [seldonio/cluster-manager](https://hub.docker.com/r/seldonio/cluster-manager/tags/) | 0.2.2 |
+| Seldon Service Orchestrator | [seldonio/engine](https://hub.docker.com/r/seldonio/engine/tags/) | 0.2.2 | 
+| Seldon API Gateway | [seldonio/apife](https://hub.docker.com/r/seldonio/apife/tags/) | 0.2.2 | 
+| [Seldon Python 3 Wrapper for S2I](docs/wrappers/python.md) | [seldonio/seldon-core-s2i-python3](https://hub.docker.com/r/seldonio/seldon-core-s2i-python3/tags/) | 0.1 |
+| [Seldon Python 2 Wrapper for S2I](docs/wrappers/python.md) | [seldonio/seldon-core-s2i-python2](https://hub.docker.com/r/seldonio/seldon-core-s2i-python2/tags/) | 0.1 |
+| [Seldon Core Python Wrapper](docs/wrappers/python-docker.md) | [seldonio/core-python-wrapper](https://hub.docker.com/r/seldonio/core-python-wrapper/tags/) | 0.7 | 
+| [Seldon Java Build Wrapper for S2I](docs/wrappers/java.md) | [seldonio/seldon-core-s2i-java-build](https://hub.docker.com/r/seldonio/seldon-core-s2i-java-build/tags/) | 0.1 |
+| [Seldon Java Runtime Wrapper for S2I](docs/wrappers/java.md) | [seldonio/seldon-core-s2i-java-runtime](https://hub.docker.com/r/seldonio/seldon-core-s2i-java-runtime/tags/) | 0.1 | 
+| [Seldon R Wrapper for S2I](docs/wrappers/r.md) | [seldonio/seldon-core-s2i-r](https://hub.docker.com/r/seldonio/seldon-core-s2i-r) | 0.1 | 
diff --git a/util/api_tester/api-tester.py b/util/api_tester/api-tester.py
@@ -55,7 +55,7 @@ def generate_batch(contract,n):
             batch = gen_continuous(range,shape)
             batch = reconciliate_cont_type(batch,feature_def["dtype"])
         elif feature_def["ftype"] == "categorical":
-            batch = gen_categorical(feature_def["values"],n)
+            batch = gen_categorical(feature_def["values"],[n,1])
         feature_batches.append(batch)
     return np.concatenate(feature_batches,axis=1)
 

diff --git a/wrappers/python/seldon_requirements.txt b/wrappers/python/seldon_requirements.txt
@@ -1,6 +1,6 @@
-numpy==1.11.2
-pandas==0.18.1
-grpcio==1.1.3
-Flask==0.11.1
+numpy==1.14.5
+pandas==0.23.4
+grpcio==1.14.0
+Flask==1.0.2
 futures
 redis==2.10.5
diff --git a/wrappers/s2i/R/Makefile b/wrappers/s2i/R/Makefile
@@ -1,4 +1,5 @@
-IMAGE_NAME = docker.io/seldonio/seldon-core-s2i-r
+VERSION=0.1
+IMAGE_NAME = docker.io/seldonio/seldon-core-s2i-r:${VERSION}
 
 SELDON_CORE_DIR=../../..
 
@@ -8,7 +9,7 @@ build:
 	docker build -t $(IMAGE_NAME) .
 
 push_to_dockerhub:
-	docker push $(IMAGE_NAME):latest
+	docker push $(IMAGE_NAME)
 
 
 .PHONY: test