Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added TFServing JSON Support #699

Merged
merged 6 commits into from
Jul 17, 2019

Conversation

axsaucedo
Copy link
Contributor

This pull request contains the code that adds support for raw JSON to be sent to the TF Server, so it allows for the more flexible interface that TF Serving provides. This issue addresses #694

@axsaucedo axsaucedo requested a review from ukclivecox July 16, 2019 19:35
@axsaucedo
Copy link
Contributor Author

This has been tested locally and works as expected, but when run in the kubernetes cluster the TFServing image fails to pull from the Google bucket with the following error:

2019-07-16 18:22:10.905196: I tensorflow_serving/model_servers/main.cc:154] Building single TensorFlow model file config:  model_name: mnist-model model_base_path: gs://seldon-mnist-model-bucket/mnist-model
2019-07-16 18:22:10.905412: I tensorflow_serving/model_servers/server_core.cc:444] Adding/updating models.
2019-07-16 18:22:10.905441: I tensorflow_serving/model_servers/server_core.cc:499]  (Re-)adding model: mnist-model
2019-07-16 18:22:11.776606: I external/org_tensorflow/tensorflow/core/platform/cloud/retrying_utils.cc:77] The operation failed and will be automatically retried in 1.45458 seconds (attempt 1 out of 10), caused by: Unavailable: Error executing an HTTP request (error code 6, error message 'Couldn't resolve host name')
        Performing request. Detailed error: Couldn't resolve host 'metadata'

@axsaucedo
Copy link
Contributor Author

@cliveseldon it seems that the container cannot reach the bucket. It's currently public, and it has been attempted with both the default bucket and a bucket created for this. Is there any pieces that I should be aware in regards to access for buckets? Are they only reachable from a GCloud cluster?

@axsaucedo
Copy link
Contributor Author

The issue was the TFServing image which was outdated and causing issues. It now has been updated to tensorflow/serving:latest, and the changes have been reflected in the Notebook JSON + Helm. The example now provides support for JSON.

Copy link
Contributor

@ukclivecox ukclivecox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

predict_grpc_prev is an old method and should be removed

@ukclivecox ukclivecox merged commit f696443 into SeldonIO:master Jul 17, 2019
@axsaucedo
Copy link
Contributor Author

This introduces the functionality required for #693

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants