tensorflow raises exception when importing

[jwkvam]

Work Around

#779 (comment)

Context

I'm not sure if this is a Zappa bug but since it involves tensorflow, I hope it would be of interest to others at least.

Expected Behavior

Tensorflow import without error.

Actual Behavior

Traceback:

Traceback (most recent call last):
  File "/var/task/handler.py", line 508, in lambda_handler
  return LambdaHandler.lambda_handler(event, context)
  File "/var/task/handler.py", line 239, in lambda_handler
  handler = cls()
  File "/var/task/handler.py", line 128, in __init__
  self.app_module = importlib.import_module(self.settings.APP_MODULE)
  File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
  __import__(name)
  File "/tmp/zappaserv/app.py", line 9, in <module>
  import tensorflow as tf
  File "/tmp/pip-build-bxV1IK/tensorflow/tensorflow-1.0.1.data/purelib/tensorflow/__init__.py", line 24, in <module>
  File "/tmp/pip-build-bxV1IK/tensorflow/tensorflow-1.0.1.data/purelib/tensorflow/python/__init__.py", line 124, in <module>
  File "/tmp/pip-build-bxV1IK/tensorflow/tensorflow-1.0.1.data/purelib/tensorflow/python/platform/test.py", line 50, in <module>
  File "/tmp/pip-build-c1VilV/mock/mock/__init__.py", line 2, in <module>
  File "/tmp/pip-build-c1VilV/mock/mock/mock.py", line 71, in <module>
  File "/tmp/pip-build-c1VilV/pbr/pbr/version.py", line 460, in semantic_version
  File "/tmp/pip-build-c1VilV/pbr/pbr/version.py", line 447, in _get_version_from_pkg_resources
  File "/tmp/pip-build-c1VilV/pbr/pbr/packaging.py", line 750, in get_version
Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. It's also possible that there is a mismatch between the package name in setup.cfg and the argument given to pbr.version.VersionInfo. Project name mock was given, but was not able to be found.

Possible Fix

Unsure

Steps to Reproduce

app.py

import os
from flask import Flask
app = Flask(__name__)
app.debug = True

import tensorflow as tf
import numpy as np

@app.route('/')
def index():
    print(np.__version__)
    y = np.ones(5)
    return "Hello, world!" + str(y), 200

# We only need this for local development.
if __name__ == '__main__':
    app.run()

settings.json

{
    "dev": {
        "app_function": "app.app",
        "s3_bucket": "zappa-qxuea5nia",
        "memory_size": 1024,
        "timeout_seconds": 300,
        "debug": true,
        "use_precompiled_packages": false,
        "slim_handler": true,
        "keep_warm": false
    }
}

Your Environment

• Zappa version used:
• Operating System and Python version:
  • Ubuntu 14.04.5 LTS
  • Python 2.7.13

pip freeze

appdirs==1.4.3
argcomplete==1.8.2
base58==0.2.4
boto3==1.4.4
botocore==1.5.31
click==6.7
docutils==0.13.1
durationpy==0.3
Flask==0.12.1
funcsigs==1.0.2
futures==3.0.5
hjson==2.0.2
itsdangerous==0.24
Jinja2==2.9.6
jmespath==0.9.2
kappa==0.6.0
lambda-packages==0.14.2
MarkupSafe==1.0
mock==2.0.0
numpy==1.12.1
packaging==16.8
pbr==2.0.0
placebo==0.8.1
protobuf==3.2.0
pyparsing==2.2.0
python-dateutil==2.6.0
python-slugify==1.2.1
PyYAML==3.12
requests==2.13.0
s3transfer==0.1.10
six==1.10.0
tensorflow==1.0.1
toml==0.9.2
tqdm==4.11.2
troposphere==1.9.2
Unidecode==0.4.20
Werkzeug==0.12
wsgi-request-logger==0.4.6
zappa==0.40.0

[Miserlou]

Thanks for reporting this, definitely need this to be supported.

Surely related to: Miserlou/lambda-packages#31

[jwkvam]

Still trying to root cause this, but have narrowed it down to a problem with the mock package in combination with slim_handler: true. It seems it can't figure out the version info in this case.

[Miserlou]

Ahhhh, in that case, also related to:

#755
#776

[jwkvam]

There might be some relevant info with the following issues. At it's core. This line of code

_v = VersionInfo('mock').semantic_version()

fails from https://github.com/testing-cabal/mock/blob/master/mock/mock.py#L71, in particular, the call to semantic_version().

https://github.com/testing-cabal/mock/search?q=semantic_version&type=Issues&utf8=%E2%9C%93

[jwkvam]

Well considering that all pbr is doing for mock is returning its version info I think I'm going to stop trying to figure this one out and just replace those lines of code above with some constants. I don't know enough about python packaging to effectively debug this.

Work Around

Replace https://github.com/testing-cabal/mock/blob/master/mock/mock.py#L71-L73 with

__version__ = '2.0.0'
version_info = (2, 0, 0, 'final', 0)

A patch file is here so you can automate this.

Now tensorflow runs!

[Miserlou]

Good bug hunting! Looks like something funky is going on upstream - but where? Do you want to file related an upstream ticket for this?

[jwkvam]

Yea I was thinking about what I could tell them, without them pointing a finger back saying what does "slim_handler" do. I'm looking at this issue which appears similar.

Obviously I'd rather not patch mock every time I deploy a new tensorflow service.

[jwkvam]

Upstream responded saying that the dist-info metadata is missing. They also pointed to this issue as reference.

[Miserlou]

That should be easy to patch by changing the ZIP_EXCLUDES, want to send a lil' PR? :)

[jwkvam]

Sure I can give it a shot :)

[jwkvam]

It seems that the dist-info is already being kept in the zipfile. I'm thinking the problem is that it can't find it. I'll probably have to understand testing-cabal/mock#385 to make progress here.

[Miserlou]

Okay. I'd also make sure that it isn't related to #755 (comment) and the related discussion on Slack w/ @mcrowson

[jwkvam]

It's probably worth pointing out that this issue doesn't exist in python 3. I suppose since it uses the mock functionality provided by unittest.
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/pip_package/setup.py#L54-L57