Make android tools scripts Python3-compatible for building android apps #4097
Comments
|
I think that #3855 should resolve this issue, but I'll test it to confirm. |
|
Thanks @aj-michael - I was able to reproduce @angersson's errors and can confirm that #3855 resolves the first one. However, the second error persists. It can probably be resolved by updating modules in |
|
As promised, I've created a branch containing python 3 compatibility updates to bazel tools. @angersson @aj-michael @jin @dkelmer @ahumesky you can test the fix by running: If it looks good to you, I'll make a PR. Please note that this branch depends on PR #3855, so we would probably want to have that merged first. Thanks! |
|
I was able to replicate the fixes, but I'll need to fetch NDK 15 to fully test building the app in a clean environment due to #4068. |
|
I verified build success with nvidia-docker, #!/usr/bin/env bash
set -euxo pipefail
TMPDIR=$(mktemp -d)
trap '' PIPE
git clone https://github.com/akira-baruah/bazel.git ${TMPDIR}/bazel-akira
cd ${TMPDIR}/bazel-akira
git checkout python3-tools # branch with bugfixes
bazel build //src:bazel
cd ${TMPDIR}
curl -o ${TMPDIR}/sdk-tools.zip https://dl.google.com/android/repository/sdk-tools-linux-3859397.zip
unzip ${TMPDIR}/sdk-tools.zip -d ${TMPDIR}/sdk
echo "y" | ${TMPDIR}/sdk/tools/bin/sdkmanager "build-tools;26.0.1" "platforms;android-26" "extras;android;m2repository"
pip install virtualenv
virtualenv --python=python3 --system-site-packages ${TMPDIR}/py3
PS1="" # Avoids `set -u`-related error with bin/activate
source ${TMPDIR}/py3/bin/activate
git clone http://github.com/tensorflow/tensorflow ${TMPDIR}/tf
cd ${TMPDIR}/tf
curl -o ${TMPDIR}/ndk.zip https://dl.google.com/android/repository/android-ndk-r14b-linux-x86_64.zip
unzip ${TMPDIR}/ndk.zip -d ${TMPDIR}/ndk/
cat >> ${TMPDIR}/tf/WORKSPACE <<HEREDOC
android_sdk_repository(
name = "androidsdk",
api_level = 26,
build_tools_version = "26.0.1",
path = "${TMPDIR}/sdk",
)
android_ndk_repository(
name="androidndk",
path="${TMPDIR}/ndk/android-ndk-r14b",
api_level=14)
HEREDOC
set +o pipefail
yes '' | ${TMPDIR}/tf/configure
set -o pipefail
cd ${TMPDIR}/tf
${TMPDIR}/bazel-akira/bazel-bin/src/bazel build //tensorflow/contrib/lite/java/demo/app/src/main:TfLiteCameraDemo --cxxopt='--std=c++11' --config=android_armSweet. |
Description of the problem / feature request / question:
Google's TFLite team has discovered that the TF Lite Android demo application cannot build via Bazel with Python 3, because the android tools Bazel uses aren't Python-3 compatible. The first breaking point looks like
third_party/gflags, but past that there seem to be additional version incompatibilities.If possible, provide a minimal example to reproduce the problem:
Here's a very fast example:
$ git clone http://github.com/bazelbuild/bazel /tmp/bazel $ cd /tmp/bazel $ python3 >>> import tools.android.aar_embedded_jars_extractorI'm not actually sure how to do the same as the above in python 2, but that sequence of commands results in the same errors we see when trying to build the demo app (
No module named 'gflags'). I can sort of fix this with$ pip install python-gflags, which makes Python not even try to use the local files (this seems like an ultimately buggy workaround). The import is fixed, but the build (below) is still busted with a str-to-buffer message problem which I've included at the very bottom of the issue.Here's how I ran the TF Lite app build. The build requires the SDK setup steps for Tensorflow's WORKSPACE.
Environment info
bazel info release): release 0.7.0Have you found anything relevant by searching the web?
Anything else, information or logs or outputs that would be helpful?
This incompatibility means that the TF Lite app can't be built (afaik) unless users install Python 2. Not a tremendous problem, but pretty inconvenient.
From the build commands above, I get something like this:
And here's the str-to-buffer problem after installing
pip install python-gflags:The text was updated successfully, but these errors were encountered: