Add Raspberry PI build #11675

ebrevdo · 2017-07-21T22:09:02Z

Does what it says on the box; follow instructions in third_party/toolchains/cpus/arm/build_raspberry_pi.sh

To build: ./configure bazel build --sandbox_debug -c opt --copt=-march=armv6 --copt=-mfpu=vfp \ --copt=-funsafe-math-optimizations --copt=-ftree-vectorize \ --copt=-fomit-frame-pointer --cpu=armeabi \ --crosstool_top=@local_config_arm_compiler//:toolchain \ --verbose_failures --build_python_zip ...targets...

update platform.h and crosstool template for define RASPBERRY_PI.

petewarden

One minor comment.

petewarden · 2017-07-21T22:39:25Z

tensorflow/core/platform/platform.h

-// Since there's no macro for the Raspberry Pi, assume we're on a mobile
-// platform if we're compiling for the ARM CPU.
+// Require an outside macro to tell us if we're building for Raspberry Pi.
+#if !defined(RASPBERRY_PI)


Can you add this define to the Makefile for the Pi too? Hopefully that will go away eventually, but for now it would be good to keep them in sync.

petewarden · 2017-07-22T01:57:59Z

tools/arm_compiler/build_raspberry_pi.sh

+#    sudo tee -a /etc/apt/sources.list.d/armhf.list
+#  # Ignore errors about missing armhf packages in other repos.
+#  sudo apt-get update
+#  sudo apt-get install libpython-all-dev:armhf


I had to use aptitude instead of apt-get on my Goobuntu machine, since otherwise I saw:

Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
libpython-all-dev:armhf : Depends: libpython-dev:armhf (= 2.7.5-5ubuntu3) but it is not going to be installed
Depends: libpython2.7-dev:armhf but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

Interesting. I updated the comment to suggest using aptitude instead.

It did work for you in the end though?

petewarden · 2017-07-22T19:19:07Z

tools/arm_compiler/build_raspberry_pi.sh

+# To install the architecture includes for python on ubuntu trusty;
+# run:
+#  sudo dpkg --add-architecture armhf
+#  echo "deb [arch=armhf] http://ports.ubuntu.com/ trusty main universe" \


I think there should be a pipe '|' in here?

gunan · 2017-07-22T20:29:02Z

tensorflow/workspace.bzl

+
+
+# This performs an unsafe unzip operation - since zip files may have
+# relative and absolute paths, it should only be executed on trusted files.


@meteorcloudy Can bazel handle zip files with symlinks in windows now?
Would this break windows bazel build (not currently run in presubmits)

no; this part is only executed if you run with the --crosstool argument in the .sh script. otherwise none of these commands are executed.

gunan · 2017-07-22T20:31:19Z

tools/arm_compiler/BUILD

@@ -0,0 +1,49 @@
+package(default_visibility = ["//visibility:public"])


Should we move this and its crosstool file under third_party/cpus/cc ?

moved to third_party/toolchains/cpus/arm.

gunan · 2017-07-22T20:32:31Z

tensorflow/workspace.bzl

+
+def _unzip(repo_ctx, zip_file, strip_prefix):
+  _run_cmd(repo_ctx,
+           ["unzip", repo_ctx.path(zip_file),


should we protect this with an OS check?
How platform independent is unzip command?
macos should be OK, but what would this do on windows @meteorcloudy

isn't this going to be running inside a sh shell within windows? i.e., if mkdir and cp work, unzip should too...

(anyway; i'm pretty sure that the crosstool compiler won't run on windows anyway...)

gunan · 2017-07-22T20:33:19Z

arm_compiler.BUILD

@@ -0,0 +1,81 @@
+package(default_visibility = ['//visibility:public'])


Should this file also be under third_party/ ?

Switched to tar file for ARM tool chain

…ty/toolchains/cpus/arm.

…codename.

Hacked around cxx_builtin_include_directory problem in CROSSTOOL

petewarden · 2017-07-23T07:35:05Z

I've got a gist showing how to build this on a clean docker image, and I've verified the resulting wheel works on my Pi!
https://gist.github.com/petewarden/6aa9650fe72d88889d24efd76598f8cb

Adding in @samjabrahams to the thread too, since he's the expert on Pi Python building.

samjabrahams · 2017-07-23T17:43:59Z

Very cool; I'm really happy to see this getting added in. I won't be able to test it personally until August (due to all my RPis being in storage), but I'm excited to give it a shot as soon as I can.

Couple questions/comments:

We're targeting the armv6 instruction set- is the plan to have Raspberry Pi 1/Zero support?
We're not using NEON in the current build (I'm assuming because of the ARMv6 target), but I think it's provides pretty significant optimizations (@petewarden knows more about the details of this than myself). Hopefully, the difference in speed between this and a build with NEON is negligible, but if not, would it be possible to consider adding an option to the build to use it? Granted, this might cause some headaches in the various scripts and would require using GCC 4.8 (I think we could swap in this gnueabi, but it's older and also not my area of expertise).

Finally, it's satisfying that #3469 is getting addressed almost exactly a year to the day after it was opened :D

ebrevdo · 2017-07-24T16:44:17Z

Anything else left to fix for this? @petewarden should i move any of your docker comments into my PR; or do you want to have a separate .sh file after this one is pushed? I'll be OOO for 2 weeks starting Wednesday, so any changes should happen by tomorrow.

ebrevdo · 2017-07-24T16:45:07Z

@samjabrahams happy to help. i am happy to add an argument or some comments in the .sh file for making an optimized build for arm7; or alternatively building for arm7 + neon and showing how to make an arm6 build (probably leaning towards this latter approach).

petewarden · 2017-07-24T17:45:21Z

@ebrevdo I've got an outstanding PR to add ARM7/NEON support and update the comments here, can you take a look? ebrevdo#3

gunan · 2017-07-25T07:17:04Z

Looks good after a rebase.
Test failures do not look related to me, we can rerun them after the rebase.

Added an extra system include

ebrevdo · 2017-07-25T16:17:05Z

Added @petewarden's changes to make default build armv7/NEON. This required upgrading the version of Eigen we use. @benoitsteiner is this OK?

petewarden · 2017-07-25T16:49:38Z

@benoitsteiner I believe this addresses issue #9697

vrv · 2017-07-26T04:33:18Z

@tensorflow-jenkins test this please

petewarden · 2017-07-26T15:23:36Z

The remaining MacOS failures look like timeouts or flakes.

samjabrahams · 2017-07-26T17:15:29Z

Awesome! Once I find some time, I'll note in my repo that official support is available and link to the relevant instructions.

ebrevdo · 2017-07-27T04:40:15Z

I think we need to wait for pete to update the version of eigen and fix any breakages before we can make it official.

…

On Jul 26, 2017 7:16 AM, "Sam Abrahams" ***@***.***> wrote: Awesome! Once I find some time, I'll note in my repo that official support is available and link to the relevant instructions. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#11675 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABtim2EniNKKRsYrVCUsRo19FMlk9X0sks5sR3RggaJpZM4Of8SU> .

petewarden · 2017-07-27T15:40:27Z

That's correct. We're blocked on #9697 for the Pi 3 build, and once that's done I also need to create a Jenkins CI build to run nightly, so we'll have automatically generated binaries for the wheel. I hope to get to that soon.

ptone · 2017-07-29T18:07:32Z

Once done, will these wheels be available on PyPI for pip installation?

https://pypi.python.org/pypi/tensorflow

ebrevdo · 2017-07-30T04:08:01Z

if we can figure out how to fake the architecture to bdist_wheel after the cross compile, maybe.

…

On Jul 29, 2017 8:08 AM, "Preston Holmes" ***@***.***> wrote: Once done, will these wheels be available on PyPI for pip installation? https://pypi.python.org/pypi/tensorflow — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#11675 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABtimzu6XxauAHkk4qMOICAB_en1xJSdks5sS3UEgaJpZM4Of8SU> .

ptone · 2017-07-30T12:32:37Z

I believe the grpc team sorted this out somehow - I can forward you an internal thread.

https://pypi.python.org/pypi/grpcio/1.4.0

ptone · 2017-07-30T16:31:32Z

I'm not sure what happened, (git-gone-funky?), but the update to the Eigen version did not make it into master:
c848583#diff-455a4c7f8e22d7c514e8c2caa27506c5R162

As of the current master commit the eigen archive is still set to rev f3a22f35b044

petewarden · 2017-07-30T19:53:53Z

@ptone Sorry, I should have been more clear in the comments. We're blocked on updating Eigen (#9697) because the newer Eigen version breaks on MacOS. We'll need to see if other Eigen versions work on both, but I'm not working on that at the moment.

If anyone wants, they can update the Eigen hash to the Pi 3-capable one locally and build the wheel, but we won't be able to support it officially until the MacOS issue is resolved.

AnchoretBaladev · 2018-02-25T11:40:01Z

Hello @petewarden I'm trying to cross compile tensorflow for Raspberry PI 3 from Ubuntu 16.04LTS using the "build_raspberry_pi.sh" present inside "/tensorflow/tools/ci_build/pi/" with the following steps:-
1:- Installing Bazel
2:- Cloning the Tensorflow and runnining the "build_raspberry_pi.sh"
but not getting success!!
Can you please guide me what are the needed steps to follow to cross compile!!! Thanks in Advance!!

ebrevdo added 3 commits July 21, 2017 13:01

Additional changes to get raspberry pi build working.

c7a78d7

update platform.h and crosstool template for define RASPBERRY_PI.

Add build_raspberry_pi.sh with some documentation.

0ca2379

googlebot added the cla: yes label Jul 21, 2017

ebrevdo requested review from gunan and petewarden July 21, 2017 22:09

Cleanups for build_raspberry_pi.sh

56127f4

petewarden approved these changes Jul 21, 2017

View reviewed changes

contrib/makefile/Makefile in PI mode: add -DRASPBERRY_PI.

35a7c9f

ebrevdo requested review from satok16 and wolffg as code owners July 21, 2017 23:18

petewarden reviewed Jul 22, 2017

View reviewed changes

ebrevdo added 2 commits July 21, 2017 19:09

Recommend using aptitude instead of apt-get in build_raspberry_pi.sh.

fe798da

More comments to build_raspberry_pi.sh

0e673f4

petewarden reviewed Jul 22, 2017

View reviewed changes

gunan reviewed Jul 22, 2017

View reviewed changes

petewarden and others added 10 commits July 22, 2017 21:31

Switched to tar file for ARM tool chain

a32d821

Merge pull request #1 from petewarden/tf_master

d554afe

Switched to tar file for ARM tool chain

Merge branch 'master' of github.com:tensorflow/tensorflow into tf_master

148009c

Merge remote-tracking branch 'origin/tf_master' into tf_master

e7d30d3

More fixes to arm build scripts; move tools/arm_compiler -> third_par…

2b47c6b

…ty/toolchains/cpus/arm.

Update comments in build_raspberry_pi: get ubuntu ports for your lsb …

cd4008f

…codename.

Hacked around cxx_builtin_include_directory problem in CROSSTOOL

298cf40

Merge remote-tracking branch 'upstream/tf_master' into tf_master

3c379a6

Merge pull request #2 from petewarden/tf_master

0efde24

Hacked around cxx_builtin_include_directory problem in CROSSTOOL

Added an extra system include

6666367

petewarden mentioned this pull request Jul 23, 2017

tensorflow in armv7l #445

Closed

Added Pi Two/Three NEON support, with fix for Eigen version

c848583

gunan approved these changes Jul 25, 2017

View reviewed changes

Merge pull request #3 from petewarden/tf_master

a74bded

Added an extra system include

Merge branch 'master' into tf_master

550260b

Undo pete's Eigen change - it breaks OS X build. Will be fixed later.

f1bb14d

vrv merged commit b244912 into tensorflow:master Jul 26, 2017

samjabrahams mentioned this pull request Aug 17, 2017

Feature request: Please release an official binary for Raspberry Pi #5729

Closed

vrv mentioned this pull request Jun 8, 2018

custom Android op configuration #10254

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Raspberry PI build #11675

Add Raspberry PI build #11675

ebrevdo commented Jul 21, 2017 •

edited

Loading

petewarden left a comment

petewarden Jul 21, 2017

ebrevdo Jul 21, 2017

petewarden Jul 22, 2017

ebrevdo Jul 22, 2017

petewarden Jul 22, 2017

gunan Jul 22, 2017

ebrevdo Jul 22, 2017

gunan Jul 22, 2017

ebrevdo Jul 22, 2017

gunan Jul 22, 2017

ebrevdo Jul 22, 2017

ebrevdo Jul 22, 2017

gunan Jul 22, 2017

ebrevdo Jul 22, 2017

petewarden commented Jul 23, 2017

samjabrahams commented Jul 23, 2017

ebrevdo commented Jul 24, 2017

ebrevdo commented Jul 24, 2017

petewarden commented Jul 24, 2017

gunan commented Jul 25, 2017

ebrevdo commented Jul 25, 2017

petewarden commented Jul 25, 2017

vrv commented Jul 26, 2017

petewarden commented Jul 26, 2017

samjabrahams commented Jul 26, 2017

ebrevdo commented Jul 27, 2017 via email

petewarden commented Jul 27, 2017

ptone commented Jul 29, 2017

ebrevdo commented Jul 30, 2017 via email

ptone commented Jul 30, 2017

ptone commented Jul 30, 2017

petewarden commented Jul 30, 2017

AnchoretBaladev commented Feb 25, 2018



		# This performs an unsafe unzip operation - since zip files may have
		# relative and absolute paths, it should only be executed on trusted files.

		@@ -0,0 +1,49 @@
		package(default_visibility = ["//visibility:public"])

		@@ -0,0 +1,81 @@
		package(default_visibility = ['//visibility:public'])

Add Raspberry PI build #11675

Add Raspberry PI build #11675

Conversation

ebrevdo commented Jul 21, 2017 • edited Loading

petewarden left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

petewarden commented Jul 23, 2017

samjabrahams commented Jul 23, 2017

ebrevdo commented Jul 24, 2017

ebrevdo commented Jul 24, 2017

petewarden commented Jul 24, 2017

gunan commented Jul 25, 2017

ebrevdo commented Jul 25, 2017

petewarden commented Jul 25, 2017

vrv commented Jul 26, 2017

petewarden commented Jul 26, 2017

samjabrahams commented Jul 26, 2017

ebrevdo commented Jul 27, 2017 via email

petewarden commented Jul 27, 2017

ptone commented Jul 29, 2017

ebrevdo commented Jul 30, 2017 via email

ptone commented Jul 30, 2017

ptone commented Jul 30, 2017

petewarden commented Jul 30, 2017

AnchoretBaladev commented Feb 25, 2018

ebrevdo commented Jul 21, 2017 •

edited

Loading