Add ARM support #2546

penta314 · 2019-05-09T15:33:44Z

It has been found that the binaries are not built for ARM, hence people trying to run a node in for example Rasberry Pi 3 are getting crashes.

prestonvanloon · 2019-05-10T03:50:07Z

This requires build docker images/binaries with --platforms=@io_bazel_rules_go//go/toolchain:linux_arm64.

The validator client builds for ARM, but not the beacon-chain due to some cgo dependencies in go-ethereum.

prestonvanloon · 2019-05-10T18:50:45Z

I made a brief attempt at trying to cross compile for ARM, but running into weird issues with cgo.
Commit ea790c5

diglos · 2019-05-13T13:23:23Z

Hi,
It fails compiling directly on ARM64 as well, see attached log.
beacon-chain-build-arm64.txt

prestonvanloon · 2019-06-03T21:56:49Z

The binaries should build on ARM as of e71e91c, but we don't officially support ARM binaries at this time.

diglos · 2019-06-04T10:22:13Z

Hi,
yes, it compiles fine on ARM64, thanks for noticing. I will be doing some testing with this.
Regards!

SoySauceFor3 · 2019-06-18T14:18:43Z

@diglos Hi,
I am trying to use this on raspberry pi, and on the way figuring out how to compile this repo from the source. Can you give me some guidance? thanks!

diglos · 2019-06-18T14:45:18Z

Hi Xinya,
the tricky part is compile bazel for ARM. This is possible on a ARM64 board with 4GB RAM but I don't know if the Raspberry pi can do this. You may try this guide (section 3):
https://github.com/samjabrahams/tensorflow-on-raspberry-pi/blob/master/GUIDE.md
and, once you have the bazel binary, follow this page instructions to build the prysm validator and beacon chain.

SoySauceFor3 · 2019-06-18T15:27:25Z

I will try this, thanks!

prestonvanloon · 2019-06-19T01:14:53Z

I think this was blocked by crosstool rewrite (bazelbuild/bazel#5380).

Most likely, we need to add some cc toolchains for cross compilation. See: https://docs.bazel.build/versions/master/tutorial/cc-toolchain-config.html

There is a tutorial here for clang: https://github.com/bazelbuild/rules_go/blob/master/go/crosstool.rst

Other artifacts:

Now blocked by bazel-contrib/rules_go#2089

jbriquet · 2019-06-19T02:17:49Z

Here are the notes I have taken when compiling bazel for an ARM64 (4 cores A53, 2GB RAM).

Add repository to find openjdk-8 source
sudo add-apt-repository ppa:webupd8team/java

Install JDK8 and dependencies
sudo apt-get install build-essential openjdk-8-jdk python zip unzip

Download Bazel

wget https://github.com/bazelbuild/bazel/releases/download/0.25.3/bazel-0.25.3-dist.zip
unzip bazel-0.25.3-dist.zip -d bazel-0.25.3-dist

Install Bazel on targeted device
cd bazel-0.25.3-dist
export BAZEL_JAVAC_OPTS="-J-Xmx1g -J-Xms200m" \
export EXTRA_BAZEL_ARGS="--host_javabase=@local_jdk//:jdk"  \
./compile.sh


# Successful compilation on ARM64 - 2cores, 4GB

INFO: Elapsed time: 1449.174s, Critical Path: 139.99s
INFO: 1777 processes: 1538 local, 239 worker.
INFO: Build completed successfully, 1824 total actions

prestonvanloon · 2019-06-19T04:42:15Z

Thanks @jbriquet! My last comment is referring to cross complation. We’d love to build ARM64 binaries for you without unreasonable build environment, but I am glad to see that you were able to build one after 1450s(!!!).

ARM builds will soon be a priority for us and we’ll be providing regular ARM binaries for tagged releases soon :)

diglos · 2019-06-19T07:38:29Z

Great news!
I'm releasing beacon-chain and validator binaries in the next Ethereum on ARM release for NanoPC-T4 and RockPro64. So far, they work quite well :-)

rauljordan · 2019-07-23T02:17:37Z

This is resolved by #2970

prestonvanloon · 2019-07-24T17:53:23Z

Verifying that building on linux amd64 for arm64 works!

bazel build //beacon-chain //validator --platforms=@io_bazel_rules_go//go/toolchain:linux_arm64

prestonvanloon · 2019-10-03T22:30:06Z

Recent BLS PR has broken ARM builds #3670. Reopening this issue for tracking.

diglos · 2019-11-19T15:33:01Z

Hi,
I just ran into this issue trying to compile Prysm for the Raspberry Pi 4. Thanks for reopening this.

diglos · 2019-11-29T12:16:54Z

Hi,
Any ETA to fix this?
Thanks!

nisdas · 2019-12-05T11:04:08Z

hey @diglos, with #4181 merged in, we have ARM support added in

diglos · 2019-12-05T14:05:06Z

Hi,
Thanks @nisdas.
It compiles on ARM64 (NanoPC-T4) but fails on ARM32 (Raspberry Pi 4) for both, beacon chain and validator. Please check attached log.

prysm-arm7-error.txt

nisdas · 2019-12-06T17:39:42Z

hey @diglos , thanks for testing this out. It seems like we might not be giving the correct archive for bls, I will try and see where it fails

wikidemic · 2019-12-15T15:13:30Z

There’s an army of RPi4 standing by to join test network.
Any progress towards successful compile yet?

nisdas · 2019-12-19T07:36:49Z

Hey sorry currently we haven't been able to get it to work on arm32 yet, only arm64 is currently supported. I havent been able to dedicate the time to resolving this, I hope this can be done soon

mangoleap · 2019-12-19T21:31:13Z

I'm trying to get it to run on a NanoPC T4 (ARM 64). Installed docker and pulled the images.

When I run

docker run -v $HOME/prysm-data:/data -p 4000:4000 --name beacon-node gcr.io/prysmaticlabs/prysm/beacon-chain:latest --no-genesis-delay --datadir=/data

I get the error

standard_init_linux.go:211: exec user process caused "exec format error"

That's all, nothing else. Any idea how to proceed?

wikidemic · 2019-12-20T03:10:28Z

I

I'm trying to get it to run on a NanoPC T4 (ARM 64). Installed docker and pulled the images.

When I run

docker run -v $HOME/prysm-data:/data -p 4000:4000 --name beacon-node gcr.io/prysmaticlabs/prysm/beacon-chain:latest --no-genesis-delay --datadir=/data

I get the error

standard_init_linux.go:211: exec user process caused "exec format error"

That's all, nothing else. Any idea how to proceed?

@diglos seems to be on the ball here. If you don’t answer for your docker image question, suggest you look up diglox image here on github - diglos/userpatches.

I’m considering NanoPC T4 purchase from Friendly Arm this week. So I’m tracking developer responsiveness here before I pull trigger. Let us know if you make progress!

prestonvanloon · 2019-12-20T03:26:20Z

The current docker image is compiled for AMD64 so it will not work on ARM64. It's not compiled for ARM and I am not sure about any workaround.

To give you guys a sense of timeline, we most likely won't get to this until after the holiday.

We'd welcome anyone to attempt a resolution here. I have a few ideas if someone wants to attempt resolving builds for ARM64.

diglos · 2019-12-20T10:35:42Z

Hi @mangoleap.

Are you running a Ethereum on ARM image?. If so, you can install it by typing (last version):

sudo apt-get update && sudo apt-get install prysm-beacon prysm-validator

@prestonvanloon, @wikidemic, it compiles fine on ARM64 (NanoPC-T4 and RockPro64) after #4181 merged. ARM7 (Raspberry Pi 4), doesn't work at the moment.

mangoleap · 2019-12-20T11:04:21Z

@diglos That worked, thank you!

If anyone wants to try it:

Config files are at

/etc/ethereum/prysm-beacon.conf
/etc/ethereum/prysm-validator.conf

Chain and validator can be started with

sudo systemctl start prysm-beacon
sudo systemctl start prysm-validator

Auto start up on boot can be enabled with

sudo systemctl enable prysm-beacon.service
sudo systemctl enable prysm-validator.service

Real time output from both services can be watched with

sudo journalctl -u prysm-beacon -f
sudo journalctl -u prysm-validator -f

rauljordan · 2020-03-06T23:34:05Z

Done by @SuburbanDad and included in our https://docs.prylabs.network

jgresham · 2020-03-06T23:35:36Z

Link to install docs: https://docs.prylabs.network/docs/install/arm/

wikidemic · 2020-03-08T06:54:11Z

I just attempted to load the ‘Unofficial’ version(2.1.0) of Bazel for ARM architecture using docs below. Unfortunately, it did not work as the current version is now at 2.2 and does not incorporate this Arm edit (by SuburbanDad) I haven’t coded in quite a while so I cann’t offer too much detail other than saying Bazel doesn’t load on my RPi4. Installation script error notes that Unofficial Bazel isn’t current version and references a config file that requires editing before it is happy. Hope that helps. Mc

…

Sent from my iPad

On Mar 6, 2020, at 6:35 PM, Johns Gresham ***@***.***> wrote: Link to install docs: https://docs.prylabs.network/docs/install/arm/ — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

SuburbanDad · 2020-03-08T11:56:09Z

@wikidemic prysm has pinned to bazel 2.1.1 for now. I added an arm64 build of 2.1.1:
https://github.com/SuburbanDad/bazel/releases/tag/2.1.1-arm64

give it a whirl

jgresham · 2020-03-15T22:39:49Z

Edit: It appears that although the raspi4 comes with a 64 bit cpu, then default OS (Raspbian Buster) on raspi4 is 32 bit. Is there any particular 64 bit OS that you all would recommend for the raspi4?

@SuburbanDad Should the bazel binaries on your releases work on a raspi4? I having issues after running step 3 from here https://docs.prylabs.network/docs/install/arm/ so I am trying to build prysm from source.

Here is the result from trying to use your bazel binaries on raspi4:

pi@raspberrypi:~/prysm $ ./bazel --version
-bash: ./bazel: cannot execute binary file: Exec format error
pi@raspberrypi:~/prysm $ file bazel
bazel: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, BuildID[sha1]=09b54219645538b60aaaf5829af803cf3fad9fd4, not stripped

Here is result from trying to run prysm on a raspi4:

pi@raspberrypi:~/prysm $ ./prysm.sh --clear-db --datadir=$HOME/prysm
pi@raspberrypi:~/prysm $ ls dist/
beacon-chain-v0.3.4-linux-armv7l  validator-v0.3.4-linux-armv7l
pi@raspberrypi:~/prysm $ ./prysm.sh beacon-chain
Latest Prysm version is v0.3.4.
Beacon chain is up to date.
Validator is up to date.
Starting Prysm beacon-chain
/home/pi/prysm/dist/beacon-chain-v0.3.4-linux-armv7l: line 1: Not: command not found

In the meantime, I am trying to build bazel 2.1.1 on a raspi4.

SuburbanDad · 2020-03-15T23:21:54Z

Hey @jgresham, Edited after reading your edit: the unofficial / dev bazel binaries I have here: https://github.com/SuburbanDad/bazel/releases should work on any debian based arm64 os. Ubuntu 19.10 is probably the easiest route. Personally I am running an unofficial distro of 18.04 from: https://jamesachambers.com/raspberry-pi-4-ubuntu-server-desktop-18-04-3-image-unofficial/ Since it has pi-specific kernel optimizations from raspbian. I am hopeful that 20.04 will have better support when it comes out next month.

…

On Sun, Mar 15, 2020, 3:40 PM Johns Gresham ***@***.***> wrote: @SuburbanDad <https://github.com/SuburbanDad> Should the bazel binaries on your releases work on a raspi4? I having issues after running step 3 from here https://docs.prylabs.network/docs/install/arm/ so I am trying to build prysm from source. Here is the result from trying to use your bazel binaries on raspi4: ***@***.***:~/prysm $ ./bazel --version -bash: ./bazel: cannot execute binary file: Exec format error ***@***.***:~/prysm $ file bazel bazel: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, BuildID[sha1]=09b54219645538b60aaaf5829af803cf3fad9fd4, not stripped Here is result from trying to run prysm on a raspi4: ***@***.***:~/prysm $ ./prysm.sh --clear-db --datadir=$HOME/prysm ***@***.***:~/prysm $ ls dist/ beacon-chain-v0.3.4-linux-armv7l validator-v0.3.4-linux-armv7l ***@***.***:~/prysm $ ./prysm.sh beacon-chain Latest Prysm version is v0.3.4. Beacon chain is up to date. Validator is up to date. Starting Prysm beacon-chain /home/pi/prysm/dist/beacon-chain-v0.3.4-linux-armv7l: line 1: Not: command not found In the meantime, I am trying to build bazel 2.1.1 on a raspi4. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2546 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOSEBJRG27FVPNPOED4ACD3RHVKMHANCNFSM4HL3T3WA> .

amaurer · 2020-03-28T19:25:34Z

I found that the prysm.sh attempted download of the binaries failed from https://api.github.com/repos/prysmaticlabs/prysm/releases/latest

Line one of the binary file was "Not Found" which resulted in an error - line 1: Not: command not found

Turns out I was hitting the rate-limit imposed by github for unauthenticated requests.

{"message":"API rate limit exceeded for x.x.x.x . (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)","documentation_url":"https://developer.github.com/v3/#rate-limiting"}

I authenticated using curl -u "username" https://api.github.com/repos/prysmaticlabs/prysm/releases/latest -o ./releases.json and edited the curl line in prysm.sh to use the json file instead. Beacon node is up and running now

prestonvanloon added CI Continuous integration related items Enhancement New feature or request labels May 10, 2019

prestonvanloon added a commit that referenced this issue May 10, 2019

trying to get ARM to compile for #2546

ea790c5

prestonvanloon mentioned this issue May 25, 2019

Update BLS #2692

Merged

prestonvanloon mentioned this issue Jun 24, 2019

Add support for cross compilation in Bazel #2849

Closed

prestonvanloon mentioned this issue Jul 15, 2019

Add support for bundling binaries and fix ARM64 builds #2970

Merged

rauljordan closed this as completed Jul 23, 2019

prestonvanloon reopened this Oct 3, 2019

prestonvanloon mentioned this issue Oct 3, 2019

Feature Request: ARM support kilic/bls12-381#2

Closed

prestonvanloon added the Help Wanted Extra attention is needed label Dec 20, 2019

rauljordan closed this as completed Mar 6, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add ARM support #2546

Add ARM support #2546

penta314 commented May 9, 2019

prestonvanloon commented May 10, 2019

prestonvanloon commented May 10, 2019

diglos commented May 13, 2019

prestonvanloon commented Jun 3, 2019

diglos commented Jun 4, 2019

SoySauceFor3 commented Jun 18, 2019

diglos commented Jun 18, 2019

SoySauceFor3 commented Jun 18, 2019

prestonvanloon commented Jun 19, 2019 •

edited

Loading

jbriquet commented Jun 19, 2019

prestonvanloon commented Jun 19, 2019

diglos commented Jun 19, 2019

rauljordan commented Jul 23, 2019

prestonvanloon commented Jul 24, 2019 •

edited

Loading

prestonvanloon commented Oct 3, 2019

diglos commented Nov 19, 2019

diglos commented Nov 29, 2019

nisdas commented Dec 5, 2019 •

edited

Loading

diglos commented Dec 5, 2019

nisdas commented Dec 6, 2019

wikidemic commented Dec 15, 2019

nisdas commented Dec 19, 2019

mangoleap commented Dec 19, 2019

wikidemic commented Dec 20, 2019

prestonvanloon commented Dec 20, 2019 •

edited

Loading

diglos commented Dec 20, 2019 •

edited

Loading

mangoleap commented Dec 20, 2019

rauljordan commented Mar 6, 2020

jgresham commented Mar 6, 2020

wikidemic commented Mar 8, 2020 via email

SuburbanDad commented Mar 8, 2020 •

edited

Loading

jgresham commented Mar 15, 2020 •

edited

Loading

SuburbanDad commented Mar 15, 2020 via email •

edited

Loading

amaurer commented Mar 28, 2020 •

edited

Loading

Add ARM support #2546

Add ARM support #2546

Comments

penta314 commented May 9, 2019

prestonvanloon commented May 10, 2019

prestonvanloon commented May 10, 2019

diglos commented May 13, 2019

prestonvanloon commented Jun 3, 2019

diglos commented Jun 4, 2019

SoySauceFor3 commented Jun 18, 2019

diglos commented Jun 18, 2019

SoySauceFor3 commented Jun 18, 2019

prestonvanloon commented Jun 19, 2019 • edited Loading

jbriquet commented Jun 19, 2019

prestonvanloon commented Jun 19, 2019

diglos commented Jun 19, 2019

rauljordan commented Jul 23, 2019

prestonvanloon commented Jul 24, 2019 • edited Loading

prestonvanloon commented Oct 3, 2019

diglos commented Nov 19, 2019

diglos commented Nov 29, 2019

nisdas commented Dec 5, 2019 • edited Loading

diglos commented Dec 5, 2019

nisdas commented Dec 6, 2019

wikidemic commented Dec 15, 2019

nisdas commented Dec 19, 2019

mangoleap commented Dec 19, 2019

wikidemic commented Dec 20, 2019

prestonvanloon commented Dec 20, 2019 • edited Loading

diglos commented Dec 20, 2019 • edited Loading

mangoleap commented Dec 20, 2019

rauljordan commented Mar 6, 2020

jgresham commented Mar 6, 2020

wikidemic commented Mar 8, 2020 via email

SuburbanDad commented Mar 8, 2020 • edited Loading

jgresham commented Mar 15, 2020 • edited Loading

SuburbanDad commented Mar 15, 2020 via email • edited Loading

amaurer commented Mar 28, 2020 • edited Loading

prestonvanloon commented Jun 19, 2019 •

edited

Loading

prestonvanloon commented Jul 24, 2019 •

edited

Loading

nisdas commented Dec 5, 2019 •

edited

Loading

prestonvanloon commented Dec 20, 2019 •

edited

Loading

diglos commented Dec 20, 2019 •

edited

Loading

SuburbanDad commented Mar 8, 2020 •

edited

Loading

jgresham commented Mar 15, 2020 •

edited

Loading

SuburbanDad commented Mar 15, 2020 via email •

edited

Loading

amaurer commented Mar 28, 2020 •

edited

Loading