Bonsai on Raspberry Pi

[roaldarbol]

Why run Bonsai on Raspberry?

Raspberry Pi's are becoming increasingly popular in research labs (e.g. autopilot). Here are a few advantages to them:

• Cheap
• Portable
• Access to CSI cameras
• SSH maybe possible for operating experiments outside the lab?

I've started testing the new Bonsai 2.8 on my Raspberry Pi 4 8GB, and thought I'd share my experiences here (and maybe get some useful feedback). Since the RPi can run both the Raspberry Pi OS and Ubuntu, I tried with both different OS's - here are my take-aways.
I'll update continuously.

Testing

I'm trying to be consistent with the installation steps across OS to get a good idea about what works and what needs more work... to work... For clarity, I've written the steps taken along with potential error messages. Let's dive in!

Steps

Preparation

• Fresh OS installation with Raspberry Pi Imager
• sudo apt update & sudo apt upgrade
• ls /dev/video* to see if CSI camera is picked up (or simply libcamera-hello on the RPi OS's)

Installation (for Linux)

• Install Mono
  • sudo apt install mono-complete -y
  • Test Mono installation
• Download and run Bonsai
  • Install extension pack
  • Install Video extension pack
• Install OpenGL
• Install OpenAL
• Install OpenCV
  • From binaries, or
  • From source
• Run VideoCaptureDevice

Raspberry Pi OS (32-bit)

• ✅ Install Mono
• ⛔️ Donwload and run Bonsai
  Can't open Bonsai (also tried Bonsai32.exe with the same outcome):

Error message

mikkel@raspberrypi:~/Documents/Bonsai $ mono Bonsai.exe

=================================================================
	Native Crash Reporting
=================================================================
Got a SIGSEGV while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries 
used by your application.
=================================================================

=================================================================
	Native stacktrace:
=================================================================
	 (No frames) 


=================================================================
	Telemetry Dumper:
=================================================================
Pkilling 0xf76bf440 from 0xf7e9d040
Entering thread summarizer pause from 0xf7e9d040
Finished thread summarizer pause from 0xf7e9d040.

Waiting for dumping threads to resume

=================================================================
	External Debugger Dump:
=================================================================
[New LWP 4800]
[New LWP 4801]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".
0xf7ca4034 in __GI___wait4 (pid=4806, stat_loc=0xffe8b008, options=0, usage=0x0) at ../sysdeps/unix/sysv/linux/wait4.c:27
27	../sysdeps/unix/sysv/linux/wait4.c: No such file or directory.
  Id   Target Id                                  Frame 
* 1    Thread 0xf7e9d040 (LWP 4799) "mono"        0xf7ca4034 in __GI___wait4 (pid=4806, stat_loc=0xffe8b008, options=0, usage=0x0) at ../sysdeps/unix/sysv/linux/wait4.c:27
  2    Thread 0xf6fff440 (LWP 4800) "SGen worker" futex_wait_cancelable (private=0, expected=0, futex_word=0x447698) at ../sysdeps/nptl/futex-internal.h:186
  3    Thread 0xf76bf440 (LWP 4801) "Finalizer"   futex_abstimed_wait_cancelable (private=0, abstime=0x0, clockid=0, expected=1, futex_word=0x43f73c) at ../sysdeps/nptl/futex-internal.h:323

Thread 3 (Thread 0xf76bf440 (LWP 4801) "Finalizer"):
#0  futex_abstimed_wait_cancelable (private=0, abstime=0x0, clockid=0, expected=1, futex_word=0x43f73c) at ../sysdeps/nptl/futex-internal.h:323
#1  do_futex_wait (sem=sem@entry=0x43f73c, abstime=0x0, clockid=0) at sem_waitcommon.c:117
#2  0xf7d93c90 in __new_sem_wait_slow (sem=0x43f73c, abstime=0x0, clockid=0) at sem_waitcommon.c:285
#3  0x002c2120 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 2 (Thread 0xf6fff440 (LWP 4800) "SGen worker"):
#0  futex_wait_cancelable (private=0, expected=0, futex_word=0x447698) at ../sysdeps/nptl/futex-internal.h:186
#1  __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x0, cond=0x447670) at pthread_cond_wait.c:508
#2  __pthread_cond_wait (cond=0x447670, mutex=0x0) at pthread_cond_wait.c:638
#3  0x00327c84 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 1 (Thread 0xf7e9d040 (LWP 4799) "mono"):
#0  0xf7ca4034 in __GI___wait4 (pid=4806, stat_loc=0xffe8b008, options=0, usage=0x0) at ../sysdeps/unix/sysv/linux/wait4.c:27
#1  0x0009f73c in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
[Inferior 1 (process 4799) detached]

=================================================================
	Basic Fault Address Reporting
=================================================================
Memory around native instruction pointer (0xf7d915c0):0xf7d915b0  bc ff ff ea 98 70 00 00 f0 4f 2d e9 14 d0 4d e2  .....p...O-...M.
0xf7d915c0  24 50 90 e5 a5 31 b0 e1 28 01 00 0a 00 c0 a0 e1  $P...1..(.......
0xf7d915d0  01 50 15 e2 20 30 bc e5 00 e0 a0 e1 80 50 a0 13  .P.. 0.......P..
0xf7d915e0  0a 00 00 ea 01 10 83 e3 9f 2f 9c e1 03 00 52 e1  ........./....R.

=================================================================
	Managed Stacktrace:
=================================================================
	  at <unknown> <0xffffffff>
	  at System.Windows.Forms.Application:EnableVisualStyles <0x00033>
	  at Bonsai.EditorBootstrapper:EnableVisualStyles <0x0001f>
	  at Bonsai.EditorBootstrapper:RunPackageOperationAsync <0x0006b>
	  at <RunAsync>d__7:MoveNext <0x0095b>
	  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder:Start <0x000d7>
	  at Bonsai.Configuration.Bootstrapper:RunAsync <0x0018f>
	  at Bonsai.Program:Main <0x0167b>
	  at <Module>:runtime_invoke_int_object <0x00113>
=================================================================
Aborted

Raspberry Pi OS (32-bit) on SSD

• ⛔️ Install Mono

Unable to install Mono. Will try again on a fresh installation.

Error messages

sudo apt install mono-complete -y     
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 mono-complete : Depends: mono-runtime (= 6.12.0.147-0xamarin17+ubuntu2004b1) but it is not going to be installed
                 Depends: mono-runtime-sgen (= 6.12.0.147-0xamarin17+ubuntu2004b1) but it is not going to be installed
                 Depends: mono-utils (= 6.12.0.147-0xamarin17+ubuntu2004b1) but it is not going to be installed
                 Depends: mono-devel (= 6.12.0.147-0xamarin17+ubuntu2004b1) but it is not going to be installed
                 Depends: mono-mcs (= 6.12.0.147-0xamarin17+ubuntu2004b1) but it is not going to be installed
                 Depends: mono-roslyn (= 6.12.0.147-0xamarin17+ubuntu2004b1) but it is not going to be installed
                 Depends: mono-csharp-shell (= 6.12.0.147-0xamarin17+ubuntu2004b1) but it is not going to be installed
                 Depends: mono-4.0-gac (= 6.12.0.147-0xamarin17+ubuntu2004b1) but it is not going to be installed
                 Depends: mono-4.0-service (= 6.12.0.147-0xamarin17+ubuntu2004b1) but it is not going to be installed
                 Depends: monodoc-base (= 6.12.0.147-0xamarin17+ubuntu2004b1) but it is not going to be installed
                 Depends: monodoc-manual (= 6.12.0.147-0xamarin17+ubuntu2004b1) but it is not going to be installed
                 Depends: libmono-cil-dev (= 6.12.0.147-0xamarin17+ubuntu2004b1) but it is not going to be installed
                 Depends: referenceassemblies-pcl but it is not going to be installed
                 Depends: msbuild but it is not going to be installed
                 Depends: ca-certificates-mono (= 6.12.0.147-0xamarin17+ubuntu2004b1) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

Raspberry Pi OS (64-bit)

• ✅ Install Mono
• ✅ Download and run Bonsai
• ✅ Install OpenGL
• ✅ Install OpenAL
• ⛔️ Install OpenCV
  • ⛔️ From binaries
  • ✅ Add release repositories for libjasaper
  • ⛔️ Download and install pre-built binaries
  • From source (on-going)
• ⛔️ Run VideoCaptureDevice

Mostly works! However, the CSI camera is not being picked up by Bonsai, possibly because OpenCV isn't install (@glopesdev does VideoCaptureDevice depend on OpenCV?).

Ubuntu Desktop 23.04 (64-bit)

• ✅ Install Mono
• ✅ Download and run Bonsai
• ✅ Install OpenGL
• ✅ Install OpenAL
• ⛔️ Install OpenCV
  • ⛔️ From binaries
    • ✅ Add release repositories for libjasaper
    • ⛔️ Download and install pre-built binaries
  • From source, not tested yet.
• ⛔️ Run VideoCaptureDevice

Mostly works! However, I am trying to record video with a RPi HQ camera (actually an Arducam IR HQ camera), and Bonsai doesn't pick it up. I'll try with another CSI camera and see if that works.

Final comments

OpenCV installation

• To build OpenCV from source, you also need to install CMAKE (sudo apt install cmake).
• Depending on where you have cloned the repository, change cd ~/opencv to cd opencv.
• Prepare for lots of warnings, and wait a while.

[glopesdev]

@roaldarbol Thank you for your feedback, it will be great to compare notes. I was able to install and run the new Bonsai 2.8 on my RPi 4 Model B just before publishing this release and it ran fine on the latest RPi OS 32-bit. I downloaded the latest Imager and flashed a clean SSD just to be sure.

I did have to run the bootstrapper using mono Bonsai32.exe rather than use the normal Bonsai.exe otherwise it would halt and eventually crash I believe with a similar error message (but cannot recall, was planning to test again this weekend).

I was able to get so far as to get OpenGL and OpenCV running loading images, and I believe I was also able to start a USB camera last time, but I need to try and rebuild everything again so we can compare notes. I will reply here with notes on progress and I can also publish some .deb packages to make it easier to install OpenCV (the build does take a long time!).

[dlhagger]

Jumping in here - might be the same or different - but I have Bonsai running on a Jetson Nano NVIDA development board (aarch64) with 18.04 ubuntu OS installed. I've been able to follow all the Linux install steps, and have built openCV with CUDA support successfully (I have tested with a python script to stream webcam footage and save to a file) - but for the life of me cannot seem to get CameraCapture or VideoCaptureDevice to recognize the camera. Any thoughts? How were you able to get CameraCatpure working on Linux? Would really like to get this up and running to see what kind of performance I can get on an edge compute device with CUDA support for capturing animal behavior experiments.

[chiyu1203]

I am curious about this discussion. Did you manage to get Jetson Nano working with CameraCapture?

Jumping in here - might be the same or different - but I have Bonsai running on a Jetson Nano NVIDA development board (aarch64) with 18.04 ubuntu OS installed. I've been able to follow all the Linux install steps, and have built openCV with CUDA support successfully (I have tested with a python script to stream webcam footage and save to a file) - but for the life of me cannot seem to get CameraCapture or VideoCaptureDevice to recognize the camera. Any thoughts? How were you able to get CameraCatpure working on Linux? Would really like to get this up and running to see what kind of performance I can get on an edge compute device with CUDA support for capturing animal behavior experiments.

[dlhagger]

Gave up on this - couldn't get it to work but would like to come back to it one day because the idea of CUDA acceleration on a edge compute that can run Bonsai would make me really happy. Have you had any luck yourself?

[chiyu1203]

Same here! I have not had any luck so far. I can not get OpenCV-related packages (such as CameraCapture) and BonVision packages working in both Raspbery PI and a Ubuntu PC. I was surprised by my struggle with the Ubuntu PC since there was some discussion threads. However, my PC is too new (Ubuntu 22.04) for OpenCV2.4 and the installation became buggy. Maybe I should install Windows (+.NET Framework) in the Raspberry Pi and see if that works

[dlhagger]

Let me know if you get anything to work?

[aaronsathya]

This would be absolutely a game-changer for bonsai. Any further updates on this?