use pybind11 instead of boost.python

[2bndy5]

I've been looking heavily into pybind11 as an alternative to boost.python because it would simplify the build process's dependency on having libboost-python-dev installed. pybind11 is a minimized (header-only) offshoot of boost.python. It doesn't need to be installed, rather it is recommended to be used as a git submodule in the repo for a python package that wraps C++ into python (or vice versa).

I want to rewrite the python wrappers using pybind11 so the build process is simplified.

pybind11 can be found on github here
pybind11 docs are hosted on rtd.io
objections? discussion?

[TMRh20]

Sounds good to me, let’s just leave the boost stuff in place until this is tested and ready.

…

On Nov 12, 2020, at 6:22 PM, Brendan ***@***.***> wrote:  I've been looking heavily into pybind11 as an alternative to boost.python because it would simplify the build process's dependency on having libboost-python-dev installed. pybind11 is a minimized (header-only) offshoot of boost.python. It doesn't need to be installed, rather it is recommended to be used as a git submodule in the repo for a python package that wraps C++ into python (or vice versa). I want to rewrite the python wrappers using pybind11 so the build process is simplified. objections? discussion? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[2bndy5]

agreed (naturally)

[2bndy5]

1. I started working on this in my fork's add-submodules branch. However, I think I'll hold off and wait till pybind11 releases v2.7.0 because we need RF24::read() to return a mutable bytearray object, and pybind11 currently only has type conversion for returning a python bytes object (which is immutable). I do see this type conversion to bytearray feature has been merged to pybind11's working branch and added to the v2.7.0 milestone. If I push forward by returning a bytes object, then the end-user would have to use the copy c'tor to convert RF24::read()'s result as a bytearray:

   received = bytearray(radio.read())

   which would be inconsistent with the current boost.python implementation, which directly uses Python's C API to do the conversion (I'm guessing boost.python also doesn't/didn't have explicit type casting to a bytearray object).
2. I'm also interested in possibly pre-defining IF_SERIAL_DEBUG to allow python users to use python's native logging std lib, but it seems that would need to involve embedding an instance of the python interpreter... Maybe for a later time when we have hammered out the rest of RF24Log lib. I was also imagining the similar use case where the python user could use a python lib for SPI transactions (like the adafruit_pureio library) for a soft SPI if needed... Again this point is much further down the line (not an immediate concern).

[2bndy5]

1. pybind11 v2.7.0 is released now (with type conversion to bytearrays) 🎉 !
2. can wait till the package is building with manylinux CI

[2bndy5]

I'm making progress on this issue. Currently all the new wrappers build with CMake... Orienting setup.py should be a breeze now. After that obstacle, I just have to start hardware testing on a actual RPi. Note the build CI has not been updated yet.

@TMRh20 Just a heads up: I had to introduce an undefined macro named USE_RF24_LIB_SRC for RF24Network and RF24Mesh libs. This new macro (only defined via pyRF24/CMakeLists.txt) allows the libs to build against the submodule sources instead of looking for the install locations (/usr/local/include/RF24*). See these commits for the changes to source code:

1. nRF24/RF24Network@2a36d4b
2. nRF24/RF24Mesh@c88e79d

The new macro should also be handy for user projects that don't have RF24 libs installed, but include the repos as submodules like this repo will.

All progress is happening on my fork's add-submodules branch

[TMRh20]

Sounds good

…

On Aug 13, 2021, at 5:53 PM, Brendan ***@***.***> wrote:  I'm making progress on this issue. Currently all the new wrappers build with CMake... Orienting setup.py should be a breeze now. After that obstacle, I just have to start hardware testing on a actual RPi. Note the build CI has not been updated yet. @TMRh20 Just a heads up: I had to introduce an undefined macro named USE_RF24_LIB_SRC for RF24Network and RF24Mesh libs. This new macro (only defined via pyRF24/CMakeLists.txt) allows the libs build against the submodule sources instead of looking for the install locations (/usr/local/include/RF24*). See these commits for the changes to source code: ***@***.*** ***@***.*** The new macro should also be handy for user projects that don't have RF24 libs installed, but include the repos as submodules like this repo will. All progress is happening on my fork's add-submodules branch — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

[2bndy5]

quick update: I've been able to successfully build and import the python package (on my add-submodules branch). However, handling buffer protocol objects to/from python is broken. I've also got docs building in sphinx from the package source code's doc-strings (which means help(RF24.<func or attr>) will show the same docs in the REPL 👍🏼).

[2bndy5]

I now have the pyrf24 pkg building and installing w/o problems. So far, the rf24 module is working, but now I need to run HW tests for rf24network and rf24mesh modules.

Building the pkg is hard coded to use the SPIDEV driver because it doesn't require sudo privileges.

[2bndy5]

ITS WORKING! 🎉

I'm excited because of all the build system tribulations, segmentation faults, and misunderstanding of pybind11 docs that I had to overcome. I just verified all examples for all network layers are functioning as expected. 💯

I had so much fun with the core rf24 module that I decided to port my FakeBLE API from the CirPy lib into this pkg. It was more of an experiment with trying to derive fake BLE functionality from the rf24 binding (via inheritance). Ultimately, due to tribulations with python's super() resolution, the fake_ble module (written in pure python) wraps around the rf24 module (like network and mesh API does) and is distributed as part of the pyrf24 pkg. Also includes an example that demonstrates TX/RX-ing BLE payloads... 🥇