Python Wrapper for realsense-net

[xlDownxl]

Hello everyone,

I would like to use my RealSense camera D455 with a Raspberry Pi over an Ethernet connection as it is described on this page by Intel: https://dev.intelrealsense.com/docs/open-source-ethernet-networking-for-intel-realsense-depth-cameras.

Instead of the C++ used in the Tutorial, I would like to use Python, which brings me to my problem: How do I run the commands #include <librealsense2-net/rs_net.hpp> and rs2::net_device dev("10.0.0.99"); in Python?
I installed the librealsense library from source with the -DBUILD_NETWORK_DEVICE=ON and -DBUILD_PYTHON_BINDINGS on my Ubuntu 16 host machine, but I simply do not know how to access the realsense-net module and its functions.

I would be glad if someone could help me out!

[MartyG-RealSense]

Hi @xlDownxl You may find it easier to use the earlier EtherSense RealSense ethernet networking project in the link below, which is based around Python by default.

https://dev.intelrealsense.com/docs/depth-camera-over-ethernet-whitepaper

https://github.com/IntelRealSense/librealsense/tree/master/wrappers/python/examples/ethernet_client_server

In regard to the open-source ethernet networking project, it states that "integration with Python, Android and other wrappers in librealsense ecosystem will be provided in future releases based on feedback from the community".  There has been a case where it was successfully connected to from the C# wrapper, but I am not aware of a similar example for Python.

#7848

[MartyG-RealSense]

Hi @xlDownxl Do you require further assistance with this case, please? Thanks!

[xlDownxl]

Thank you @MartyG-RealSense for your support!

I used the proposed Ethersense code to a great success. Still, it would be great to see the Python wrappers for those functions in the SDK integrated in future releases.

[MartyG-RealSense]

Thats very much @xlDownxl for the update!

[mirkocomparetti-synesis]

@xlDownxl how did you manage to make the ethersense work?
It does not really work for us and we are having issues with the python library IntelRealSense/realsense-ros#1804 (comment)
How did you compile the libraries? Which options did you select?
Thanks!

[xlDownxl]

Hey @mirkocomparetti-synesis,

I encountered the same issue, but I think to replace import pyrealsense2 with import pyrealsense2.pyrealsense2 as rs fixed the error when trying to call the pipeline() method.
Maybe try that out and let me know if that worked.

[mirkocomparetti-synesis]

Thanks @xlDownxl
I just tried but still no luck

Python 3.8.5 (default, Jan 27 2021, 15:41:15) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyrealsense2.pyrealsense2 as rs 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'pyrealsense2.pyrealsense2'

while import pyrealsense2 as rs works fine.

On another ubuntu machine (amd64) I installed the libraries from the repos and the rs.pipeline() object exists.

We assume that the way to build the libraries is somehow corrupted and it does not build everything.
Which version of realsense repos did you build?
Thanks!

[xlDownxl]

Maybe try to add the flag -DFORCE_RSUSB_BACKEND=ON for the build as proposes here: https://dev.intelrealsense.com/docs/open-source-ethernet-networking-for-intel-realsense-depth-cameras.
If that is not changing anything, the only thing that might help is to use the linux image they provide at the link above for the pi.

[mirkocomparetti-synesis]

Thanks @xlDownxl !
I'll try with that flag as well and I'll let you know. We started from their image in the first place, but with no luck. Anyway some time has passed, so we will try that again as well.
Thanks!

[mirkocomparetti-synesis]

@xlDownxl so, that flag didn't change... the problem is the python library that gets partially compiled and those are the only available methods

>>> pyrealsense2.__
pyrealsense2.__class__(          pyrealsense2.__file__            pyrealsense2.__init__(           pyrealsense2.__ne__(
pyrealsense2.__repr__(          pyrealsense2.__delattr__(        pyrealsense2.__format__(         pyrealsense2.__init_subclass__(
pyrealsense2.__new__(            pyrealsense2.__setattr__(          pyrealsense2.__dict__            pyrealsense2.__ge__(
pyrealsense2.__le__(             pyrealsense2.__package__         pyrealsense2.__sizeof__(          pyrealsense2.__dir__(
pyrealsense2.__getattribute__(   pyrealsense2.__loader__          pyrealsense2.__path__            pyrealsense2.__spec__
pyrealsense2.__doc__             pyrealsense2.__gt__(             pyrealsense2.__lt__(             pyrealsense2.__reduce__(
pyrealsense2.__str__(          pyrealsense2.__eq__(             pyrealsense2.__hash__(           pyrealsense2.__name__
pyrealsense2.__reduce_ex__(      pyrealsense2.__subclasshook__(

it looks like the cmakefile is broken and brings to an incomplete, yet succesfull, build.
@RealSense-Customer-Engineering @RealSenseCustomerSupport @RealSenseSupport

[MartyG-RealSense]

Hi @mirkocomparetti-synesis You mention above that you are using an amd64 computer with Python 3.8.5. If you have not tried this already, there is now the option since librealsense SDK version 2.43.0 to use pip install pyrealsense2 with Python 3.8 and 3.9 to install the Python wrapper on x86 / x64 computers (e.g desktop and laptop computers) instead of building the wrapper from source.

#8229 (comment)

[mirkocomparetti-synesis]

Thank you @MartyG-RealSense
In fact, the build problem is only on the raspberrypi, while I wrote #8564 (comment) that on the amd64 architecture it works fine from repos.
For us the raspberrypi is mandatory.

[MartyG-RealSense]

If you are using the Raspbian OS with Pi 4, I would recommend using a librealsense installation guide that is designed specifically for Pi 4. A popular guide with RealSense users of Pi 4 is the Acrobotic one, which is similar in structure to Intel's official Raspbian guide for Pi 3.

https://github.com/acrobotic/Ai_Demos_RPi/wiki/Raspberry-Pi-4-and-Intel-RealSense-D435

It is worth mentioning that some RealSense Pi users have had build problems with using the protobuf versions suggested in these guides and have had success with using more modern versions such as protobuf 3.14 instead.

[mirkocomparetti-synesis]

@MartyG-RealSense I'll have a look at it.
The only compilation problem is about python and thus ros; the rs-server works fine on the raspberry pi.

So I guess the library itself is somehow broken. More details here IntelRealSense/realsense-ros#1804 (comment)

[MartyG-RealSense]

That comment says that the error that you are experiencing is Module pyrealsense2 has no attribute 'pipeline'. The most recent Pi case I handled with this Python error is in the link below.

#6986 (comment)

In that case, where the RealSense user had used the Acrobotic installation guide, they found that the error occurred if they used the Python IDE Thonny but did not occur when running in the terminal.

[mirkocomparetti-synesis]

At the moment I'm on ubuntu because the official build instructions were ubuntu centered, but I'm dowloading raspios as we speak and I'll try with your guide.
In my case, anyway, the problem is in the terminal, as per my screenshot in IntelRealSense/realsense-ros#1804 (comment)

Thanks!

[MartyG-RealSense]

Using Ubuntu with RealSense on Pi has its advantages. For example, the only successful builds of the RealSense ROS wrapper on Pi 4 that I know of have been on Ubuntu Server instead of Raspbian OS. A downside is that package files (such as pip pyrealsense2 packages and librealsense Debian install packages) do not work on devices with Arm processors such as Raspberry Pi and Nvidia Jetson, though that applies to Raspbian OS too of course.

In regard to the image of the terminal, I have never known import pyrealsense2.pyrealsense2 as rs to work. It is always written in programs as import pyrealsense2 as rs

I look forward to hearing about how your Raspbian OS build went!

[mirkocomparetti-synesis]

The fact that there has been only one successful ubuntu install does not look encouraging.

The problem here is that the build procedure of the library is corrupted and it does not build the python libraries in the right way.
The problem is similar to #6820 that has been reported months ago and it was referred to a broken cmake for the python lib: the symptoms are the same, but now there has been 10 releases in between so using the suggestion there is too old.

RaspiOs is compiling

[MartyG-RealSense]

I researched past cases further that involve the Module pyrealsense2 has no attribute 'pipeline' and located one where a Pi user who built with the Acrobotic guide resolved the problem by copying the .so files to the same directory as their script.

#7905 (comment)

[mirkocomparetti-synesis]

Hello @MartyG-RealSense
So even with the procedure you suggested, the python lib is still built in a weird way

>>> imprt pyrealsense2 as rs
>>> rs.__
pyrealsense2.__class__(          pyrealsense2.__file__            pyrealsense2.__init__(           pyrealsense2.__ne__(
pyrealsense2.__repr__(          pyrealsense2.__delattr__(        pyrealsense2.__format__(         pyrealsense2.__init_subclass__(
pyrealsense2.__new__(            pyrealsense2.__setattr__(          pyrealsense2.__dict__            pyrealsense2.__ge__(
pyrealsense2.__le__(             pyrealsense2.__package__         pyrealsense2.__sizeof__(          pyrealsense2.__dir__(
pyrealsense2.__getattribute__(   pyrealsense2.__loader__          pyrealsense2.__path__            pyrealsense2.__spec__
pyrealsense2.__doc__             pyrealsense2.__gt__(             pyrealsense2.__lt__(             pyrealsense2.__reduce__(
pyrealsense2.__str__(          pyrealsense2.__eq__(             pyrealsense2.__hash__(           pyrealsense2.__name__
pyrealsense2.__reduce_ex__(      pyrealsense2.__subclasshook__(

those are all the methods available.

[MartyG-RealSense]

There seems to be a typo on the word 'import' where it is entered as imprt. Does correcting it to import pyrealsense2 as rs make a difference to the output?

[mirkocomparetti-synesis]

@MartyG-RealSense you are right. I typed the import row directly in github when I placed the comment and I mistyped it. It was fine in the editor, otherwise ParseError: bad input on line 1 would have appeared.
Unfortunately the error still stands.

[MartyG-RealSense]

@mirkocomparetti-synesis I carefully reviewed your problem from the beginning on this case and also looked at your other case 1804 (which is handled by a different RealSense team member) in order to seek further insights.

Other than completely reinstalling the system (Ubuntu, librealsense and Python wrapper) to see whether the error goes away - as has happened in some past cases - you might also try adding CMake build instructions to tell CMake about the Python version that you are using, with -DPYTHON_EXECUTABLE=$(which python3)

#4375

Given that 1804 mainly discusses ethernet solutions such as rs-net and EtherSense, or using ROS, could you provide information please about why your Raspberry Pi project requires the RealSense Python wrapper. As far as I know, just having basic Python installed without the RealSense Python wrapper should be sufficient for EtherSense or ROS.

[mirkocomparetti-synesis]

Hello @MartyG-RealSense
Our target is the following: the raspberry pi has the camera connected and acts as a server that streams the camera information to other clients. The client is on another machine (ubuntu desktop) and has to connect to the camera using a python client.

Do you have any idea on how to handle our use case that we didn't think of?

What we tested is:

• intel's prebuilt pi image: does not work
• rs-server: works fine, but no python support for network camera
• etherserver: does not work, as python on the pi is not well built
• ros: does not work (most probably because the ros implementation relies on python and we have the above problem)

In the mean time we will try to do the process again from sketch (hopefully the 4th time's a charming).
Thanks

[MartyG-RealSense]

Regarding the issue with rs-server and the network camera not working: at the time of writing this, rs-server only supports a limited range of resolutions and FPS speeds described in the link below. Were you using a resolution / FPS supported in the list please?

https://dev.intelrealsense.com/docs/open-source-ethernet-networking-for-intel-realsense-depth-cameras#section-3-6-software-limitations

[mirkocomparetti-synesis]

@MartyG-RealSense are you talking about the prebuit image?
In that case, the sdk example didn't compile on the client machine.

Anyway, since we need to use python, AFAIK there is no support for python net cameras.

[MartyG-RealSense]

I was referring to the comment rs-server: works fine, but no python support for network camera ... the resolution and FPS limitations in the link should apply to an rs-server that was compiled from source on Linux as well as the one in the pre-built SD card image.

When planning to use librealsense with support for networking, the CMake build flag -DBUILD_NETWORK_DEVICE=ON should be included in the CMake build instruction. Can you confirm please that you included this instruction? Also, the open-source ethernet networking white paper strongly recommends building librealsense for Raspberry Pi using -DFORCE_RSUSB_BACKEND=ON

#7312 (comment)

#7123

I have seen a small number of cases, such as your own, where rs-server has been used on Pi with Python. I have not personally seen a case where the setting up of a camera as a network camera on Pi using Python has been demonstrated though.

[mirkocomparetti-synesis]

Sure, with that comment I meant that rs-server works on the pi. The fact that python does not support network cameras is something I read here (in this very issue).

In general, rs-server on the raspberrypi and the realsense-viewer on a client machine works fine (if you use 2.42 as there is a bug that come in with 2.43 that we already discussed in another issue).

Anyway, those are the cmake flags we used on the raspberry pi

cmake ../ -DCMAKE_BUILD_TYPE=Release -DBUILD_PYTHON_BINDINGS:BOOL=ON -DBUILD_EXAMPLES=true -DBUILD_GRAPHICAL_EXAMPLES=false -DCHECK_FOR_UPDATES:BOOL=ON -DBUILD_SHARED_LIBS:BOOL=ON -DBUILD_NODEJS_BINDINGS:BOOL=OFF -DBUILD_NETWORK_DEVICE:BOOL=ON -DENFORCE_METADATA:BOOL=ON -DFORCE_RSUSB_BACKEND:BOOL=ON

We selected this camera because there was a prebuilt image for raspberry pi, but we were not lucky enough, maybe.

Thanks!

[MartyG-RealSense]

My recollection is that the pre-built SD card image for Pi 4 provided in the ethernet white-paper was built around the old 2.34.0 SDK version from 2020. I have not heard that it has been updated since. If you are using a firmware that is 5.12.8.200 or newer then a minimum of SDK 2.39.0 is needed due to changes in the firmware from 5.12.8.200 onwards. I typically recommend firmware 5.12.7.100 for use with SDKs older than 2.39.0.

If ypu typically use your camera with modern SDK versions then I wouldn't recommend downgrading the firmware just to test it with the old SD card build.

[MartyG-RealSense]

Hi @xlDownxl and @mirkocomparetti-synesis Because of your interest in RealSense networking on Python, I thought that you may be interested to know that the new 2.49.0 librealsense version that is now available adds an lrs-net viewer for Python called net-viewer.py to the Examples section of the Python wrapper.

https://github.com/IntelRealSense/librealsense/blob/master/wrappers/python/examples/net_viewer.py