Unable to build pyrealsense2_net

[colinmsnow]

Required Info
Camera Model	D435i
Firmware Version	05.12.15.50
Operating System & Version	Linux
Kernel Version (Linux Only)	4.9.253-tegra
Platform	NVIDIA Jetson Nano B01
SDK Version	2.47.0 / 2.49.0
Language	Python
Segment	Robot

Issue Description

Hi,

I am trying to build the Python realsense library pyrealsense2_net and run the net_viewer.py example to get frames from a networked realsense device. The remote device is a Raspberry Pi 4 setup via the guide to ethernet networking.

I am building realsense from source using the commands:

cmake .. -DBUILD_NETWORK_DEVICE=ON -DBUILD_PYTHON_BINDINGS=ON -DPYTHON_EXECUTABLE=$(which python3) -DBUILD_SHARED_LIBS:BOOL=ON
make
sudo make install

I have also tried with only the first two flags but added shared_libs and python_executable based on comments on some similar issues.

Running the above commands successfully builds pyrealsense2 and a folder for pyrealsense2_net but the network library has almost no methods and does not implement net_device which is critical for using networked cameras.

Here is what happens if I import the two libraries and see their methods:

>>> import pyrealsense2
>>> dir(pyrealsense2)
['BufData', 'D400', 'DEPTH', 'L500', 'SR300', 'STAEControl', 'STAFactor', 'STCensusRadius', 'STColorControl', 'STColorCorrection', 'STDepthControlGroup', 'STDepthTableControl', 'STHdad', 'STRauColorThresholdsControl', 'STRauSupportVectorControl', 'STRsm', 'STSloColorThresholdsControl', 'STSloPenaltyControl', 'T200', 'TRACKING', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', '__version__', 'align', 'any', 'any_intel', 'auto_calibrated_device', 'calib_target_type', 'calibrated_sensor', 'calibration_change_device', 'calibration_status', 'calibration_type', 'camera_info', 'color_sensor', 'colorizer', 'composite_frame', 'config', 'context', 'debug_protocol', 'debug_stream_sensor', 'decimation_filter', 'depth_frame', 'depth_huffman_decoder', 'depth_sensor', 'depth_stereo_sensor', 'device', 'device_calibration', 'device_list', 'disparity_frame', 'disparity_transform', 'distortion', 'dsm_params', 'enable_rolling_log_file', 'event_information', 'extension', 'extrinsics', 'filter', 'filter_interface', 'firmware_log_message', 'firmware_log_parsed_message', 'firmware_logger', 'fisheye_sensor', 'format', 'frame', 'frame_metadata_value', 'frame_queue', 'frame_source', 'hdr_merge', 'hole_filling_filter', 'intrinsics', 'l500_visual_preset', 'log', 'log_message', 'log_severity', 'log_to_callback', 'log_to_console', 'log_to_file', 'max_usable_range_sensor', 'motion_device_intrinsic', 'motion_frame', 'motion_sensor', 'motion_stream', 'motion_stream_profile', 'non_intel', 'notification', 'notification_category', 'option', 'option_range', 'options', 'pipeline', 'pipeline_profile', 'pipeline_wrapper', 'playback', 'playback_status', 'pointcloud', 'points', 'pose', 'pose_frame', 'pose_sensor', 'pose_stream', 'pose_stream_profile', 'processing_block', 'product_line', 'quaternion', 'recorder', 'region_of_interest', 'reset_logger', 'roi_sensor', 'rs2_deproject_pixel_to_point', 'rs2_fov', 'rs2_project_color_pixel_to_depth_pixel', 'rs2_project_point_to_pixel', 'rs2_transform_point_to_point', 'rs400_advanced_mode', 'save_single_frameset', 'save_to_ply', 'sensor', 'sequence_id_filter', 'serializable_device', 'software_device', 'software_motion_frame', 'software_notification', 'software_pose_frame', 'software_sensor', 'software_video_frame', 'spatial_filter', 'stream', 'stream_profile', 'syncer', 'temporal_filter', 'terminal_parser', 'texture_coordinate', 'threshold_filter', 'timestamp_domain', 'tm2', 'units_transform', 'updatable', 'update_device', 'vector', 'vertex', 'video_frame', 'video_stream', 'video_stream_profile', 'wheel_odometer', 'yuy_decoder', 'zero_order_invalidation']
>>> import pyrealsense2_net
>>> dir(pyrealsense2_net)
['__doc__', '__loader__', '__name__', '__package__', '__path__', '__spec__']

I have tried SDK versions 2.47.0 and 2.49.0 and they seem to work the same.

Is there something I am missing here? Additional flags I need to set or libraries to install?

Thanks,
Colin

[MartyG-RealSense]

Hi @colinmsnow The ethernet guide recommends including the CMake build flag -DFORCE_RSUSB_BACKEND=ON when building the SDK from source on Raspberry Pi. This advice is also applicable to building the SDK from source on Jetson, as demonstrated by the Jetson Nano compatible installation guide at #6964 (comment)

[colinmsnow]

I tried again building with:

cmake .. -DBUILD_NETWORK_DEVICE=ON -DBUILD_PYTHON_BINDINGS=ON -DPYTHON_EXECUTABLE=$(which python3) -DBUILD_SHARED_LIBS:BOOL=ON -DFORCE_RSUSB_BACKEND=ON -DCMAKE_BUILD_TYPE=release -DBUILD_EXAMPLES=true -DBUILD_GRAPHICAL_EXAMPLES=true

It builds successfully but I still have the same problem with missing methods in pyrealsense_net. I have been following the instructions in #20 and have moved the init.py file to the directory that holds the python package. This makes pyrealsense2 work but there is no equivalent init.py for the net module. I tried it with the same init file as pyrealsense2 and also a blank init and neither caused any change.

The build put the python packages in /usr/local/lib/python3.6 and the pyrealsense_net folder contains the following files:

pyrealsense2-net.cpython-36m-aarch64-linux-gnu.so  pyrealsense2-net.cpython-36m-aarch64-linux-gnu.so.2.47  pyrealsense2-net.cpython-36m-aarch64-linux-gnu.so.2.47.0

This seems like it should be correct but I have no way of verifying whether these files are flawed or whether they are not being imported properly.

[MartyG-RealSense]

The instructions in IntelRealSense/meta-intel-realsense#20 (comment) relate to a branch of Linux called Yocto and not Ubuntu.

After having installed the pyrealsense2 wrapper and included net support in the build with -DBUILD_NETWORK_DEVICE=ON, what happens if you then attempt to launch the net_viewer.py example?

[colinmsnow]

I built it using the flags:

cmake .. -DBUILD_NETWORK_DEVICE=ON -DBUILD_PYTHON_BINDINGS=ON  -DBUILD_SHARED_LIBS:BOOL=ON -DFORCE_RSUSB_BACKEND=ON -DCMAKE_BUILD_TYPE=release -DBUILD_EXAMPLES=true -DBUILD_GRAPHICAL_EXAMPLES=true

Running make and sudo make install results in the folder /usr/local/lib/python3.6/pyrealsense2 that contains the files:

pybackend2.cpython-36m-aarch64-linux-gnu.so         pyrealsense2.cpython-36m-aarch64-linux-gnu.so         pyrealsense2-net.cpython-36m-aarch64-linux-gnu.so
pybackend2.cpython-36m-aarch64-linux-gnu.so.2       pyrealsense2.cpython-36m-aarch64-linux-gnu.so.2.49    pyrealsense2-net.cpython-36m-aarch64-linux-gnu.so.2.49
pybackend2.cpython-36m-aarch64-linux-gnu.so.2.49.0  pyrealsense2.cpython-36m-aarch64-linux-gnu.so.2.49.0  pyrealsense2-net.cpython-36m-aarch64-linux-gnu.so.2.49.0

If I run net_viewer.py in this folder I get an import error saying that pyrealsense2_net does not exist (though it successfully imports pyrealsense2 first):

Traceback (most recent call last):
  File "test_realsense_net.py", line 12, in <module>
    import pyrealsense2_net as rsnet
ModuleNotFoundError: No module named 'pyrealsense2_net'

This makes sense because there is no folder for pyrealsense2_net and python does not know how to find it, but the folder for pyrealsense2 appears to contain the net libraries. Is this the expected behavior? And if so how do I get python to recognize these pyrealsense2-net.cpython files as a library?

[MartyG-RealSense]

Another RealSense user in #7606 also encountered the ModuleNotFoundError: No module named 'pyrealsense2_net' error when trying to import pyrealsense2-net. A RealSense team member asked whether they had rs-server running on Linux. I therefore wonder whether rs-server should be active first before the net-viewer.py example program is run.

Thanks very much for your patience during this case!

[colinmsnow]

After reading the link you provided it seems like the problem encountered in the issue was that rs-server only supports Linux and so it is not possible to compile pyrealsense2_net on windows. I am working on Linux and building the SDK successfully builds rs-server and puts it in /usr/local/bin/rs-server. I assume this is what is needed to build pyrealsense2_net. I can run rs-server and it works fine but it doesn't do anything because there is no realsense connected to the local device.

/usr/local/bin$ ./rs-server
Rs-server is running
Waiting for Device...

Running rs-server has no effect on the python modules and I still get the same error.

[MartyG-RealSense]

If this were the open source ethernet networking project then I would expect to have to make an SSH connection to rs-server from the host computer before being able to stream data from the remote camera to the host. The net-viewer.py Python example has no accompanying documentation to indicate what prerequisites and procedures are needed in order to establish a network connection though.

As the available investigative leads about this subject have been covered already in this discussion, since you are using Linux and Python it may be worth considering using EtherSense instead, which was a RealSense networking project designed for Python that was the predecessor of rs-server.

https://github.com/krejov100/EtherSense

[colinmsnow]

I think my last post might have been a bit confusing... I have a remote device (Raspi 4) which is running rs-server and streaming data to the Jetson. I know this works as I can access the stream both through the realsense-viewer and the c++ API. The net_viewer.py file is failing before it even attempts to make this connection when it tries to import the library pyrealsense2_net. The issue is not with anything at runtime but with either the library files or the way they are set up on the computer when it is built by the SDK. Is there any prebuilt version of librealsense which includes net libraries I could take a look at to see how the files are arranged? Or any other documentation on pyrealsense_net besides that one example file? I suspect it might be a problem with how the SDK places the build files after building.

I would like to avoid using EtherSense as my server computer is already set up and working reliably and future plans for this project will likely depend on using both c++ and python to access the remote device which is only possible with realsense-net.

[MartyG-RealSense]

In the ethernet networking white-paper, Intel provide a download link for a pre-built 3 GB Raspberry Pi 4 SD card image containing librealsense 2.34.0 and the networking files for testing purposes and to compare the file structure to your own installation.

https://dev.intelrealsense.com/docs/open-source-ethernet-networking-for-intel-realsense-depth-cameras#section-2-3-preparing-the-sd-card

[colinmsnow]

I am using that image on my remote device and to my knowledge, while it does build the net device, it does not build python bindings so does not have pyrealsense2 or pyrealsense2_net.

[MartyG-RealSense]

I reviewed the case again from the beginning but did not find any further leads that mat be helpful unfortunately, as rs_viewer.py was introduced relatively recently in SDK 2.49.0 and so there are few references about it at the time of writing this. I do apologize.

[colinmsnow]

I tried the same set of actions on another computer (Dell Latitude running Ubuntu 18.04) and got exactly the same result so I suspect it is not an issue with my machine or configuration but rather in either how the installer places files or a lack of documentation on how to use them after they are built. I noticed that @maloel wrote the net_viewer example and a few others including @lramati seem to be actively developing the python net module. Would any of you be able to provide some clarity on this or @MartyG-RealSense do you know anyone else who might have knowledge about it? Assuming this is not a computer-specific issue it seems like it is something that is pretty essential to the use of the pyrealsense2_net project.

[michezio]

I'm having the same issue on a Windows machine: I successfully compiled both pyrealsense2.cp38-win_amd64.pyd and pyrealsense2-net.cp38-win_amd64.pyd but when placing them into the site-package folder, pyrealsense2 is correctly loaded while pyrealsense2_net (which I had to rename to change the hyphen into an underscore to import in Python) is not, with the following error:

ImportError: DLL load failed while importing pyrealsense2_net: Routine di inizializzazione della libreria di collegamento dinamico (DLL) non riuscita.

which means initialization routine of the DLL failed.

I tried to include also realsense2.dll and realsense2-net.dll in the same folder but same error appears.

I also noticed that installing the pip version of pyrealsense2 includes pyrealsense2-net into the package so I tried importing that one (after renaming):

import pyrealsense2 as rs
import pyrealsense2.pyrealsense2_net as rsnet

Same error appears

[MartyG-RealSense]

I have highlighted this case to Intel to seek advice. Thanks very much for your patience.

[MartyG-RealSense]

Hi @colinmsnow and @michezio Intel are still investigating this issue. Thanks very much for your patience!

[MartyG-RealSense]

There is currently no further information from Intel to report about this issue.

[Adamskiiiiii]

I am also having the exact same issue on window. Is there any new update regarding the situation?

[MartyG-RealSense]

Hi @Adamskiiiiii The most recent discussion of the ModuleNotFoundError: No module named 'pyrealsense2_net' error was a week ago at the time of writing this at #10218 though that was again an Ubuntu related case.

The next version of the RealSense networking API is being worked on and due "soon", though I do not have an estimate of a release timetable to provide.

[MartyG-RealSense]

I have no further information from Intel to provide at the time of writing this.

@colinmsnow @michezio @Adamskiiiiii Could each of you update me about whether you are still experiencing problems with pyrealsense2.net please. Thanks!

[michezio]

@MartyG-RealSense Hi, I decided to switch to C++ code in order to use the network module. In C++ it builds fine and works correctly up to version 47, as I described here #9971

[MartyG-RealSense]

@michezio Thanks for the information!

[Adamskiiiiii]

@MartyG-RealSense pyrealsense2.net still doesn't work, so I have switched to C++ networking instead (and encountered a different issue, at #10262). Thank you for the update anyway!

[MartyG-RealSense]

Thanks very much @Adamskiiiiii for your update! I have replied to #10262

@colinmsnow I will close this case after 7 days from the time of writing this if I do not hear from you. Thanks!

[colinmsnow]

I was never able to resolve this issue or get pyrealsense2.net working in any capacity. We have moved on to another solution.
Thanks so much for trying!

[MartyG-RealSense]

Thanks very much @colinmsnow for the update! As everyone involved in this case has either moved to C++ or chose another solution, I will close this case. Thanks again!

[deepak-asink]

Hello MartyG,
I am also facing the same problem, Is there any update from intel side.

[MartyG-RealSense]

Hi @deepak-asink There is no update to provide regarding issues with pyrealsense2.net. Intel are working on the next version of RealSense networking, although I do not have a time estimate available for its release.

You could explore whether the Python EtherSense ethernet networking project in the link below that was the predecessor of the current networking system can meet your needs.

https://github.com/IntelRealSense/librealsense/tree/master/wrappers/python/examples/ethernet_client_server

[deepak-asink]

Thanks @MartyG-RealSense , I am able to use Ethersense for my project but the frames I am getting are at very low fps on client side. Can you suggest some fix for that.

[MartyG-RealSense]

A general rule for RealSense ethernet networking will be that the connection will have a faster data transfer rate than USB 2 but slower than USB 3 due to the speed limitations of ethernet cables compared to USB cables. This is described in the table below,.

At #8213 (comment) there was also a case with a Python ethernet setup resembling the EtherSense project where connection problems were found to be caused by use of USB 2 cabling instead of USB 3.

[deepak-asink]

I am asking about the existing compression strategies used by Intel in rs-server which I didn't get with Ethersense.

[MartyG-RealSense]

The white paper document for the networking system based around rs-server states that it makes use of LZ4 for depth compression and the libjpeg-turbo open-source library for infrared compression.

Versions of LZ4 for Python can be found by visiting https://lz4.github.io/lz4/ and scrolling down to the section headed Interoperable LZ4 ports and bindings

[Adamskiiiiii]

I solved the problem by replacing import pyrealsense2_net as rsnet with from pyrealsense2 import pyrealsense2_net as rsnet.

[MartyG-RealSense]

Excellent to hear, @Adamskiiiiii - thanks so much for sharing the solution that worked in your particular case :)

[CharmingZh]

from pyrealsense2 import pyrealsense2_net as rsnet

@Adamskiiiiii, @MartyG-RealSense
I tried to replacing the code with from pyrealsense2 import pyrealsense2_net as rsnet, but I got an error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'pyrealsense2_net' from 'pyrealsense2' (.../Anaconda/anaconda3/envs/KAN-Forest/lib/python3.10/site-packages/pyrealsense2/pyrealsense2.cpython-310-x86_64-linux-gnu.so)

Could you please tell me how you managed to install the pyrealsense2_net module?

[maloel]

Hi @CharmingZh
The realsense2_net library, and its accompanying Python module, have been deprecated and are no longer available except in older versions of the library. It is no longer supported.

[CharmingZh]

@maloel Okay, that's soooo sad ; (
thanks bro, it seems that I have to switch my project into C++.

[MartyG-RealSense]

Hi @CharmingZh Switching to C++ would not help unfortunately as the deprecation of realsense2-net applies to C++ too.

An alternative RealSense networking tool for Python called EtherSense remains available.

https://github.com/IntelRealSense/librealsense/tree/master/wrappers/python/examples/ethernet_client_server