Install python wrapper on Windows 10 and Python 3.9

[ricardo-reis-1970]

Required Info
Camera Model	D435
Firmware Version	952323021325
Operating System & Version	Windows 10
Platform	PC
SDK Version	2.40.0.2482
Language	python
Segment	others

Issue Description

I have a big medical device application on Python 3.9. One of the components uses a D435 and this has been posing difficulties, due to the limitation on the python version supported. The current workaround has been running the this component as a standalone app, communicating with the main app via remote calls. The issues raised with this approach are many, from maintainability to physical resources.

As such, I've been trying to compile the python wrapper for some 5 days, quite unsuccessfully. I read all the way through:

• Python wrapper installation instructions, particularly the Windows section
• Issue 7636
  The Windows instructions are particualrly incomplete, assuming people read through the Linux section, so zero on the self-contained front there. In any case, as the aforementioned issue highlights, the overall documentation set amounts mostly to a placeholder for better days. Issue 6053 makes brilliant points, among which "You want to sell bajillions of these, Why make it so hard for people to just get the data?". Certainly a blanket assessment.

The most frustrating part with Intel was coming across Pull Request 8138 and realise that, apparently, all my troubles would go away if anyone there could just spend 15 seconds adding 2 lines just to declare a compatibility that has already been successfully tested.

I just spent a large amount of time installing Visual Studio Community 2017 and compiling through the convoluted process of CMake and all that. I would not have minded at all if in the end I had seen success. VS 2017 — a tool I don't use at all — reported all success, but I'm not sure that cmake-gui did its job airtight, because you guys don't have a clear cut set of command-line instructions for Windows. Even cmake, which is supposed to produce makefiles, is the gui version.

What I could appreciate would be blank slate instructions, including versions of everything (compilers, C++ redistributables) and the steps. I use hundreds of Python libraries, from REST API, to ORMs, NoSQL, computer vision, data analysis, web crawling... If it's python, chances are I used it. I never built one of those in a foreign language. I don't know how to transform a set of scattered files into a python library. My final outcome — until you guys have something I can pip install — should be a folder that I could refer to from a pyproject.toml along the lines of

[tool.poetry.dependencies]
pyrealsense2 = {path = "C:/path/to/pyrealsense2"}

Thanks in advance
Ricardo

[MartyG-RealSense]

Hi @ricardo-reis-1970 You are the first user of Python 3.9 with the Python wrapper that I have seen. Unless there is an installation complication with using 3.9 with the wrapper that has not been experienced until now, you should be able to build the wrapper from source code on 3.9.

Although a pip installation package for pyrealsense2 is available for Python 3.7, it sounds as though your medical device application is dependent on Python 3.9, therefore ruling out installing with pip on 3.7. Is that correct, please? If the project is not dependent on 3.9, then a pip install on 3.7 could be attempted.

The simplest approach to buiding from source would be to build librealsense and the Python wrapper together at the same time using the BUILD_PYTHON_BINDINGS CMake build flag with CMake's visual GUI. There is a clearly written tutorial for doing so on Windows 10 at the link below.

https://rahulvishwakarma.wordpress.com/2019/08/17/realsense-435i-depth-rgb-multi-camera-setup-and-opencv-python-wrapper-intel-realsense-sdk-2-0-compiled-from-source-on-win10/

This guide is based on Intel's official documentation for building librealsense on Windows from source code:

https://github.com/IntelRealSense/librealsense/blob/master/doc/installation_windows.md

The alternative approach is to build the wrapper from source code on its own after librealsense has already been installed. If building librealsense and Python at the same time with CMake is not possible for your particular project, or if the linked-to tutorial above does not work for you, then we can certainly go through the steps for a Windows source-code install of the Python wrapper.

[ricardo-reis-1970]

Hi @MartyG-RealSense,

Thank you for taking a Sunday to reply. HEre are a few thoughts:

1. Official Documentation: this follows the rule of "I'm sure you'll figure it out", with a ton of gaps, assumptions and "you bought our expensive camera, so now tough it up". It's helpful if you're either interested in progressing to square one or if you already know it all, in which case... Since I was born on square one, this one is not for me.
2. This guide possibly just made me compile the whole thing. I have the SDK 2.40 installed from the installer, and when I run the executables in build\release, they show version 2.41. HORRAH! Also, I just built the whole thing against python 3.9, which is a fantastic feeling and the first time in my life I'm the first person in the World to do something.

Unfortunately, the tutorial by Rahul Vishwakarma that you kindly provided diverges from my case in a fundamental aspect: he is working with Anaconda, meaning all he needs to do is to drop a few files in a special place and it's done. I, on the other hand, am using nativa Python, so I need to have a standard repeatable distributable version, so I either have something from your team on Pypi (with the pull request merged), or I need to have something like it in a local folder that I can put in a pyproject.toml, as per my previous message.

Right now, all I have is a bittersweet Crassus victory: I compiled everything and it's looking great. I just can't do anything with it. I tried python setup.py bdist_wheel from the wrappers\python folder, only to get:

> python setup.py bdist_wheel
Traceback (most recent call last):
  File "setup.py", line 5, in <module>
    from pyrealsense2._version import __version__
  File "C:\Users\ricar\Documents\Sandboxes\Python\RealSense\librealsense\wrappers\python\pyrealsense2\__init__.py", line 2, in <module>
    from .pyrealsense2 import *
ModuleNotFoundError: No module named 'pyrealsense2.pyrealsense2'

This kept happening even after I copied pybackend2.cp39-win_amd64.pyd and pyrealsense2.cp39-win_amd64.pyd from the build\Release folder into wrappers\python\pyrealsense2. In any case, the absence of a _version.py file would possibly break it on the next line. I copied these following the instruction in the very wrappers\python\pyrealsense2\__init__.py file, but I'm not sure whether I should be copying dlls or libs to the same folder or somewhere else.

Just as a side note, the previous installation — python 3.6.6, pyrealsense2 installed from pypi — resulted in the follwing files:

 3,677,184 pybackend2.cp36-win_amd64.pyd
25,695,232 pyrealsense2-net.cp36-win_amd64.pyd
28,574,720 pyrealsense2.cp36-win_amd64.pyd
        28 _version.py
        87 __init__.py

Upon compiling the new version, I don't have the -net equivalent and the other two are much smaller, around 1.5 MB.

Essentially, I'm trying to do something of a pypi replica (which I have no idea how to make), or a repository that I could push to my git and import directly into my project. I'm very glad for your help, but now I feel closer to the end, which is all the more frustrating.

What am I missing? Is there anyone I could contact? On my end, this is getting past missing sprints and into missing milestones.

Many thanks!
Ricardo

[MartyG-RealSense]

Pypi is part of pip. installation, as is the command python setup.py bdist_wheel. So using this command is not necessary for a Python 3.9 installation, which should be a source-code build.

Did you install librealsense using the installer file from the Assets file list on the SDK's Releases page? If so, this may be why it is not making a connection to Python 3.9. This 'binary' installation of librealsense from the installer file targets Python 2.7.

I would therefore recommend building librealsense and the Python wrapper together on Windows 10 from source code using the CMake visual GUI if possible (the process outlined in Intel's guide).

https://github.com/IntelRealSense/librealsense/blob/master/doc/installation_windows.md

If you are installing the Python wrapper with this method then you do not have to use the Python wrapper page's installation guidelines at all. You only need to use the link above.

To install the Python wrapper at the same time as librealsense with the CMake visual GUI, you should just take the one additional step of making sure that the BUILD_PYTHON_BINDINGS option is ticked and then compile.

So basically, place a tick in all the CMake boxes shown on the illustration in Intel's guide, plus the BUILD_PYTHON_BINDINGS box too.

---

I should additionally emphasize not to go any further down through the CMake installation guide than the illustration showing what options to tick. You do not need to do the metadata instruction section, as metadata support can be enabled much more easily by doing so within the RealSense Viewer program after installation has been completed.

[ricardo-reis-1970]

So, what you're saying is: I must uninstall all RealSense 2.0 SDK (I do have 2.40 installed) and then have a go at this whole process?

[MartyG-RealSense]

It would be advisable to remove 2.40.0 and then use the CMake GUI method of installing from source code

[ricardo-reis-1970]

Hi again.

This feels like it's almost there... but not just yet. Following some instinct and reading in between the lines, I compiled the whole thing, but not from the original repo. I cloned this one: https://github.com/smeng9/librealsense, as smeng9 seems to be in your team and proposed a bump up in the version. The only difference between his repo and the original is in wrappers\python\setup.py where he declares versions of python up to 3.9.

So, I built everything and got no errors, but my Cmake config did not include the pybinds11. After completing, I went on build\Release and copied pybackend2.cp39-win_amd64.pydandpyrealsense2.cp39-win_amd64.pydto a folder calledpyrealsense2insidewrappers\python\pyrealsense2`. By this time, trial&error is becoming my top skill.

After that, I ran python setup.py bdist_wheel because — and allow mw to insist on this point! — I don't need some file that I can drop somewhere. What I need is a repeatable solution that involves my putting some reference in a pyproject.toml file. The Anaconda version will not cut it.

As it is right now, I can install it through pip install pat6h\to\librealsense\wrappers\python. After that, I can run python and do import pyrealsense2 as rs. And now I have ts.pyrealsense2, a completely empty module.

I also have a pyrealsense2-2.41.0.2666-cp39-cp39-win_amd64.whl inside wrappers\python\dist, and I tried installing from it, but the result was the same. This wheel has 8M, whereas the previous one had around 24MB.

There is still something missing, and I am under the impression that something is being left out from the cmake-gui step. I still don't generate a pyrealsense2-net.cp39-win_amd64.pyd (do I need that?) and the other two .pyd are 1/3 of the cp36 version size. I uninstalled SDK 2.20. Should I install 2.41 now?

[MartyG-RealSense]

I do not have the knowledge to advise on a custom Python wrapper installation process that is outside of the normal recommended methods, unfortunately. I looked up the PR for Python 3.8 and 3.9 by @smeng9 - it provides useful information about why you cannot downgrade to Python 3.7, but @smeng9 is the best person to comment on their own code.

#8138

If the Python wrapper was being installed from source code on its own (not alongside librealsense) then it would be normal to include a CMake flag called DPYTHON_EXECUTABLE to specify exactly which Python 3 version to target. The wrapper's Windows instructions provide the following information about this:

---

If you have multiple python installations on your machine you can use -DPYTHON_EXECUTABLE= For example: -DPYTHON_EXECUTABLE=C:/Python27/python.exe

---

So conceivably, if Python 3.9 is at C:/Python39 on Windows then the CMake flag may be:

-DPYTHON_EXECUTABLE=C:/Python39/python.exe

[ricardo-reis-1970]

Dear @MartyG-RealSense,

I feel as though I imposed on your time, patience and kindness long enough. I had the feeling that @smeng9 was part of the team, and a relevant one. I'd like to reach out to them, although I'm not sure exactly how to accomplish that. Their Github profile is quite... shielded: no messages, no issues, no comments, nothing. I don't even have a name.

Is there any way I could get through to them? I'm [email protected] and I have an urgent case in my hands.

Many thanks and kind regards
Ricardo

[MartyG-RealSense]

I am happy to continue to be your support contact on this case. I appreciate your patience as I navigate the complexities of your particular case (which I have no previous precedent to refer to for guidance).

I do not have a way to communicate with smeng9 other than with name-tagging with @ to highlight comments to them as you yourself have done.

If you are not building librealsense and the Python wrapper together at the same time, it is usually essential that librealsense is installed first before attempting to then build the Python wrapper (though we are admittedly outside of the normal procedures in this case).

I suspect that this project would be easier to implement if the computing device that the camera was attached to was a Raspberry Pi 4. You could then set up librealsense and Python on one machine and burn the entire system as an SD card image file. That system image could then be installed on other Pis by flashing the entire system onto a blank SD card in a single action.

Intel created a similar pre-made RealSense downloadable image file for Pi 4 (though not with the Python wrapper included in the image).

https://dev.intelrealsense.com/docs/open-source-ethernet-networking-for-intel-realsense-depth-cameras#section-2-3-preparing-the-sd-card

If you need to use Windows, you can also create an iso image of a Windows 10 computer that could be downloaded or stored on a storage medium such as a dvd. Google for create windows iso for more details.

https://support.microsoft.com/en-us/windows/create-an-iso-file-for-windows-10-38547366-1dcb-7afd-1726-9eb222d72705

[smeng9]

Ok the real problem is Intel keep forgetting including python 3.8 - 3.10 in the build and release pipeline. Everything is supported, but they just make things complicated and it introduces a lot of headaches for users.

So some alternative way to solve the headache without recompiling is:
Download from pypi the released package for 3.7
Find the python binding binary in the whl
Use hex editor to patch the version check code corresponding to

librealsense/wrappers/python/third_party/pybind11/include/pybind11/detail/common.h

Line 235 in d8f5d42

"Python version mismatch: module was compiled for " \

Then change target python version and install, it will work

[MartyG-RealSense]

Thanks so much for your contributions @smeng9 !

[smeng9]

@MartyG-RealSense No problem, just ask development team to release package for higher python versions in the next version. This will make the life easier for everyone!

[Nir-Az]

Hi @smeng9 ,

librealsense use pybind11 third-party for python binding.
Until lately the pybind11 version integrated with librealsense did not support python > 3.7
We worked on integrating the latest version that supports python > 3.7 and since this PR (not a 2 lines fix :)), development branch build from source include this update and should support it.

Note that according to pybind11 documentation, using python 3.9 with earlier version with cause undefined behavior.

[MartyG-RealSense]

Thanks very much @Nir-Az for the information about your Pybind PR and its successful merging!

@smeng9 Thanks again for your work.I filed an official feature request for a pyrealsense2 pip package for Python 3.8 last year. I am not involved in development activities though.

[ricardo-reis-1970]

Dear all,

Thank you for your contributions so far. This all seems so easy for you, but for one not involved with the intricacies of compiling large amounts of code in a foreign language (C++) or building packages, this is all very opaque. As an old school engineer, unpacking a wheel and hex editing a binary doesn't sound too bad (on paper, that is), but building the whole package does.

Python developers run pip install and things happen automagically that most of us never even realise, like files being copied into .venv\Lib\site_packages and what not. We simply don't know where these goodies come from or how they get to talk to Python, but thanks to people like you, we can use them.

This goes to say that I don't think I could (in useful time or any time at all) get to a package I could attach to my projects. I usually avoid asking people to do my job, except when (it's really not my job) && (I can't really do it). Is there any possibility one of you could get me a compiled version — a folder, or a wheel — for me to patch things up until Intel decides to finally publish it? This way, I could just put it next to the project in my repo and point all the relevant pyproject.tomls to it.

And then my manager could get off my back, I could get off yours and the sun would shine again in Geneva Switzerland!

[MartyG-RealSense]

Hi @ricardo-reis-1970 This is a subject that @smeng9 and @Nir-Az will have better knowledge of than myself. As your project is time-critical though, I would recommend not to make your plans dependent on the near-term provision of an official pip package for Python 3.9, as there are no guarantees of time schedule that I can provide.

At this point, the hex editing solution offered by @smeng9 in #8229 (comment) seems the most straightforward path to creating an unofficial pip wheel solution. @smeng9 Is there any step-by-step guidance that you could offer to enable @ricardo-reis-1970 to carry out this workaround themselves, please?

[Nir-Az]

Building from source require multiple flags that also depends on your OS,

I would use the patch @smeng9 suggested (or wait for it to be releases by Intel) but with python 3.9.1 as mentioned at pybind11 documentation here

Warning

Combining older versions of pybind11 (< 2.6.0) with Python 3.9.0 will trigger undefined behavior that typically manifests as crashes during interpreter shutdown (but could also destroy your data. You have been warned.)

We recommend that you update to the latest patch release of Python (3.9.1), which includes a fix that resolves this problem. If you do use Python 3.9.0, please update to the latest version of pybind11 (2.6.0 or newer), which includes a temporary workaround specifically when Python 3.9.0 is detected at runtime.

[ricardo-reis-1970]

I think you guys underestimate how desperately lost I am here. I'll take that as a undeserved compliment.

@smeng9's process:

Download from pypi the released package for 3.7
Find the python binding binary in the whl
Use hex editor to patch the version check code corresponding to

Line 235 in d8f5d42

Then change target python version and install, it will work

But this is again very cryptic. Here's what I managed so far:

1. Download from pypi: I hope this is with pip-download. I used that and I got 10 wheels, latest being pyrealsense2-2.41.0.2666-cp37-cp37m-win_amd64.whl. Check!
2. I can see the whole inside of the wheel with 7zip, but no clue what the python binding binary file is.
3. Dependent on previous, but no idea what would I need to do, if I ever got here.
4. Same.

Inside that wheel, all pyd files have the string cp37 in their names. would this also need to be changed?
:-S

PS: regarding the pybind11 issue, for my project, upgrading from Python 3.9.0 to 3.9.1 would be absolutely no issue.

[Nir-Az]

Hi,
You need to choose the whl file with respect to OS, platform and python version, this is an example to the path to the file you need for:
OS: Windows
Platform: x64
Python: 3.7

"pyrealsense2-2.41.0.2758-cp37-cp37m-win_amd64\pyrealsense2\pyrealsense2.cp37-win_amd64.pyd"

you don't need to rename it.

normally (on windows OS) inside the whl you will find up to 3 pyd files.
pyrealsense2.xxx..pyd
pyrealsense2-net.xxx..pyd
pybackend2.xxx..pyd

You probably need the pyrealsense2.xxx..pyd file.

Good luck

[MartyG-RealSense]

Hi @ricardo-reis-1970 Do you require further assistance with this case, please? Thanks!

[ricardo-reis-1970]

Dear @MartyG-RealSense,

I am at odds with all this. The SW Engineers at Intel seem to have the product tested and ready for Python 3.9.[0|1], but somehow they find it too difficult to just make it available publicly. It also feels from where I'm standing that everybody involved thought it would be wiser to spend hours guiding me through uncharted territory and foreign languages than spending 5 minutes compiling a wheel and making it available. I need the fish right now, so that I can live enough to actually consider the fishing lessons.

Let me be crystal clear: I cannot edit the wheel or get to a point where I have something I could add as a source to my project. I am not a C++ dev, and much less a C++ engineer. I am a consumer of your product, or desperately trying to be.

Right now, this is making a big dent on my project. I need to run the RealSense component as a separate server, doubling the effort on embedded systems that are not known for their Xeon-grade CPUs. At some point you suggested that I moved into a Raspberry-Pi and downloaded some image. That is also not an option.

I'm also checking Linux deployments of my software, so whatever the solution I seek, it would be best if it supported both.

My options are:

1. someone from Intel pushes the latest change to deployment and I upgrade on my end — IDEAL SOLUTION, serves my purposes and everybody else's
2. someone from Intel makes a wheel, or a folder available to me supporting 21st century Python;
3. nothing happens and I keep stuck with the poor current state of affairs, rethinking the whole camera option.

I have a big deployment coming up in about 2 weeks, Investors will be attending. Failing 1 and 2, please consider this a closed topic.

[Nir-Az]

Hey @ricardo-reis-1970 ,

I will try my best to help you so you will not be stuck by delivering option 2 , (option 1 is in progress too but it will take some time since it is more complicated ,it involves DevOps activity by changing the release procedure ..)

Please assist with the following details that will fulfill your needs.
LRS Version:
Python version:
OS:
Delivery email address:

Thanks

[ricardo-reis-1970]

Dear Nir,

You have gmail.

Many thanks
Ricardo

[MartyG-RealSense]

Hi @ricardo-reis-1970 Assuredly, I do very much understand the development pressures and external pressures of your project.

@Nir-Az Thanks so much for your assistance to @ricardo-reis-1970 - as you are clearly an expert in the subjects covered in this case, you are very well placed to provide advice that can assist, so I appreciate your time and effort!

[Nir-Az]

@MartyG-RealSense No problem,
@ricardo-reis-1970 I sent you the required PYD to the mail address you gave me.

SDK Version: 2.41
Python version: 3.9.1
OS: Windows 64

Please update if it suits your needs.

[MartyG-RealSense]

Hi @ricardo-reis-1970 Do you require further assistance with this case, please? Thanks!

[ricardo-reis-1970]

Hi @MartyG-RealSense,

I got a pyd file that I just drop next to my project. Neither ideal nor clean, but Intel just released 2.42 for Valentine's Day and still no Python 3.9. Most unfortunately, and despite abundant proof that it works — I am the proof right now — your company still neglects its users, or pushes them to using old software. There is nothing you can do there.

Considering the relevance of Python on Computer Vision, there's the ongoing feeling / rumor that Intel is about to simply drop the whole RealSense product line. There is nothing you can do there either.

You were exceedingly helpful, and so were the engineers involved, particularly @Nir-Az. Thank you both. I'll be closing this issue and focusing on learning how to build the lib from scratch.

Best regards
Ricardo

[MartyG-RealSense]

Thanks very much for the update Ricardo. As one of the first librealsense users of Python 3.9 to make a report on this forum, your difficult experience undoubtedly provided useful information and feedback.

[MartyG-RealSense]

Hi Ricardo, I wanted to update you on progress. SDK version 2.43.0 has just been released and has the following comment in its release notes:

[Python] Generate wheels for Python 3.8 + 3.9
Package classifiers updated to support python 3.8 + 3.9

[ricardo-reis-1970]

Thanks @MartyG-RealSense!

I just found it here. However, please be advised that the latest release (linked from here) seems to be 2.42. 2.43 is marked as pre-release.

So, I deleted the pyd file that I had been given by @Nir-Az (!תודה רבה לך), I added this line to my pyproject.toml:

pyrealsense2 = "^2"

and a funny thing happened: it works!!

Just for good measure, I cleaned the poetry cache, destroyed the virtual environment and ran a poetry update. It still works.

For future reference: I hope Intel can find the availability and manpower to keep these new releases up to date with latest language versions for the supported languages. Following emerging trends in Functional Programming, adding support for Rust and Haskell would open the RealSense world to a wide and growing number of engineers interested in extremely reliable software that currently look elsewhere for live video sources.

Many many thanks!
Ricardo

[MartyG-RealSense]

Thanks very much Ricardo for your detailed feedback!

There is already an actively maintained RealSense Rust binding available outside of the RealSense SDK.

https://gitlab.com/tangram-vision-oss/realsense-rust