Python interface

[chapulina]

Desired behavior

We should provide a Python interface to Ignition Math to make its APIs available to more users.

We've been using SWIG to generate Ruby interfaces, and can do the same for Python. Issue #101 is tracking the addition of SWIG interfaces.

Alternatives considered

• Don't use SWIG: we could add C APIs instead of using SWIG, but given our experience using SWIG for Ruby so far, that seems to work well and will make it easier to generate code for various languages.
• Other languages: we've started adding Ruby interfaces in the past, but Python is more widely used in robotics, so we'll focus more on Python for now. Using SWIG, it's easier to add support to other languages in the future as needed.

Implementation suggestion

• Finish adding SWIG interfaces to the remaining classes (Documentation, Style and scripting interfaces update #101)
• Create and install Python interfaces alongside Ruby
  https://github.com/ignitionrobotics/ign-math/blob/6534dab9a2ddaaa7659e7ec132f218068a767365/src/CMakeLists.txt#L41
• Add tests for Python interfaces
• Package Python files into a new python3-ignition-math6 Debian package

Additional context

Ignition Math's Python interface will be useful for gazebosim/gz-sim#790.

[ahcorde]

I want to bring a topic to discuss, Right now we are using SWIG to generate the Python interfaces. I would like to consider pybind11 as library to implement this wrapper. pybind11 is a lightweight header-only library that exposes C++ types in Python that we can use it as an alternative to SWIG.

The main difference is that pybind11 uses C++ templates with compile time introspection to generate wrapper code, where SWIG uses a custom C++ parser.

In Pybind11 types need to be wrapped explicitly by specifying the classes and (member) functions. This has the downside of having to write and maintain the wrappers but has the advantage of being explicit and allowing wrappers to be made more Pythonic.

SWIG requires to learn a new tools but with pybind11, developers only need proficiency in two languages (C++ and Python).

I would say that maybe it does make sensor for ign-math to use SWIG, because it has classes that make math (I believe without threading) but for other packages we may require to have more control about what's happening such as ign-gazebo.

I don't know which is going to be the final decision but I think we should stick with one library for all the packages. And right now it's the time because only few classes are already ported with SWIG in ign-math.

By the way, pybind11 is used in rclpy

@chapulina @scpeters Thoughts ?

[chapulina]

Thanks for bringing up pybind11. I think it makes sense for ign-math to stick with SWIG because:

• We already use it for various classes
• As you mentioned, the math classes are pretty isolated and don't involve threading
• It would be nice to complete Ruby support eventually
• It would be nice to add support for other scripting languages too (i.e. Javascript interface #87)

For ign-gazebo however, pybind11 sounds like it may make our lives easier, so I think we could start looking into using that to tackle gazebosim/gz-sim#790.

I think it's ok for us to use different approaches in each library, since they serve different purposes and have different requirements.

[traversaro]

I think it's ok for us to use different approaches in each library, since they serve different purposes and have different requirements.

One tricky aspect in using different approaches for different (but that depend on each other) libraries is that it may be tricky to easily bind in Python the methods in ignition-gazebo that take in input and return ign-math* types. If both ign-math and ign-gazebo use either SWIG or pybind11, then both system offer facilities to easily permit to create a object with ign-math bindings, and pass it then to the interface in ign-gazebo that expects a ign-math object. If ign-math bindings are based on swig and the ign-gazebo are based on pybind11, this interoperability may not be so straightforward, at least in my experience. See ami-iit/bipedal-locomotion-framework#214 for a related discussion (in that case iDynTree was the library with SWIG-based bindings, and bipedal-locomotion-framework the library with pybind11-based bindings).

[traversaro]

In Pybind11 types need to be wrapped explicitly by specifying the classes and (member) functions. This has the downside of having to write and maintain the wrappers but has the advantage of being explicit and allowing wrappers to be made more Pythonic.

Just for information, there are several projects that build on top of pybind11 that promise to at least reduce the amount of boilerplate to write. To be honest however I never used any of those in an actual project, but you can find some references to them in:
Unfortunately, we still did not explored their use, but you can find a list of them in:

• https://github.com/jslee02/awesome-cpp-python-binding-generator#pybind11
• https://gitlab.kitware.com/autopybind11/autopybind11#alternatives-comparison (note: autopybind11 is one such project, the README contains a few other project for comparison, but also autopybind11 should be considered)

[chapulina]

If ign-math bindings are based in swig and the ign-gazebo are based on pybind11, this interoperability may not be so straightforward

Thanks for the heads up, @traversaro . I was hoping that there would be a way for ign-gazebo to consume ign-math's python interface without caring about how it was generated, but I guess things may be more entangled than that.

I see you use SWIG in gym-ignition. How is that going? Would you recommend we use that throughout our stack or look into pybind11 as an alternative?

[traversaro]

I was hoping that there would be a way for ign-gazebo to consume ign-math's python interface without caring about how it was generated, but I guess things may be more entangled than that.

Just to clarify, that may be indeeed possible. However, we never tried, while the use case that is instead fully supported by both SWIG and pybind11 is use in their API classes that are exposed by SWIG and pybind11 respectively.

I see you use SWIG in gym-ignition. How is that going? Would you recommend we use that throughout our stack or look into pybind11 as an alternative?

Probably @diegoferigo can provide a more educated answer. Personally, I am a big fan of the "no glue code" approach of swig or other similar systems, and there are several powerful ways to customize the output code such as typemaps (http://www.swig.org/Doc4.0/Typemaps.html), even to make it more "Pythonic" in the Python case, but those feature may not be obvious to use to developers without SWIG experience. However, in the past we found problems applying SWIG to existing codebases that made heavy use of C++17 features, especially if the codebase was already there and it did not evolved considering SWIG additional constraints from the beginning, see ami-iit/bipedal-locomotion-framework#129 for that specific case I am referring to. In that case, even coming up with a MVP in SWIG was complicated, so we went for pybind11 as it seemed to work fine.

Probably in this case, it could make sense to try to first wrap a few classes in a complex project that could create the most challenges, such as ign-gazebo, and only afterwards take a decision also for the "simple" projects. This may feel strange as it goes against the tipical workflow of proceding by incremental complexities, but in this case proceeding by incremental complexity may risk to lead to integration challenges.

[diegoferigo]

I see you use SWIG in gym-ignition. How is that going? Would you recommend we use that throughout our stack or look into pybind11 as an alternative?

In the past few years we've got some experience with SWIG and pybind11, both have pros and cons, and the choice really depends on where the maintainers want to put their effort. From what I've read about your target, neither of them are 100% compatible. I'll keep this short, if I can provide more details let me know, I can elaborate more if needed. Here below a rather unorganized list of thoughts.

The most important difference between two options is the language support, pybind11 only supports Python (and any language that can call Python code like Matlab). So, if you really need Ruby, pybind11 is not an option [1]. On one hand, SWIG is fairly straightforward if the public APIs do not use extensively C++17 types (e.g. std::optional is not officially supported) and highly templated code with modern features. SWIG autogenerates by default all the classes / functions without requiring extra glue code, that is quite handy. On the other hand, pybind11 usage is very smooth, supports all existing (and future?) standards, works very good with templated code, and, more importantly, is actively maintained with a growing community and extensive documentation. With pybind11 is easier to be more creative and do particular things, instead if you get stuck with SWIG, well, good luck. Typemaps are quite difficult to implement and the engineering effort could quickly escalate. I often had to modify the APIs as workaround, which is ok if bindings are designed together with APIs, but it could not be straightforward for existing projects. The only pybind11 real downside is that each class / method / function has to be manually exposed, but with stable enough APIs like Ignition, I believe that the effort is manageable and it provides a direct entrypoint to perform minor APIs refactoring in order to obtain a more idiomatic experience in Python. As @traversaro suggested, third-party projects could offer to some extent an autogeneration pipeline, but we have never explored them yet. Last remark, pybind's native Eigen <-> NumPy support is pure gold if you need it.

In case you want to play with both of them, I maintain a simple example you can use as starting point. Maybe this can be useful.

This being said, from my knowledge and experience of your APIs, I'd feel to recommend playing with pybind11. However, the language support could be the real blocker here. Not sure if this is the best point to drop links, but especially for ign-gazebo, this functional documentation could be insightful. I'm not aware of any similar functionality offered by SWIG.

[1] Then, it would be interesting to understand whether Ruby is used extensively by anyone outside OR or, in the case Python bindings would become complete, if there would be any interest from the community of using Ruby instead of Python. For JS the situation is a bit different, but maybe WebAssembly or similar technologies could offer a more modern approach to expose C++ code to the web.

I see you use SWIG in gym-ignition. How is that going? Would you recommend we use that throughout our stack or look into pybind11 as an alternative?

Requoting the same question, for ScenarIO (the C++ backend behind gym-ignition) we use SWIG for historical reasons. When the project started we never used pybind11 and we didn't even considered it initially. Today, I often think to switch to pybind11, and it's likely to happen in a future major release.

[traversaro]

However, the language support could be the real blocker here. Not sure if this is the best point to drop links, but especially for ign-gazebo, this functional documentation could be insightful. I'm not aware of any similar functionality offered by SWIG.

Just FYI, cross-language callback functionalities are provided by SWIG "Directors" feature (see http://www.swig.org/Doc4.0/SWIGDocumentation.html#SWIGPlus_target_language_callbacks), but at the moment that feature does not support std::function (see swig/swig#341).

[sloretz]

I did a bit of investigation into python bindings generators for rclpy and summarized it in this google doc. There's a section with the advantages and disadvantages of SWIG. Ultimately we chose pybind11 for rclpy.

If ign-math bindings are based in swig and the ign-gazebo are based on pybind11, this interoperability may not be so straightforward

+1 to this, to give more detail: Say you're binding an Ignition Gazebo C++ function that accepts an ignition::math::Vector3d as an argument. Both SWIG and Python are going to generate code that converts to/from the C++ type ignition::math::Vector3d to a Python object. The Python objects they generate are different. An API generated with Pybind11 won't know how to automatically convert from the SWIG type - the conversion between the two would have to be coded by hand.

Does ign-math need a Python interface? Why would someone choose ign-math when numpy is available? If the justification is only because there's a desire to use scripts in ign-gazebo then there are other options. How about pybind11 bindings in ign-gazebo that accept numpy types in the API and do the conversion to ign-math internally? The advantage is it would be easier to interface with non-ignition libraries using numpy.

[chapulina]

Thanks for the inputs so far, everyone! We're discussing them all. I just wanted to address one comment which I think is important:

Does ign-math need a Python interface? Why would someone choose ign-math when numpy is available?

The equivalent for C++ is often asked: why would someone choose ign-math over something like Eigen?

It's important to point out that Ignition Math is not "just" another linear algebra library. It provides lots of robotics-specific functionality on top of a few basic types, which are really useful in the context of simulation and other robotic applications. Picking a few classes from the complete list:

• PID
• MassMatrix3
• SpeedLimiter
• Sphere, Cylinder, Box, Capsule, Ellipsoid
• Color
• Temperature
• Graph
• DiffDriveOdometry
• SphericalCoordinates
• ...

We recognize the advantages of specialized libraries like Eigen and NumPy, and strive to be inter-operable with them. We already provide conversion functions for Eigen, and once we have a Python API, it would be great to provide conversions to NumPy as well.

[scpeters]

We recognize the advantages of specialized libraries like Eigen and NumPy, and strive to be inter-operable with them. We already provide conversion functions for Eigen, and once we have a Python API, it would be great to provide conversions to NumPy as well.

I just noticed the eigenpy project that provides efficient bindings between numpy and Eigen using boost-python:

EigenPy provides:

full memory sharing between Numpy and Eigen avoiding memory allocation
full support Eigen::Ref avoiding memory allocation
exposition of the Geometry module of Eigen for easy code prototyping
standard matrix decomposion routines of Eigen such as the Cholesky decomposition, SVD decomposition, QR decomposition, and etc.
full support of SWIG objects

the boost dependency is not ideal, but otherwise it sounds pretty compelling

[azeey]

Looks like pybind11 has built-in support for Eigen and numpy https://pybind11.readthedocs.io/en/stable/advanced/cast/eigen.html.

[chapulina]

We added SWIG wrappers to all of Ignition Math, then converted them all to pybind11 wrappers and kept the SWIG files for the Ruby interface. That was a lot of work by @LolaSegura , @WagnerMarcos , @francocipollone and @ahcorde 👏

What's left to close this issue:

• Release the bindings and confirm that the debs work as expected
• Write a tutorial on getting started with ign-math Python bindings

[chapulina]

Done!