Formulae with python bindings only build against 1 version of python

[scpeters]

Currently our formulae that build python bindings use [email protected] (see #2580). There are python packages in homebrew-core that support multiple versions of python using pip (see pybind11 for example), and a few that use meson (see py3cairo for example).

Currently this is a problem because [email protected] has just been made the default python version in Homebrew/homebrew-core#182840, so any CI scripts that assume which python resolves to [email protected] will cause test failures:

• https://build.osrfoundation.org/job/sdformat-ci-sdf15-homebrew-amd64/15/

Also, any users attempting to use python bindings on macOS will fail if they don't explicitly use [email protected].

Options to fix this:

• Currently our documentation for building from source with macOS recommends installing the latest version of python3. So whenever a new python3 version is made default; we could invalidate all bottles and rebuild for the next python3 version
• Declare a specific python version as supported for each Gazebo collection on macOS. This would reduce our support matrix but would require updating documentation and some user action to use older python versions.
• Figure out how to build bindings for multiple versions of python in our formulae, and rebuild bottles to add support for new python versions when they are released.
  • Ionic
    • gz-math
      • gz-math8: patch to build multiple python bindings #2839
      • Permit building python bindings separately from gz-math library gazebosim/gz-math#636
    • sdformat
      • Permit building python bindings separately from libsdformat library gazebosim/sdformat#1491
      • sdformat15: support multiple python versions #2836
    • gz-msgs gz-msgs11: support multiple python versions #2842
    • gz-transport
    • gz-sim
  • Harmonic
    • gz-math
      • gz-math7: patch to build multiple python bindings #2840
      • Permit building python bindings separately from gz-math library (backport #636) gazebosim/gz-math#640
    • sdformat
      • sdformat14: support multiple python versions #2849
      • Permit building python bindings separately from libsdformat library (backport #1491) gazebosim/sdformat#1497
    • gz-msgs gz-msgs10: support multiple python versions #2843
    • gz-transport
    • gz-sim

[scpeters]

edited to add a 3rd option

[scpeters]

• Currently our documentation for building from source with macOS recommends installing the latest version of python3. So whenever a new python3 version is made default; we could invalidate all bottles and rebuild for the next python3 version

this is not fun, but matches what we have to do when other dependencies are updated; it shouldn't be too frequent, though it could break existing workspaces using the older python version

[scpeters]

• Figure out how to build bindings for multiple versions of python in our formulae, and rebuild bottles to add support for new python versions when they are released.

the only way I can think to do this is to build each package multiple times with different -DPython3_EXECUTABLE=... cmake arguments, which will slow down formula build time, though it shouldn't affect regular CI builds which will use a single python version

[traversaro]

• Figure out how to build bindings for multiple versions of python in our formulae, and rebuild bottles to add support for new python versions when they are released.

the only way I can think to do this is to build each package multiple times with different -DPython3_EXECUTABLE=... cmake arguments, which will slow down formula build time, though it shouldn't affect regular CI builds which will use a single python version

Not sure if it is useful in this context , but in conda packages I build the C++ library and the Python library as independent CMake projects, in this way I can just build the C++ library once and the Python bindings for each python supported version, to do so I have a small patch (like https://github.com/conda-forge/gz-math-feedstock/blob/main/recipe/standalone_bindings.patch) for each project. The only exception for this is gz-sim, there to have support for systems written in Python you need to rebuild also the C++ version for each Python minor version.

[scpeters]

Not sure if it is useful in this context , but in conda packages I build the C++ library and the Python library as independent CMake projects, in this way I can just build the C++ library once and the Python bindings for each python supported version, to do so I have a small patch (like https://github.com/conda-forge/gz-math-feedstock/blob/main/recipe/standalone_bindings.patch) for each project. The only exception for this is gz-sim, there to have support for systems written in Python you need to rebuild also the C++ version for each Python minor version.

thanks, that's helpful context. I think we could consider that patch for upstream. If you don't mind, I'll cherry-pick that patch from traversaro/ign-math@d02282e and open a pull request against gz-math8 to see what other maintaienrs think

[traversaro]

If you don't mind, I'll cherry-pick that patch from traversaro/ign-math@d02282e and open a pull request against gz-math8 to see what other maintaienrs think

Sure. I never did that myself as I wanted to avoid to add a patch for an untested workflow in which upstream did not have interest, and the patch was small enough that it was not problematic to mantain it downstream, but I will be happy if that is integrated upstream.

[scpeters]

If you don't mind, I'll cherry-pick that patch from traversaro/ign-math@d02282e and open a pull request against gz-math8 to see what other maintaienrs think

Sure. I never did that myself as I wanted to avoid to add a patch for an untested workflow in which upstream did not have interest, and the patch was small enough that it was not problematic to mantain it downstream, but I will be happy if that is integrated upstream.

prototyping in gazebosim/gz-math#636 and #2836