NumPy 2.0 support (AttributeError: "itemset")

[jwnimmer-tri]

Is your feature request related to a problem? Please describe.

NumPy 2.0 has been released. See https://numpy.org/devdocs/numpy_2_0_migration_guide.html.

Currently, importing certain pydrake versions crashes with this error:

AttributeError: `itemset` was removed from the ndarray class in NumPy 2.0.

Known to fail:

• stable releases 1.30 and earlier
• nightly builds 0.0.20240618 and earlier

Describe the solution you'd like

Drake should be compatible with both NumPy 1.x and 2.x.

Describe alternatives you've considered

In the meantime, pydrake users should depend on numpy < 2 in their requirements file.

Alternatively, you can use a nightly build (which itself depends on numpy < 2).

[jwnimmer-tri]

For starters we need to merge up our pybind11 fork to incorporate pybind/pybind11#5050, i.e., to v2.12.0. That means:

• Open a pull request to https://github.com/RobotLocomotion/pybind11/ that merges the upstream from 2.11 to 2.12.
  • To to this, make a local branch and do the merging.
• Open a pull request to Drake that bumps our COMMIT pin to your branch's git sha (so that we get CI coverage).

[jwnimmer-tri]

To unbreak our wheel builds, I believe can do something along these lines:

--- a/tools/wheel/image/provision-python.sh
+++ b/tools/wheel/image/provision-python.sh
@@ -42,7 +42,7 @@ ln -s /usr/local/bin/python /usr/bin/python
 # keeping them up to date if they are?
 pip install \
     matplotlib \
-    numpy \
+    'numpy < 2' \
     pyyaml \
     semantic-version \
     setuptools \

[ggould-tri]

https://drake-jenkins.csail.mit.edu/view/Production/job/linux-jammy-unprovisioned-gcc-wheel-nightly-release/111 is this issue; I am using the magic word //:install in this comment so that future buildcop searches will find this issue when they search for the failing target.

[jwnimmer-tri]

I think possibly the last remaining obstacle is to figure out how to rework RobotLocomotion/pybind11@0da7ac7 to no longer rely on itemset, which is related to pybind/pybind11#1181 and pybind/pybind11#1152.

[jwnimmer-tri]

The first thing to try is if we can call __setitem__ with a slice, instead of itemset.

[bernhardpg]

FYI If I now try to do a fresh install of Drake and install all the prereqs I am still running into this issue.

I run:

python3 -m venv drake_env
source drake_env/bin/activate
cd drake
./setup/mac/install_prereqs.sh

While I don't get an error from the above setup, I do run into the AttributeError: `itemset` was removed from the ndarray class in NumPy 2.0. error when I try to build and install Drake from source.

It seems that brew is still installing NumPy 2.0 and that numpy < 2 in requirements.txt only affects a different numpy install (installed by pip) which is not the one used when building.

I can run succesfully run brew uninstall numpy but then trying to build and install with Bazel gives the new error (it does not help to run bazel clean --expunge)

AttributeError: module 'numpy' has no attribute 'lib'

Is there a current workaround for this? Or is it currently not possible to build Drake from source on mac?

[bernhardpg]

Update/temporary fix: I found that applying this change WITHOUT setting up in a virtual environment seems to have worked without any issues for me (so far)

[jwnimmer-tri]

On the Drake master branch as of just now, we hope that numpy 2.0 works correctly.

Tomorrow's nightly wheel build (i.e., drake >= 0.0.20240710) should also work.

Next week's stable release 1.31.0 should also work.

---

The actions remaining on this ticket are to investigate and clean up a couple of CI tests that we had to disable in order to pass:

• bindings/pydrake/gym/test/drake_gym_test.py
• bindings/pydrake/examples/gym/play_cart_pole.py