numpy: Should use later version of NumPy

[EricCousineau-TRI]

We're feeling some pain from being stuck to Ubuntu 18.04's old version of NumPy (currently 1.13.3, released ~2.5 years ago).

One main issue is supporting #10461, which is useful for things like this:
RussTedrake/underactuated#333

Some potential complications:

• Dependencies. We may not be able to use as many NumPy-dependent Ubuntu packages. May require figuring out how to consume a PIP environment.
  • That being said, NumPy does an awesome job of ABI compatibility.
• Installation and distribution: If we use a distribution of platform that is not easily installed by the platform, it may require some finagling (either vendoring the version, or making PIP / virtualenv a requirement).

\cc @RussTedrake @jwnimmer-tri

[jamiesnape]

Focal is going to be 1.17.4 as it stands. If we want to change, ideally it would be to that (on Bionic).

[jamiesnape]

My opinion would be "some pain" is probably not enough to warrant moving, though. If there is a feature you need that cannot be worked around in a reasonable way, then that would be a reason to change, but you are likely to find that the moving away from apt is going to be very painful, so that needs to be balanced.

[jwnimmer-tri]

Yes. To the extent we need pip versions instead of apt versions on Ubuntu as runtime dependencies, I think we need pip install drake (#1183) first, along with WORKSPACE support for ingesting pip. Trying to use pip prerequisites in a non-pip-managed pydrake install sounds like a nightmare. (To be clear, cherry picking build-time prereqs from pip can be fine -- because we don't need to install those, the blast radius is more contained.)

I've been thinking about Ubuntu 20.04 timelines a bit. Realistically, it seems like late 2020 (at the earliest) would be the earliest we could drop 18.04 support. Might be worth opening a issue(s) soon to discuss both (1) timeline for 20.04 spin up, and (2) timeline for 18.04 spin down. Edit: #13102 for 20.04 spin-up; #13391 for 18.04 spin down.

Also relates: #8116 and several other discussions we've had in the past about bleeding edge numpy.

[jwnimmer-tri]

@EricCousineau-TRI Can you speak to whether 1.17.4 on Focal is sufficiently new to give us the matmul support that we need?

If yes, I'm tempted to say the answer here is just "Use Focal or macOS", and leave Bionic undisturbed, from a conservation-of-effort perspective.

[EricCousineau-TRI]

From looking at the tags including this NumPy commit, yes, it looks like NumPy>=1.17.0 will have matmul:
numpy/numpy@8bf8d28

So yes, I'm more than happy with that resolution!

[EricCousineau-TRI]

Would you like me to close this now, or wait til we have CI support for Focal? #13102
https://drake-jenkins.csail.mit.edu/view/Linux%20Focal/

[jwnimmer-tri]

I think my vote is to resolve this as duplicate of #10461 (and possibly pasting some of this consensus there). That issue is the outcome we objectively care about.

It's probably worth prototyping the matmul solution once Focal is slightly more blue; within the next two weeks or so, I expect it to get added to Production CI.

[EricCousineau-TRI]

SGTM. Resolution is this comment and the one below: #12878 (comment)

Closing as duplicate of #10461, will post there.