apt-get install drake-dev

[jamiesnape]

Relates #1183.

[jwnimmer-tri]

Per f2f -- first steps will be along the lines of #13168, trimming and clarifying our list of dependencies.

[jwnimmer-tri]

To be clear, the goal here is that the https://drake-apt.csail.mit.edu/ PPA offers these packages. We're not going to ITP them into Ubuntu nor Debian directly.

[jwnimmer-tri]

Bumping to high priority -- hope to be finished by mid-August.

[jwnimmer-tri]

Per our call last week -- while eventually we'd like to have fine-grained deb packages to minimize dependencies and footprint (e.g., shared libraries, vs dev headers, vs examples, vs pydrake, etc.), for now a worthwhile MVP would be just one large package similar to our current tarball with everything in it. We could carefully choose a name so that in the future it could turn into a meta-package that conveniently brings in (depends on) all of the smaller packages we'd eventually publish. Or, if it's easy and makes sense to carve out drake vs drake-visualizer packages (because the visualizer has the largest footprint), we could also do that as part of MVP. We'd like to release the first packages ASAP, so the fine grained packages will have to wait a little longer.

[jamiesnape]

Debian certainly don't make it easy to install to anywhere other than /usr, but we are making progress. First version of packages (drake-dev and drake-doc) will be available for people to test very soon (those two packages will be roll-up packages eventually, but for now they are the owner of all the relevant files).

[jamiesnape]

Things to clean up in future in-tree for all our packages:

• Ensure vtk explicitly links to all its dependencies.
• Fix file permissions for non-executables.
• Install models licenses into share/doc.
• Move binaries from share/drake to lib/drake.
• Patch director to remove shebangs from non-executable third-party Python files.
• Remove cruft from doxygen (*.map, *.md5, doxygen.log), sphinx (.buildinfo), empty jars (drake-lcm-spy.jar, lcm-logplayer-gui.jar, lcm-spy.jar), Bazel launcher scripts (cart_pole_passive_simulation, pendulum_lqr_monte_carlo_analysis, run_planar_scenegraph_visualizer), and add tests to keep everything clean.
• Remove unused headers (ibex/, maybe vtk/) and any unused CMake configuration.
• Strip .comment and .buildinfo sections from binaries.
• Use correct file extension for Python C/C++ extensions (e.g., .cpython-36m-x86_64-linux-gnu.so, etc.) and install to lib/python3/{dist-packages or site-packages}.

[jamiesnape]

A few issues with dh_strip on Bionic, but we already remove the important stuff, so not blocking. Focal was fine. 400+ lintian overrides and some suppressions but also not a big issue for packages not in Debian/Ubuntu repos.

I will upload some packages for testing tomorrow and then work on perfecting the automation.

[jamiesnape]

Note, configuration (drake-dev, drake-doc) is at https://github.com/RobotLocomotion/debian-drake (semi-restricted for now, will be public once packages go live). General versions and/or templates of the many of those files will land in this repo and be synced and/or generated there before each package build.

[jwnimmer-tri]

Current guess is nightlies will be 0.0.YYYYMMDD, and tagged builds can use the actual 0.23.0 or whatever.

[jwnimmer-tri]

First version of packages (drake-dev and drake-doc) will be available ....

I apologize, but I did not read this posting carefully enough upthread when it occurred. The drake-doc package is not part of the minimum viable first release for the apt packages. I did not intend it to be scoped into this ticket, and I failed to notice when it became so.

I don't want us to invest in supporting it on an ongoing basis. We don't publish compiled docs as part of the current release process, and I don't want to induce that extra maintenance cost for the apt package world. Please remove its supporting code from the work in progress and remove the drake-doc binaries from the apt site.

[jamiesnape]

Well, it was mentioned multiple times, is a common convention, there is a strong argument that all our packages should have docs, and indeed it is the only part of the realize process that is automated (for the website). Perhaps you might like to poll people whether offline documentation would be useful, especially as we moved it out of the main build, as it is barely extra maintenance cost over the website?

[jwnimmer-tri]

On the docs, I hesitate to commit to doc packages because I intend for us to completely rework the software tooling for documentation generation, by consolidating our wrapping tools and moving logic from drake-ci cmake into python within the drake tree directly, so keeping the only use cases as "developer instructions" and "jenkins" will make that easier.

For a victory condition for this ticket, I think a single debian package (with same contents as our binary release) is sufficient. That will satisfy users who want apt's automatic upgrades to happen (without the hassle of manual fetching tarballs), albeit with less fine-grained package sizes that might be necessary.

There are side benefits to additional splits of the packages, and to publish documentation snapshots, but we should file those as follow-up issues and put onto the back burner.

[jamiesnape]

0.26.0 drake-dev packages are now available (documentation is not, per the above). I guess we are pretty close to victory, given our current manual release process. Not quite the sleek Ferrari garage of packages that I was hoping for, more a single Fiat Multipla that needs a kick to get started each month.

[jwnimmer-tri]

I realized a bit more victory conditions I'd like to add. I'm fine with either checklisting them here or opening distinct issues:

(1) Document the existence of debs somewhere, perhaps https://drake.mit.edu/from_binary.html, including the recipe for installing.

(2) Port one of our online notebooks (colab, is it?) to use the apt stanza instead of the download-tgz stanza, as a final proof-of-life that the apt packages are satisfactory.

Depending on your input / preference, I could also see (3) update drake-external-examples to reference or incorporate the apt mechanism as a final bow on the present.

[jwnimmer-tri]

For (3) per f2f -- we should have CI for the apt packages, so adding drake-external-examples using apt sounds like a great way to kill two birds with one stone. We'll scope that in here.

[jamiesnape]

+@BetsyMcPhail to add the documentation. Minimal instructions are in the README.md for RobotLocomotion/debian-drake.

[jamiesnape]

(2) Port one of our online notebooks (colab, is it?) to use the apt stanza instead of the download-tgz stanza, as a final proof-of-life that the apt packages are satisfactory.

Probably does not make sense now that our preferred platform uses Python 3.7. I guess we could do something for binder, but I think we were planning on moving away from that anyway, so the utility would be limited.

(3) Is still in scope, though maybe we move that part to a separate issue either in this repo or over there so we can close this out once (1) is done.

[jwnimmer-tri]

Yup, I agree that we should drop (2), for those same reasons.

I'm fine if you'd like to split (3) into a new issue, or not, either way. A new issue should have equal priority to this one, i.e., we still finish it before starting other new work.

[jwnimmer-tri]

With the final two PRs shown in the timeline above, I think all work here is done. Let me know if I've forgotten anything.