Provide more granular libdrake-*.so targets for deployment / install

[EricCousineau-TRI]

Given the caveats listed in #7283 (polluting base libdrake.so with Gurobi and MOSEK on install, which requires the environment be setup), and the potential for other use cases, like including something like an optional libdrake-ros.so per Naveen and Russ's suggestion, to make ROS not a required dependency of libdrake and not need weird select() logic, it would be nice to have an established workflow for this, which is still convenient for development workflows (a la #6760).

\cc @naveenoid

[jwnimmer-tri]

Relates #1769 and #4131.

The plan so far is to get the CMake / MATLAB transition done, get drake-external-examples up and running under CI with platform review and pre-merge checks etc, get a stable release process and API set out the door, and only then start slicing Drake into smaller libraries.

[EricCousineau-TRI]

To follow up, would that mean that, if ROS were to come in as part of Drake, that it would not be a separate *.so until that point?
Or would we make an exception for something like libdrake-ros.so to permit it to be used without requiring that ROS be installed + /opt/ros/kinetic/setup.bash has to have been sourced? (I say "make an exception", because it seems like it would use the same procedure as slicing into smaller libs.)

(I am asking on behalf of a f2f with Naveen, namely to see what options might be available - and because it's semi-related to the Python bindings setup.)

[jwnimmer-tri]

Is this in the context of, e.g., publishing ROS sensor and TF messages from a simulation? Could we have a static executable do that (given a SDF, simulate it), instead of having libdrake.so involved? In any case, we can re-arrange the roadmap any way we want, but I don't think there's any clear write-up of what we're trying to do with ROS yet, so it's hard for me to say more.

[naveenoid]

Tagging @RussTedrake in

[naveenoid]

So the plan (evolved in the last couple of days) is now for drake to natively support subscribing and publishing ROS messages and this requirement is going to be a lot sooner than anticipated. Hence the original question.

[jwnimmer-tri]

As far as the build system goes, though, the relevant question is different.

Within the scope of the near-term priority request, do we only need that capability within a Drake program, or do we also need a Drake shared library that, e.g., speaks both ROS and drake::systems at the same time, and then some other second-party code links all the pieces together? In other words, is the place where ROS and Drake meet fully encapsulated within Drake's source tree, or do second-party library users need to mix and match Drake bits and ROS bits to their heart's desire?

Edit to add: And if the second-party users do need to mix and match, are they doing so via Bazel, CMake, or whichever we want?

[RussTedrake]

My two cents:

I think that we can initially segregate the ROS linking from libdrake.so, but this will be one of the first pieces where we are likely want to redistribute drake libraries with ROS inside.

I think that we should be able to use (very minimal pieces of) system-installed ROS, in the hopes that other projects that also use system ROS do not conflict. I’m also OK with saying that consumers not using system ROS (on both ends) will have to deal with building the relevant drake libraries from source — probably they would do this via cmake/catkin.

Does that help?

Cc @gizatt who might already have a useful demonstration of this

[gizatt]

The configuration Russ is referring to lives here in an experimental rejigger of the IIWA example to publish ROS Pointcloud2's. The relevant CMake target is rgbd_to_ros_pointcloud2_driver, and its linking connection to ROS is:

# Hah I'm circumventing catkin
find_package(roscpp REQUIRED)
find_package(image_transport REQUIRED)

add_library(rgbd_to_ros_pointcloud2_driver image_driver/rgbd_to_ros_pointcloud2_driver.cc image_driver/rgbd_to_ros_pointcloud2_driver.h)
target_include_directories(rgbd_to_ros_pointcloud2_driver PUBLIC 
    ${roscpp_INCLUDE_DIRS} 
    ${image_transport_INCLUDE_DIRS}
)
target_link_libraries(rgbd_to_ros_pointcloud2_driver
    z
    pthread
    ${roscpp_LIBRARIES}
    ${roscpp_LIBRARIES}
    ${image_transport_LIBRARIES}
    ${drake_LIBRARIES}
    ${gflags_LIBRARIES})

It compiles and runs! (Until it segfaults trying to send an lcmt_iiwa_status... but that's an unrelated issue that I think is my fault.)

[sammy-tri]

Within the scope of the near-term priority request, do we only need that capability within a Drake program, or do we also need a Drake shared library that, e.g., speaks both ROS and drake::systems at the same time

I think that in the near term we can get by linking a static executable from within drake which includes the ROS pub/sub goo we want in an example and avoid putting it into libdrake.so. (Russ, let me know if I'm wrong!) Eventually I believe the vision does include second party users.

In other words, is the place where ROS and Drake meet fully encapsulated within Drake's source tree, or do second-party library users need to mix and match Drake bits and ROS bits to their heart's desire?

I believe mix and match.

Edit to add: And if the second-party users do need to mix and match, are they doing so via Bazel, CMake, or whichever we want?

Probably whichever we want. I suspect CMake will be requested before bazel.

[jamiesnape]

Relates #7293 given issues on Mac with some versions of MATLAB.

[jwnimmer-tri]

Relates #7293 given issues on Mac with some versions of MATLAB.

I don't think this is correct, or at most its a second-order effect. All of libdrake's features should be available in MATLAB. If certain dependencies are difficult to get working with MATLAB, that would be a problem whether there is a single libdrake.so or a family of libdrake.so's -- at least one of the items in the family would still have the same problem(s) as the all-in-one libdrake does.

That's not to say we shouldn't have finer-grained dependencies or libraries, or to take into account users who only want to pay the dependency cost of a portion of Drake. However, in terms of the Drake project as a whole, and the burden on core developers, that MATLAB and some of our dependencies fight is a problem on its own, no matter how we package out librar(ies) for release.

[jamiesnape]

I disagree, some parts of Drake are just not applicable to MATLAB, e.g., MATLAB has its own visualization and so VTK would not be required.

[jwnimmer-tri]

The simulated RGB(D) sensor uses VTK at runtime, and will be an important part of the toolbox.

[jamiesnape]

Yes, and it would mostly likely (and probably will be required to) access that through protobuf.

[jamiesnape]

The point being there is a blacklist of libraries from the perspective from MATLAB and any functionality will be to be accessed some other way. Drake should segregate that functionality into a separate library that is safe for MATLAB, so it is exactly related to this issue.

[EricCousineau-TRI]

A couple of things have started to need this; considering a bump in priority at some point, possibly after Bionic + Python3 stuff all land.

[jwnimmer-tri]

While a few the "big picture" ideas that motivate the above discussions are still relevant, I think all of the actual details from the discussions above have been overtaken by events. We no longer link into matlab, and we have other plans for ROS integration.