Optionally keep external source in build directory

[mwoehlke-kitware]

Add an option (only available in non-PODS configuration¹) to keep the sources for external projects in the build directory, rather than the source directory. This ensures that it is always possible to have multiple build trees for a single (drake) source tree, at the expense of added inconvenience and disk usage (because the sources for external projects are duplicated).

(¹ i.e. running CMake directly rather than using the root Makefile.)

Note: This is OFF by default. CI will only test that adding the option didn't break anything.

This should fix #1859 and possibly #2094 (when using the option), though I'm on the fence if I want to close the issue(s) when this lands. I think we should still strive to fix individual packages that "pollute" the source tree when not using this option.

---

This change is

[sherm1]

@mwoehlke-kitware, I tried this in Windows. First I deleted my drake-distro/externals directory, then I built using an out-of-source build directory. Most of the externals did go into the build directory as expected. However, a new drake-distro/externals directory got created with some cruft. For example, here is drake-distro/externals/eigen:

The list of polluters was: bot_core_lcmtypes, bullet, eigen, googletest, gtk, lcm, spotless, swigmake, and yaml_cpp.

Is this the behavior you are expecting?

[mwoehlke-kitware]

@sherm1; sanity check, did you turn on SOURCE_PURE? Your description sounds consistent with the current (i.e. without this PR) behavior, which makes me wonder if you have SOURCE_PURE off. (Note that it is off by default...)

[mwoehlke-kitware]

Something is fishy... I just tried on my Windows machine, and I don't even get an externals directory in my source tree, but I also don't get the sources where they're supposed to be...

[mwoehlke-kitware]

Oops; had copy/paste errors in some variable names. Should be fixed now. (Build on my Windows machine is in progress¹, but seems okay so far, and sources are in the expected location.)

Don't forget to turn on SOURCE_PURE 😄.

(¹ [86/126] Performing build step for 'bullet'... no failures to that point...)

[sherm1]

@mwoehlke-kitware -- sorry I missed SOURCE_PURE before. Updated from master then pulled your latest changes, and this time ran with SOURCE_PURE set ON (I do not have Matlab). Used an out-of-source build directory. Built ALL_BUILD target from within Visual Studio 2015 Win64, RelWithDebInfo configuration. Got 4 failures, including:

21>..\lcm\lcm.c(8): fatal error C1083: Cannot open include file: 'glib.h': No such file or directory [C:\Users\Sherm\Documents\GitHub\sherm1\drake-distro-build-rtest\externals\lcm\build\lcm-prefix\src\lcm-build\lcm-python\install_python.vcxproj] [C:\Users\Sherm\Documents\GitHub\sherm1\drake-distro-build-rtest\externals\lcm\build\lcm.vcxproj]
23>  -- Found eigen3
23>  -- barycentricInterpolation will be skipped because matlab was not found
23>  -- Performing Test COMPILER_HAS_DEPRECATED_ATTR
23>  -- Performing Test COMPILER_HAS_DEPRECATED_ATTR - Failed
23>  -- Performing Test COMPILER_HAS_DEPRECATED
23>  -- Performing Test COMPILER_HAS_DEPRECATED - Success
23>  -- Could NOT find yaml-cpp (version >= ) using pods_find_pkg_config. PKG_CONFIG_PATH = C:\Users\Sherm\Documents\GitHub\sherm1\drake-distro-build-rtest\drake\lib\pkgconfig;C:\Users\Sherm\Documents\GitHub\sherm1\drake-distro-build-rtest\install\lib\pkgconfig
23>  -- Could NOT find gurobi (version >= ) using pods_find_pkg_config. PKG_CONFIG_PATH = C:\Users\Sherm\Documents\GitHub\sherm1\drake-distro-build-rtest\drake\lib\pkgconfig;C:\Users\Sherm\Documents\GitHub\sherm1\drake-distro-build-rtest\install\lib\pkgconfig
23>  -- Could NOT find nlopt (version >= ) using pods_find_pkg_config. PKG_CONFIG_PATH = C:\Users\Sherm\Documents\GitHub\sherm1\drake-distro-build-rtest\drake\lib\pkgconfig;C:\Users\Sherm\Documents\GitHub\sherm1\drake-distro-build-rtest\install\lib\pkgconfig
23>  -- Could NOT find snopt_c (version >= ) using pods_find_pkg_config. PKG_CONFIG_PATH = C:\Users\Sherm\Documents\GitHub\sherm1\drake-distro-build-rtest\drake\lib\pkgconfig;C:\Users\Sherm\Documents\GitHub\sherm1\drake-distro-build-rtest\install\lib\pkgconfig
23>  -- Could NOT find gurobi (version >= ) using pods_find_pkg_config. PKG_CONFIG_PATH = C:\Users\Sherm\Documents\GitHub\sherm1\drake-distro-build-rtest\drake\lib\pkgconfig;C:\Users\Sherm\Documents\GitHub\sherm1\drake-distro-build-rtest\install\lib\pkgconfig
23>  -- Could NOT find snopt_c (version >= ) using pods_find_pkg_config. PKG_CONFIG_PATH = C:\Users\Sherm\Documents\GitHub\sherm1\drake-distro-build-rtest\drake\lib\pkgconfig;C:\Users\Sherm\Documents\GitHub\sherm1\drake-distro-build-rtest\install\lib\pkgconfig
23>  -- Found bullet
23>  -- checking for module 'bullet'
23>  --   found bullet, version 2.83
23>  -- Could NOT find snopt_c (version >= ) using pods_find_pkg_config. PKG_CONFIG_PATH = C:\Users\Sherm\Documents\GitHub\sherm1\drake-distro-build-rtest\drake\lib\pkgconfig;C:\Users\Sherm\Documents\GitHub\sherm1\drake-distro-build-rtest\install\lib\pkgconfig
23>  -- Could NOT find gurobi (version >= ) using pods_find_pkg_config. PKG_CONFIG_PATH = C:\Users\Sherm\Documents\GitHub\sherm1\drake-distro-build-rtest\drake\lib\pkgconfig;C:\Users\Sherm\Documents\GitHub\sherm1\drake-distro-build-rtest\install\lib\pkgconfig
23>  -- Writing addpath_drake.m and rmpath_drake.m to C:/Users/Sherm/Documents/GitHub/sherm1/drake-distro-build-rtest/install/matlab
23>  -- Configuring done
23>  -- Generating done
23>...drake-distro/drake/systems/plants/RigidBodyFrame.h(6): 
fatal error C1083: Cannot open include file: 'drake/drakeRBSystem_export.h': No such file or directory [C:\Users\Sherm\Documents\GitHub\sherm1\drake-distro-build-rtest\drake\systems\plants\drakeRBM.vcxproj]

[sherm1]

Also, it did create drake-distro/externals in my source directory, with the same set of subdirectories, but now those subdirectories are all empty.

[mwoehlke-kitware]

..\lcm\lcm.c(8): fatal error C1083: Cannot open include file: 'glib.h': No such file or directory

Oh. Oh, lovely. It took a while to track down what's even trying to compile this. Turns out it is setup.py (yes, a Python script invoking the C++ compiler!) in LCM itself. Apparently lcm — at least its Python component — has a hard requirement¹ that glib is in a very particular relative path from the working directory when said setup.py is executed.

I'm... just going to refrain from comment on that...

(¹ Unless GLIB_PATH is set, which apparently it isn't.)

Also, it did create drake-distro/externals in my source directory

I went back and did a VS build, and did see this... but after deleting it, nothing is recreating it. I also can't figure out what is creating it in the first place.

[jwnimmer-tri]

(Adding "do not review" and "do not merge" to take the place of "waiting for code update" that got nixed a couple weeks ago.)

[jwnimmer-tri]

@mwoehlke-kitware Does this PR still make sense? I wonder if #2487 has superseded it, perhaps?

[mwoehlke-kitware]

Does this PR still make sense? I wonder if #2487 has superseded it, perhaps?

1. Probably not, and 2) not "superseded" perhaps, but the two are incompatible. At any rate, I believe we're more or less committed to killing off in-source builds. Closing.