can I separate build files and binaries from the source tree?

[sherm1]

Following the build-from-source instructions on Windows, the resulting CMake-generated files and binaries end up in the source tree (drake/build, drake/pod-build, drake/drake/pod-build at least). I would like to be able to keep the binaries separately, so I can maintain Win32 and Win64 binaries for the same source tree. Also for general source hygiene it is nice to be able to build from a read-only source tree.

Can this be done with the current build system? If not I think it would be a good modification.

[billhoffman]

@mwoehlke-kitware will your "unpod" build branch do this?

[mwoehlke-kitware]

it should... mostly... but not for libbot; that would need a separate change (since it is an external repo)

[RussTedrake]

A lot of thought went into the current design, but it was potentially based on
old practices. Let me try to convey the current model, to give you context and
re-open the discussion. It was based off a model called PODS which was a local
effort to handle complex dependencies:
https://sourceforge.net/p/pods/home/Home/
Perhaps the most important point is that we only ever do local “installs”.
install pushes things from the relative pod-build folders into the root build
folder.

We got used to working with lots of "POD collections”/super-builds, e.g. one for
each project, which all had the nice property that if you cd into a POD and type
make clean, you’d clean just that pod (by zapping pod-build). If you’re at the
super-build level then make clean would recursive clean all the pods (zap all
pod-builds and the root build).

PODS used to come in as git submodules (and svn externals before that). We've
transitioned to using cmake external projects to organize our superbuild.
Running make at the superbuild level can still take a long time. In my view,
this is because pods, cmake external projects, and essentially every tool like
this that I know about has the problem that they mix up the configure step with
the build step. I can’t configure a dependent project until i completely build
the dependee. So make runs configure and make on each subproject, which takes
time.

For this reason, I’ve still favored the pods-style organization, where it’s
always possible to cd into a subproject and build just that project. You’ll find
that you’re happier with drake if you cd into drake-distro/drake and build there
most of the time. Similarly, if you ever want to work on an external, you can cd
into that external and work locally.

The only problem that I see with this is that it is not the usual way.
Especially for cmake folks. Maybe especially for msvc folks. This might be
reason enough to consider changing it back. But i think the expert developer
experience will actually get worse (longer build times). Another solution is
just to document this better.

The workflow I’ve been happiest with on windows has been to generate and make
once at the super-build level, then open the project/soln files in the
drake-distro/drake level and build in visual studio (selecting debug/release,
etc) from then out. If that doesn’t work for you, then it should.

[jamiesnape]

A few issues I have with the current build system is that:

1. PODs also makes life more difficult for getting all the necessary results to CDash.
2. The current workflow installs various packages globally via apt-get or homebrew.
3. Builds are in-source.
4. In general, developers have not seen the PODs workflow before.
5. PODs makes it more difficult in add new operating systems or use alternative IDEs.
6. PODs is outdated and less well maintained.

I favor, in the following order:

1. The CMake workflow.
2. The ROS workflow.
3. The PODs workflow.

[jamiesnape]

If there were some functionality that we need here that a pure CMake workflow cannot achieve, then it is may be possible we could address it in a future update to CMake.

[RussTedrake]

Thanks @jamiesnape

1. understood. but that can be resolved. optimizing the development cycle is more important.
2. I don't think the CMake or ROS workflow would get around the prereqs via apt-get or homebrew.
3. I understand why you say that. But for me, they are out of source. But there are multiple builds.
4. agreed.
5. at the super-build level. but not at the subproject level, i think. think of drake-distro/drake as the target for your IDE and everything ok again.
6. totally agree. the name/association to PODS has no value here.

again, my main (only?) argument for PODs is the performance / convenience of compiling only the subproject you care about. for me, that trumps all of the above.

in drake's superbuild, almost none of the subprojects ever change, except for drake. that's why they're built into "externals". in our big DRC code-base, we had subprojects and a separate externals project which was not built on make. the install instructions had you explicitly cd into externals and run make their once, and plan to never do it again (of course it was not that simple in practice). perhaps a solution like that is the most reasonable alternative?

[jamiesnape]

1. My point here is that you would be able to locally use all the tools you see running on Jenkins and have all your local builds fully showing on CDash, including dependencies, which to me is a more optimized development cycle.
2. True, but my problem is more than they do install prereqs rather than they do not. I favor comprehensive documentation and formal provisioning scripts rather than integrating prereq installs with the build. Having a single install_prereqs file would be step toward that, I suppose. Either way I see several issues in the tracker where people are confused by the current workflow. It does feel very unusual to me.
3. The useful thing for me would be to have separate build directories and to be able to have multiple build directories for different compilers and/or cross-compilers. These would also mean not relying on environment variables to set options.
5. New operating systems, architectures, and certain compilers would be troublesome, I think. That work has already been done by CMake, so adding such would be much easier with a standard workflow. Of course, my whole argument assumes that there would be more operating systems, architectures and compilers that needed to be supported, and I do not know whether that is the case.

Your last two paragraphs are very useful. I think the best thing would be the work out the most important features that the build system for Drake needs. Whether it is PODs, CMake superbuild, or anything else is less important.

[sherm1]

With Kitware expertise on tap, would it be reasonable to consider a Kitware-best practices proposal for Drake builds that would meet all the goals you stated Russ, plus Jamie's and others?

[RussTedrake]

absolutely!

[mwoehlke-kitware]

To add my 2¢...

PODS used to come in as git submodules (and svn externals before that). We've transitioned to using cmake external projects to organize our superbuild.

Interestingly, @bradking mentioned submodules in our internal meeting today. We may want to go back to those, but keeping them "driven" by CMake external projects.

Running make at the superbuild level can still take a long time. In my view, this is because pods, cmake external projects, and essentially every tool like this that I know about has the problem that they mix up the configure step with the build step. I can’t configure a dependent project until i completely build the dependee. So make runs configure and make on each subproject, which takes time.

As I'm sure you know, if A depends on B, you can't build B until you have built A. (At best, you can tell your build tool about both so it can interleave their builds with full dependency ordering, but that's not practical for us.) I think the problem here is that you are forcing a configure step every time, when it should only be necessary (at least for CMake projects) to force a build step... if the configuration is stale, the build step will re-run it.

In my experience, this can be made to happen in a reasonable time when set up correctly, especially if using ninja, which does no-op builds much more quickly than make. (In this case, I believe it may be the download steps taking most of the time.)

For this reason, I’ve still favored the pods-style organization, where it’s always possible to cd into a subproject and build just that project. Similarly, if you ever want to work on an external, you can cd into that external and work locally.

This isn't specific to PODS, though. You can do this with any CMake external project. (FWIW, though, this is one reason why I personally don't like external projects in any form. Especially working on the code of dependencies I find much easier when they aren't coupled by a superbuild.)

I think the expert developer experience [if we move away from PODS] will actually get worse (longer build times).

I think we (i.e. MIT and Kitware) must not be on the same page. As I understand it, those parts of PODS under discussion, and which are unique to PODS, deal with particular directory structures and items such as providing a "PODS Makefile" that should not have any impact on build times. As I understand, the aspects that you are pointing out and saying that you like are shared by CMake external projects (and in fact, are currently being provided through CMake external projects). I don't believe we're proposing to get rid of those. IOW, we agree that the good features are good 😄.

Builds are in-source.

I understand why you say that. But for me, they are out of source.

I think there may be some terminology confusion here. Some folk consider "out of source" to mean that neither the source nor build trees are subsets of another (e.g. my source and build directories are about six levels deep with only two levels of common prefix). The more critical issue, however (this Issue, in fact, i.e. #1859), is that PODS does not allow one to control the location of build artifacts, which makes it impossible to have multiple coexisting builds from a single source tree.

[jhoare]

For reference on this, some discussion about moving some of the PODS/submodules to build out of source is here: #1884 (comment)

[david-german-tri]

This is in large part done, and the remaining work is covered in #2482.