Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update from @jhoare and @billhoffman to fix windows builds #1843

Merged
merged 4 commits into from
Mar 11, 2016

Conversation

RussTedrake
Copy link
Contributor

also has the hot fix from @psiorx
resolves #1836

@RussTedrake
Copy link
Contributor Author

@jhoare - PTAL at jenkins 64 bit?

@jhoare
Copy link

jhoare commented Mar 10, 2016

looking into it now.

@jhoare
Copy link

jhoare commented Mar 10, 2016

I don't know why its happening, but it looks like Appveyor (which is passing) is using a path to linker: C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\x86_amd64\link.exe while Jenkins is using a different path (presumably to the 32 bit linker?) C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\link.exe

@jamiesnape
Copy link
Contributor

Which Jenkins build? Both?

@jhoare
Copy link

jhoare commented Mar 10, 2016

I was referring to the windows 64 bit builds for this pull request. I'm not sure if it was always this way or if something has changed.

@billhoffman
Copy link
Contributor

John and I tracked down the lcm on win64 issue.

https://github.com/RobotLocomotion/drake/blob/master/appveyor.yml

The problem is here:
# 64-bit build
- CMAKE_FLAGS: -G "Visual Studio 14 2015 Win64"
# 32-bit build
- CMAKE_FLAGS: -G "Visual Studio 14 2015"

We were not setting this environment variable in the Jenkins build script. John made the lcm a non-cmake pod that uses the Makefile. The Makefile does not automatically get the CMake generator flag so it defaults to building lcm 32 bit which has errors. For now, I will add this to the build script we are using to get things working.

I would think it odd to require people to set this environment variable to get a correct 64 bit build. Is there another way to pass a CMake generator to a Makefile pod?

@RussTedrake
Copy link
Contributor Author

i always thought it was odd that cmake did not choose the 64 bit generator by default. isn't that the real source?

@billhoffman
Copy link
Contributor

No, then it would fail on the 64 bit builds. We can not rely on a CMake default. There are two options on windows 64/32 no matter which one is default, if you try and use the one that is not default it needs to be passed into all the projects. ExternalProject does this by default with CMake projects. But, if we are calling make that then calls cmake the -G needs to be passed along.

@jhoare
Copy link

jhoare commented Mar 10, 2016

I think the right solution here for lcm is to do what @RussTedrake suggested in #1368 and turn the lcm-pod into a cmake pod (with the source at the root of the repository, rather than a subdirectory). I tried to do the "easy" approach of just treating like a Makefile pod, but that seems to be brittle, as we are seeing here.

@billhoffman
Copy link
Contributor

I agree, but my concern is for other Makefile pods that might not be passing the generator around. In this case, it should be a CMake pod.

@RussTedrake
Copy link
Contributor Author

what is the resolution?

@billhoffman
Copy link
Contributor

Do you think it makes sense to add a CMAKE_GENERATOR to the Makefile pods?

ifneq "$(CMAKE_GENERATOR)" ""
CMAKE_FLAGS+=-G"$(CMAKE_GENERATOR)"
endif

Other option would be to do something like this:
set(${proj}_BUILD_COMMAND ${CMAKE_COMAND} -E env CMAKE_FLAGS="-G "${CMAKE_GENERATOR}"" ${PODS_MAKE_COMMAND} BUILD_PREFIX=${CMAKE_INSTALL_PREFIX} BUILD_TYPE=$)

For now I have added the env var to the build script we use on Jenkins and builds are looking good. I think for LCM @jhoare will change this to a cmake pod. But that does not address the issue of making sure that make pods calling cmake use the correct generator.

@RussTedrake
Copy link
Contributor Author

The way I set things up initially is that we always passed the CMAKE_FLAGS through to the makefile subprojects. Is that not working anymore?

@jamiesnape
Copy link
Contributor

You used to use an environment variable for the CMAKE_FLAGS.

@RussTedrake
Copy link
Contributor Author

yes. not anymore?

@jamiesnape
Copy link
Contributor

We add them to the initial CMake cache now.

@RussTedrake
Copy link
Contributor Author

on the build server only? the instructions for users still say to set the
environment variable.

@jamiesnape
Copy link
Contributor

On the build servers. Developers would typically use the GUI or pass the flags as command line arguments to CMake.

@billhoffman
Copy link
Contributor

OK, I missed this: http://drake002.csail.mit.edu/drake/sphinx/windows.html Create a new system environment variable CMAKE_FLAGS and set it to, e.g. -G "Visual Studio 14 2015 Win64". That is lets say a very un-cmake way of doing things... :) This still works, but I guess I would like to change it if possible. If you use a generator on the top level it should get propagated down without having to set environment variables.

@RussTedrake
Copy link
Contributor Author

I’m happy to update it. But that’s the way that we’ve been working, and might be
the fastest way to get the build servers healthy again.

On Thu, Mar 10, 2016 at 4:29 PM, Bill Hoffman [email protected] wrote:
OK, I missed this: http://drake002.csail.mit.edu/drake/sphinx/windows.html
[http://drake002.csail.mit.edu/drake/sphinx/windows.html] Create a new system environment variable CMAKE_FLAGS and set it to, e.g. -G
"Visual Studio 14 2015 Win64". That is lets say a very un-cmake way of doing
things... :) This still works, but I guess I would like to change it if
possible. If you use a generator on the top level it should get propagated down
without having to set environment variables.


Reply to this email directly or view it on GitHub
[https://github.com//pull/1843#issuecomment-195057539] .[https://github.com/notifications/beacon/AGJNNLlAbyrv8Jy0nffUDyJLEOt2dvLSks5psI0tgaJpZM4HtpBC.gif]

@jamiesnape
Copy link
Contributor

retest this please

@billhoffman
Copy link
Contributor

I did set it in the build scripts, i just want to figure it out for the long term. The build scripts seem to be working very well right now. I think all errors are real at this point.

@RussTedrake
Copy link
Contributor Author

Except that it is not catching the error that should be caught on #1818 on
win32?

@billhoffman
Copy link
Contributor

What error?
I don't see anything here:
https://drake-jenkins.csail.mit.edu/job/experimental/236/compiler=msvc-32,label=windows/consoleFull

Or in the cdash report:
https://drake-cdash.csail.mit.edu/index.php?project=Drake&showfilters=1&filtercount=2&showfilters=1&filtercombine=and&field1=label&compare1=61&value1=jenkins-windows-msvc-32-experimental-1818-236&field2=buildstarttime&compare2=84&value2=now

The test failure is not being caught by CMake:
https://drake-cdash.csail.mit.edu/testDetails.php?test=23136&build=1748
This is the find being used issue that has not been fixed. Must be that the test returns 0 when this happens so CTest marks it as passed. Is that what should be failing?

@RussTedrake
Copy link
Contributor Author

appveyor fails on the pendulum urdf test on 32-bit. it should fail. there
was a
make_shared instead of an allocate_shared, which does not use byte-aligned
allocation. the fix in this PR. but until this is merged, that other PR
should
fail.

@billhoffman
Copy link
Contributor

I see now:
21/24 Test #21: pendulumURDFDynamicsTest .................***Exception: SegFault 0.05 sec
Not sure why it is not segfaulting on the AWS instance. It is a segfault, maybe just getting unlucky with memory layout...

@RussTedrake
Copy link
Contributor Author

These tend to be very reliable segfaults on win32.

@billhoffman
Copy link
Contributor

I can not explain it... Could it have something to do with this stuff:

21: FIND: Parameter format not correct
21: FIND: Invalid switch
Maybe it does not run the test without FIND??

@RussTedrake
Copy link
Contributor Author

No. that’s standard output (and a known issue). I’m worried that the win32 build
on jenkins is actually compiling as win64.

@billhoffman
Copy link
Contributor

No, that is not happening, you can see it in the full output:

https://drake-jenkins.csail.mit.edu/job/experimental/compiler=msvc-32,label=windows/243/consoleFull

If you search for cl.exe you see
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\CL.exe
Which is the 32bit one. You can also see "-GVisual Studio 14 2015" being used.

@RussTedrake
Copy link
Contributor Author

travis is failing on "unknown command: cmake_parse_arguments" in lcmtypes.cmake. @jhoare -- Is that from your change? we must be using a cmake feature beyond our minimum required version?

@RussTedrake
Copy link
Contributor Author

that's the error on drake006, too.

@jhoare
Copy link

jhoare commented Mar 11, 2016

Yes it was likely from my change, since its in the bot_core_lcmtypes package. I copied it from https://github.com/RobotLocomotion/cmake/blob/master/lcmtypes.cmake though, which I thought was being used throughout the system, so I don't see why it is a problem.

I apologize but I won't be able to look at it until Monday, since I'll be out of the office for another project.

@billhoffman
Copy link
Contributor

I think it is just missing an include:
lcmtypes.cmake:cmake_minimum_required(VERSION 2.8.3) # for the CMakeParseArguments macro
mex.cmake:include(CMakeParseArguments)

The newest version of CMake has that as a built in function. So, all we need to do is add include(CMakeParseArguments)
to lcmtypes.cmake for a fix.

@RussTedrake
Copy link
Contributor Author

great. thank you @billhoffman. can you PR?

On Thu, Mar 10, 2016 at 8:44 PM, Bill Hoffman [email protected] wrote:
I think it is just missing an include:
lcmtypes.cmake:cmake_minimum_required(VERSION 2.8.3) # for the
CMakeParseArguments macro
mex.cmake:include(CMakeParseArguments)

The newest version of CMake has that as a built in function. So, all we need to
do is add include(CMakeParseArguments)
to lcmtypes.cmake for a fix.


Reply to this email directly or view it on GitHub
[https://github.com//pull/1843#issuecomment-195137656] .[https://github.com/notifications/beacon/AGJNNHni2xuwzR3_Ti5n3McOX06FZEMuks5psMj5gaJpZM4HtpBC.gif]

@billhoffman
Copy link
Contributor

@jhoare I put in a PR to your version:
jhoare/bot_core_lcmtypes#1

Can you merge? If not, I guess we could update this patch to pull from my fork of bot_core_lcmtypes.

@RussTedrake
Copy link
Contributor Author

I will merge this directly into open-humanoids

@RussTedrake
Copy link
Contributor Author

the drake005 error is known and unrelated (now #1848).
i'm convinced that the build servers are finally happy and healthy again with this.

RussTedrake added a commit that referenced this pull request Mar 11, 2016
@RussTedrake RussTedrake merged commit 2387950 into RobotLocomotion:master Mar 11, 2016
@RussTedrake RussTedrake deleted the bot_core branch November 8, 2016 01:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bot_core_lcmtypes failes to find lcm and lcm-java on windows
5 participants