-
Notifications
You must be signed in to change notification settings - Fork 50
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
module libraries and libcore built as shared libs #191
Conversation
This last commit, assuming travis agrees with my tests, should have things in a mergeable state. It could use further testing, installed vs non for example, and a thorough review, but everything that should be linking via shared library is now I think. The odd bit that remains is dealing with modules that need more of the API than is exposed by flux-core. There isn't a lot of that, but the kvs for example depends on several symbols from libutil. |
libcore is now built in src/common, the header has also moved libmrpc is now built in src/modules/libmrpc, it can not be built anywhere else, at all. It must be built between the kvs and the modctl modules. How precisely we avoided this before... all I can think is that the modctl lib must have been assuming the symbol would be available at runtime, and it was, so it worked. Now that it's building shared only, it checks, and if libmrpc is built in any other order, it fails. This seems almost ready for merge, pending review, except for one problem: PROBLEM: flux keygen fails make check passes all tests, but fails on keygen with either: flux-keygen: mkdir /g/g12/scogland/projects/flux/build/src/cmd/.libs/../../etc/flux: No such file or directory or: flux: `keygen' is not a flux command. See 'flux --help' The first happens when an install path is set with configure, the second when it isn't.
Both the kvs and core libraries are now built with flags that require all of their symbols to resolve at link time for them rather than load time. This should probably be a requirement on every shared library we build, if only for debugging purposes. The other main change is that the flux command now detects when it is being invoked by a libtool run script, by checking the path for containing a .libs component, and adds an extra '../' in that case. This was the cause of the keygen error from the previous commit.
…g up on travis that are being missed on hype
This looks like a really great direction to me. Not sure why tests are failing in Travis... If you want I can help look into that tomorrow. Also, I might have missed this but are |
All module libraries now require all symbols at link time, or they will immediately error out and tell you what's missing. KAP now builds correctly under check again, the new flux_event_send and recv prototypes broke it somehow, despite passing their own distcheck.
I found out what some of the issues are, they just keep cascading into On 18 May 2015, at 15:53, Mark Grondona wrote:
|
@trws, sorry about breaking KAP! It builds only if |
@garlick No worries, it was a simple enough fix at least. Getting everything to link on the travis machine is proving not to be however. Do you happen to know what gcc version we're running on there? Something is different in the way symbols are getting resolved on hype and on the build bot. All of the errors I'm getting are legitimate, but having it only fail on travis makes the debug loop pretty long. |
It's displayed early in the build log:
|
If you can get a Ubuntu vm going (virtualbox on your mac?), you may be able to work in an environment closer to the travis one. See |
There must be a better way to propagate these in automake. CMake does this automatically.
…ransitive dependencies for this to be practical
I used a Dockerfile to build the original This Dockerfile might need some slight tweaking as it was from awhile ago:
|
Oops, the above Dockerfile isn't going to work -- it was used for testing with lua-5.2. The travis CI environment is actually ubuntu 12.04LTS. Let me try to dig up the right Dockerfile this time. |
At least it's encouraging that this last version works correctly with clang. The only holdout for the gcc build seems to be missing symbols in the lua bindings, so hopefully this will be the last round anyway. The only thing I'm wondering is if it might be worth turning all of this into a flux-internal shared library, so it will pull in the symbols it needs at linking rather than manually propagating them everywhere. Or maybe there's a better way to do this in Automake, I don't know of one, but CMake for example will track the library dependencies of a static library and apply them where that library is used. Do either of you know a good way to do that across directories for Automake? |
Libtool does support some sort of inter-library dependencies. I thought automake was smart enough to do this if you linked a static convenience library with another? https://sourceware.org/autobook/autobook/autobook_77.html#SEC77 |
As Mark pointed out, using the libtool setup will track the necessary extra link options, so I have switched to that. Some of the builds now specify significantly more than they need to work, but it seems to have taken care of the issue.
This attempts to fix the parallel make check, but does not entirely succeed. Even with the .NOTPARALLEL: on the bottom dirs, automake still manages to try and build the KVS module before common is done building.
…ame failures as travis on hype
@trws, not sure if this is helpful but here's a
|
Nicely done, looks like this PR is passing now. |
Yes looks good! |
module libraries and libcore built as shared libs
Thanks for the review, and the dockerfile, may look at getting boot2docker working... anyway, I have one more update for this that splits the exports such that the flux_* symbols only exist in the libflux-core library, should help make sure that at least the API symbols are only included in the binaries once. I'll put in another pull request for that momentarily. |
This is a partial fix for #179, dependency resolution should be fixed up more reliably in the build system, and installation paths need to be dealt with for the shared libraries. Some more testing is also in order.
What this does include is that the libflux-core library no longer includes any module libraries, and is itself built as a shared library that the modules link in.