lighttpd, fftw, arpack-ng cannot be linked. Homebrew has it already.

[samueljohn]

Looks like lighttpd is built by julia, right? It's a dupe to homebrew's and that can make the brew link to fail. See below:

Linking /usr/local/Cellar/julia/HEAD... 
Error: Could not symlink file: /usr/local/Cellar/julia/HEAD/sbin/lighttpd
Target /usr/local/sbin/lighttpd already exists. You may need to delete it.
To force the link and delete this file, do:
  brew link -f formula_name

To list all files that would be deleted:
  brew link -n formula_name

[staticfloat]

This shouldn't be an issue, as USE_SYSTEM_LIGHTTPD being defined (Check out lines 53-55 in my formula) should stop it from compiling, much less installing lighttpd into <prefix>/sbin. This is what happens on my computer.

Are you using my formula or has yours been modified? All of the USE_SYSTEM_<package name> defines should stop these dependencies from being compiled and installed on the julia end.

[samueljohn]

Yep, I'll check that. Perhaps it's my changes to your formula.

[samueljohn]

I have

make PREFIX=/usr/local/Cellar/julia/HEAD USECLANG=1 USE_SYSTEM_READLINE=1 USE_SYSTEM_GLPK=1 USE_SYSTEM_GMP=1 USE_SYSTEM_LLVM=1 USE_SYSTEM_PCRE=1 USE_SYSTEM_LIGHTTPD=1 USE_SYSTEM_FFTW=1 USE_SYSTEM_LAPACK=1 USE_SYSTEM_BLAS=1 USE_SYSTEM_SUITESPARSE=1 USE_SYSTEM_ARPACK=1

in my build log

You could perhaps try to brew install lighttpd and 'fftw' and 'arpack-ng' before you brew install julia and see what you get?

[samueljohn]

I changed the title to reflect that this issue is about all three conflicts I have found, because you closed the others as dupes. Sorry for kinda "spamming" your issues here :-)

Because I use the the julia formula from #2, I am not 100% sure if you can reproduce but to me it looks like you would get the same conflicts, too.

Could it be the case that julia copies all the needed libs into it's lib directory?
Perhaps those should be removed before the brew sym-link step?

[staticfloat]

My julia formula depends on lighttpd, fftw and arpack-ng (In fact, it depends on a specialized version of arpack-ng that I maintain in my Homebrew Tap) and thus Homebrew always installs them before installing Julia.

I installed your formula on my Macbook Pro (via brew uninstall julia; brew install -vd --HEAD https://github.com/samueljohn/homebrew-julia/raw/master/julia.rb) and it installed just fine. No symlinking problems at all. Try doing a brew prune and brew cleanup after brew uninstall julia; it's possible you have some cruft left over from previous installs?

[samueljohn]

Thanks for your help, I'll check that and report back.
My impression (which I'll double check) was that Julia somehow copied
(?) the needed 3rd party libs into its own local lib dir.

[staticfloat]

Julia does indeed copy 3rd party libraries into its own local lib dir, (Check out lines 38-43 in my formula for where I manually symlink libfftw and libarpack (a suspicious coincidence that those are the two that you're having trouble with) into <julia-build-dir>/usr/lib) This needs to happen in order for julia to find these libraries at runtime, due to how Julia is linked together. (It's possible we can fix this, but I need more experience with dynamic loading, especially on OSX. If you can shed light on better ways to get Julia to find dynamic libraries in user-defined directories, I'm all ears!)

[samueljohn]

Normally the LDFLAGS are used during linking to find the necessary libs. Perhaps we have to check if julia uses the LDFLAGS in all necessary places in its Makefiles.

[staticfloat]

I'm not talking about compile-time finding of libraries, I'm talking about runtime finding of libraries. Dynamic libraries, as opposed to static ones.

Did you run the cleanup and prune after removing julia? That's the next thing I can think of......

[samueljohn]

I updated to ML and gfortran (the bottle) is not ready for ML, so I can't install julia right now :-(

To the library thing:
I am not an expert, but the linker should put the rpath into the lib such the the necessary libs can be found to run time.

[samueljohn]

Sorry, gfortran has been updated. I was in the wrong (old) branch.

I'll continue testing tomorrow.

[staticfloat]

No rush. Enjoy life. :)
-E

On Wed, Jul 25, 2012 at 3:11 PM, Samuel John <
[email protected]

wrote:

Sorry, gfortran has been updated. I was in the wrong (old) branch.

I'll continue testing tomorrow.

---

Reply to this email directly or view it on GitHub:
#3 (comment)

[staticfloat]

I expected the same (no problems, due to absolute path linking in the rpath), but I ran into problems, because the rpath was relative...... I think. It may be worthwhile messing with the formula to see if we don't need to do that!

And thanks for the update about gfortran; I'm about to upgrade to ML myself. :P

[staticfloat]

It turns out that I was the one with the screwed-up system, at least in the case of lighttpd. Although I had lighttpd installed, the symlink in my <homebrew prefix>/sbin folder was missing, so Julia was free to clobber it. I believe when Julia changed its file structure (adding the usr/ directories, etc....) this slipped through the cracks somehow. In any case, I've removed the symlinking of lighttpd, as it's no longer needed, and updated my formula online.

I seem to be getting errors using the web REPL however, as the julia process backing the session seems to exit randomly. I'll take a further look soon.

[samueljohn]

Good to hear, you are making some progress in this linking issues!

[staticfloat]

@samueljohn; I've pushed some changes which should solve everything EXCEPT fftw. I'm having build errors, and I've narrowed it down to when I include USE_SYSTEM_FFTW=1. Can you try running the latest formula I've uploaded, see if it works (brew test julia) and then try adding 'FFTW' back into the list of system dependencies on line 53 and see if it still works then? I'm looking for confirmation that fftw is indeed the bug that causes the:

error during init:
could not show value of type LoadError

[staticfloat]

@samueljohn; Is it possible for you to try this out?

[samueljohn]

Yes, sorry I have quite a backlog of things to finish first. I hope I can get to my "installing julia"-hobby this evening CET.

[samueljohn]

I failed to do this. So sorry. So busy.
Hope is I can do it tomorrow.

[staticfloat]

No problem. :D
-E

On Wed, Aug 22, 2012 at 2:03 PM, Samuel John [email protected]:

I failed to do this. So sorry. So busy.
Hope is I can do it tomorrow.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/3#issuecomment-7949332.

[samueljohn]

This is suspicious and pops up during compilation:

    PERL extras/glpk_h.jl
i686-apple-darwin11-llvm-gcc-4.2: /Users/sabae/.homebrew/include/glpk.h: No such file or directory
i686-apple-darwin11-llvm-gcc-4.2: warning: '-x c' after last input file has no effect
i686-apple-darwin11-llvm-gcc-4.2: no input files
    PERL extras/julia_message_types_h.jl

[staticfloat]

Whoopsies, you're absolutely right, I hardcoded a path in there when I really meant to use HOMEBREW_PREFIX.

Should be fixed now.

[samueljohn]

Thanks.

llvm is at 3.1 in homebrew/master now.

[samueljohn]

Also:

==> Finishing up
ln -s ../Cellar/julia/HEAD/etc/lighttpd.conf lighttpd.conf
ln -s ../Cellar/julia/HEAD/bin/launch-julia-webserver launch-julia-webserver
ln -s ../Cellar/julia/HEAD/bin/julia-release-webserver julia-release-webserver
ln -s ../Cellar/julia/HEAD/bin/julia-release-readline julia-release-readline
ln -s ../Cellar/julia/HEAD/bin/julia-release-basic julia-release-basic
ln -s ../Cellar/julia/HEAD/bin/julia julia
ln -s ../Cellar/julia/HEAD/lib/libumfpack.dylib libumfpack.dylib
ln -s ../Cellar/julia/HEAD/lib/libsuitesparse_wrapper.dylib libsuitesparse_wrapper.dylib
ln -s ../Cellar/julia/HEAD/lib/libRmath.dylib libRmath.dylib
ln -s ../Cellar/julia/HEAD/lib/librandom.dylib librandom.dylib
ln -s ../Cellar/julia/HEAD/lib/libjulia-release.dylib libjulia-release.dylib
ln -s ../Cellar/julia/HEAD/lib/libgrisu.dylib libgrisu.dylib
ln -s ../Cellar/julia/HEAD/lib/libgmp_wrapper.dylib libgmp_wrapper.dylib
ln -s ../Cellar/julia/HEAD/lib/libglpk_wrapper.dylib libglpk_wrapper.dylib
Skipping; already exists: /usr/local/lib/libglpk.dylib
ln -s ../Cellar/julia/HEAD/lib/libfftw3f_threads.dylib libfftw3f_threads.dylib
Error: The linking step did not complete successfully
The formula built, but is not symlinked into /usr/local
You can try again using `brew link julia'
==> Summary

Looks like julia still tries to overwrite glpk which may perhaps break octave (which also needs this lib)

[samueljohn]

    JULIA usr/lib/julia/sys.ji
Warning: replacing module Base

Is this a problem?

[samueljohn]

In /usr/local/Cellar/julia/HEAD/lib are some duplicates:

• libfftw3* already installed by fftw
• libarpack.dylib already installed by arpack-ng

I added

    # Remove some libs that are already at HOMEBREW_PREFIX/lib
    rm Dir["#{lib}/libfftw*"]
    rm lib/"libarpack.dylib"

to my local version.

[StefanKarpinski]

Warning: replacing module Base

This is normal. In the bootstrapping process, Julia uses one Base module to replace another one.

[samueljohn]

@StefanKarpinski thanks!

[staticfloat]

@samueljohn, Awesome! It works with FFTW being compiled by Julia then. (Because I am not using the system FFTW in this formula right now, it makes its own FFTW libraries which is why you're getting the FFTW linking problems.) Now, my real question is can you duplicate the errors above by adding 'FFTW' into the list of system dependencies on line 53. The buildtime should be drastically reduced as Julia no longer needs to build FFTW, and if this isn't just a problem on my machine, you should get the error:

error during init:
could not show value of type LoadError

[samueljohn]

@staticfloat I actually was annoyed julia compiled it's own fftw (don't get me wrong: nice that it can do it) so I added the USE_SYSTEM_FFTW=1 with success. brew builds fftw and that one seems ok for julia!

You should, perhaps remove old installs of llvm, fftw and so on to get a fresh start.

[staticfloat]

Darn. It must be something on my machine then. (Just to double-check, you
are on 10.8, right?) Time to get rid of everything and redo it all from
scratch. Compile away, my brave laptop!
Additionally, I believe the glpk and arpack issues are artifacts from an
old workaround that I no longer need because of the rpath patch I have
included at the bottom of the formula. So you shouldn't have any linking
problems at all now and should no longer need to remove the libraries at
the end.

[samueljohn]

I am on 10.8.
Also, I commented out the symlinking of libglpk and arpack-ng.

[StefanKarpinski]

Not sure if it is relevant at all, but we build FFTW twice: once for single precision and once for double precision. It's weird, but that's the way the library works.

[staticfloat]

Yes, homebrew builds it for single, double and long double in the same way.
On Aug 23, 2012 12:53 PM, "Stefan Karpinski" [email protected]
wrote:

Not sure if it is relevant at all, but we build FFTW twice: once for
single precision and once for double precision. It's weird, but that's the
way the library works.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/3#issuecomment-7981914.

[StefanKarpinski]

Ok, sorry for the noise! Carry on...

[samueljohn]

This issue here would be fixed by #6.

[ViralBShah]

I wouldn't stress too much about lighttpd. We are planning on getting rid of it and replacing with nginx.

[staticfloat]

I'm going to close this issue as linking problems shouldn't bother us anymore with proper compilation, and open a new issue for my FFTW woes.

[samueljohn]

Yes can be closed.