julia: Fix install. #6

samueljohn · 2012-08-23T21:53:42Z

julia can build in parallel.
Copy doc and examples to share/julia.
CPPFLAGS/CFLAGS changing no longer needed.
Use FC is set. (needed to support gfortran-4.7)
Copy the "test" into lib/julia (is missing from install)
julia.rb should not be executable (unix rights)

samueljohn · 2012-08-23T21:54:55Z

@staticfloat this is just to let you know my current (working) version of julia.rb.
All tests pass.

However julia-release-webserver shows me an empty page. I am not sure how to fix that.

StefanKarpinski · 2012-08-24T05:42:31Z

On Darwin it already uses BLAS and LAPACK, no need to explicitly USESYSTEM...

This is going to change back as soon as the next version of OpenBLAS is released... which, in fact already happened. So I'm going to switch back to using OpenBLAS on Darwin nowish...

samueljohn · 2012-08-24T08:28:46Z

@staticfloat then we should begin to check if Homebrew's keg-only OpenBLAS can be made to work with Julia.

@StefanKarpinski is OpenBLAS faster or has it got more features? Thanks for the hint!

samueljohn · 2012-08-24T08:44:38Z

@StefanKarpinski, right now, Homebrew's OpenBlas formula builds without LAPACK system "make", "NOLAPACK=1", is that okay for Julia?
Also, suite-sparse links against Apple's Accelerate framework. I hope that will not be a problem.

StefanKarpinski · 2012-08-24T13:57:13Z

I believe that OpenBLAS recently surpassed the system BLAS and LAPACK in performance, so yes, these days it is. I also think it may have more features. It certainly seems to have all the features we've needed so far. I'm pretty sure we use the LAPACK part too, so you may want to enable that somehow, maybe optionally? Although, if someone is using OpenBLAS for BLAS, it's hard to imagine that they wouldn't also want LAPACK features based on OpenBLAS. I think that having SuiteSparse link against a different BLAS is ok but it will bloat the resisting binaries that link against both. @ViralBShah has done a lot of work trying to reduce this bloat in Julia, but it's a slog.

ViralBShah · 2012-08-24T15:04:19Z

Openblas was about 5-10% faster than Accelerate when I had tested (Use peakflops() in julia). It is also open source, maintained, and cross-platform, and in the julia team, we will ensure that everything works well with openblas and its patched LAPACK. Accelerate will most likely not include the latest release of LAPACK. Apart from that, I don't foresee any real issue with using Accelerate instead of openblas.

Do build openblas with LAPACK, as it patches some of the key LAPACK functions and results in higher performance than standalone LAPACK. This results in libopenblas, which includes both, BLAS and LAPACK.

It is ok if suitesparse uses Accelerate - but it is preferable to use the same set of libraries everywhere.

staticfloat · 2012-08-24T17:03:13Z

julia.rb

-    cp_r "test", "#{lib}/julia/"
+
+    # link libglpk.0.dylib is installed but not libglpk. Todo: Fix the glpk formula!
+    ln_s "#{Formula.factory('glpk').lib}/libglpk.dylib", "usr/lib"


What is broken about the glpk formula? I get a libglpk.dylib and libglpk.0.dylib installed in my HOMEBREW_PREFIX/lib.

staticfloat · 2012-08-24T17:07:56Z

I like copying doc, etc... but I'm not sure if skipping Julia's install procedure entirely is such a good idea. Is this because you don't want libraries from Julia conflicting with Homebrew's libraries? (Because Julia now throws all of its libraries into <prefix>/lib?

samueljohn · 2012-08-24T17:42:14Z

wait... i repush (install working)

samueljohn · 2012-08-24T17:48:37Z

julia.rb

@@ -30,10 +33,6 @@ def patches

  def install
    ENV.fortran
-    ENV.deparallelize


really I never had problems with parallel builds. It builds blazing fast (on 4-core i7 here), so it makes a difference.
If you insist on it, why not adding if build.debug? (not tested if it is called that way, may be named similar)

I have never had problems with parallel builds either; It was just because when having troubles installed dependencies (like I did earlier on) it made it very difficult for other users to figure out what was going wrong. Having it as if build.debug? is a wonderful idea, thanks!

staticfloat · 2012-08-24T17:49:11Z

This all looks really good, I'm going to merge it with a few changes;

We don't need to depend_on metis
We now do need to depend on openblas, so I'm going to enable that.
Since you're asking for USE_SYSTEM_NGINX, I'm going to depend_on nginx as well.

samueljohn · 2012-08-24T17:50:11Z

julia.rb

@@ -45,11 +44,15 @@ def install
    # Build up list of build options
    build_opts = ["PREFIX=#{prefix}"]

+    # Tell julia about our gfortran 
+    # (this enables to use gfortran-4.7 from the tap homebrew-dupes/gcc.rb)
+    build_opts << "FC=#{ENV['FC']}"


This does not hurt if you use the usual brew install gfortran. I prefer the brew install gcc --enable-fortran in order to get the 4.7 version (which I need for numpy/scipy). That version rocks. But I need to set FC, since hombrew-dupes appends the "-4.7" and most software doesn't find gfortran then.

- julia can build in parallel. - Copy doc and examples to share/julia. - CPPFLAGS/CFLAGS changing no longer needed. - Use FC is set. (needed to support gfortran-4.7) - Copy the "test" into lib/julia (is missing from install) - julia.rb should not be executable (unix rights)

samueljohn · 2012-08-24T17:54:57Z

julia.rb

      build_opts << "USE_SYSTEM_#{dep}=1"
    end
-    
+


brew audit complains about trailing whitespace. Some git thing or so.

samueljohn · 2012-08-24T17:57:37Z

I commented out the openblas (but left it there to remember us when julia switches)
You may be right on metis (I dunno. If it is an indirect dependency it's okay. Homebrew will figure out)
nginx is not yet needed (I guess) but depending on it doesn't hurt.

samueljohn · 2012-08-24T17:58:58Z

So from now on, I won't git push --force any more and leave the merging up to you :-P

staticfloat · 2012-08-24T18:01:07Z

Cool, thanks for all your hard work! I'm hoping that either the changes
I'm doing now or using gfortran from gcc 4.7 will magically fix the errors
I'm having. If not, I'm going to have to make a mac virtual machine and
try it completely from scratch to compare between the two of us to see why
I'm getting that LoadError when compiling FFTW in Homebrew!
-E

On Fri, Aug 24, 2012 at 10:58 AM, Samuel John [email protected]:

So from now on, I won't git push --force any more and leave the merging
up to you :-P

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/6#issuecomment-8009942.

staticfloat · 2012-08-24T18:27:58Z

Note to self; next time just auto-merge and add in changes afterwards, instead of trying to fix up stuff on remote branch and then merging in. :P

samueljohn · 2012-08-24T18:28:56Z

Thanks for pulling!

When using gfortran-4.7, I needed to do the following:

export FC=`brew --prefix`/bin/gfortran-4.7
brew install julia --HEAD --default-fortran-flags -v

The fortranflags are important.

And for fftw, just try to brew rm and reinstall. Let me know ...

samueljohn · 2012-08-24T18:29:38Z

Oh, and don't forget to brew tap homebrew-dupes and brew install gcc --enable-fortran

samueljohn · 2012-08-24T18:30:22Z

Then, get a good ☕. Or two.

staticfloat · 2012-08-24T18:31:14Z

Right, I've uninstalled and reinstalled fftw countless times, along with
every other direct julia dependency. I'm hoping it's the fortran compiler
I have right now. We'll see, compiling.....

On Fri, Aug 24, 2012 at 11:29 AM, Samuel John [email protected]:

Oh, and don't forget to brew tap homebrew-dupes and brew install gcc
--enable-fortran

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/6#issuecomment-8010744.

samueljohn · 2012-08-24T18:38:30Z

I just built julia successfully from your master branch. I brew rm tbb metis and tbb were automagically reinstalled (some of the deps needed it). So you were right about that.
brew uses metis shows suite-sparse. You were right again.

If you still have difficulties in installing julia, just open a new issue here on your repro, and ping me. I nee the full log, what brew doctor says and the brew --config gisted.

StefanKarpinski · 2012-08-24T20:39:35Z

I commented out the openblas (but left it there to remember us when julia switches)

Already switched: JuliaLang/julia@926c073.

samueljohn · 2012-08-24T20:43:07Z

Damn fast, @StefanKarpinski! How can I tell for sure (speak: where to otool -L to see which lib was linked)?
I am going to test it right away (I assume when julia runtests.jl all succeeds, all is fine)

@ViralBShah Thanks for you explanation. I will use peakflops() to check the speeds :-)

StefanKarpinski · 2012-08-24T20:45:01Z

I did say "nowish" :-)

staticfloat · 2012-08-24T20:47:06Z

@samueljohn; I ran into a lot of issues when not including CFLAGS copied into CPPFLAGS. It looks like the makefiles in Julia don't have CPPFLAGS littered everywhere it should to support taking the ENV copying out, so I've put it back in.

Still compiling gcc...... :P

samueljohn · 2012-08-24T20:54:47Z

@staticfloat I need to see your complete log to say anything. Also, please brew update, since I changed one or two days ago the way CFLAGS and CPPFLAGS are set. I also need your brew --config and brew doctor output.

Surely we can fix it!

staticfloat · 2012-08-24T21:36:39Z

Here's my install log, without CPPFLAGS being copied, with the new gfortran and with --default-fortran-flags.

https://gist.github.com/3455903

samueljohn · 2012-08-24T22:10:12Z

@staticfloat Just to be sure brew rm readline followed by brew install readline does not help?
I'll look deeper into this tomorrow.

ViralBShah · 2012-08-25T02:08:11Z

I do otool -L on libarpack, libumfpack, libcholmod to check which BLAS got linked.

-viral

On 25-Aug-2012, at 2:13 AM, Samuel John wrote:

Damn fast, @StefanKarpinski! How can I tell for sure (speak: where to otool -L to see which lib was linked)?
I am going to test it right away (I assume when julia runtests.jl all succeeds, all is fine)

@ViralBShah Thanks for you explanation. I will use peakflops() to check the speeds :-)

—
Reply to this email directly or view it on GitHub.

samueljohn · 2012-08-25T09:39:10Z

Thanks. Then my julia build from 11h ago is still linked to accelerate. I guess thats what he meant with "nowish".

staticfloat · 2012-08-27T21:06:19Z

@samueljohn; I just remembered that you were having trouble with the web REPL; Instead of running julia-release-webserver, try running launch-julia-webserver. I was confused why there were seemingly two executables for this before, and it turns out launch-julia-webserver is a shell script around julia-release-webserver that sets up some environment it needs. Using launch-julia-webserver it works on my mbp.

samueljohn · 2012-08-28T10:58:46Z

@staticfloat genius! That worked :-)
It's a trap when you juli<TAB> and see the julia-release-webserver :-P

samueljohn mentioned this pull request Aug 23, 2012

lighttpd, fftw, arpack-ng cannot be linked. Homebrew has it already. #3

Closed

staticfloat reviewed Aug 24, 2012
View reviewed changes

samueljohn reviewed Aug 24, 2012
View reviewed changes

staticfloat added a commit that referenced this pull request Aug 24, 2012

Merge branch changes from @samueljohn from pull request #6

32f9f70

staticfloat closed this Aug 24, 2012

wjj2329 mentioned this pull request Oct 27, 2017

Update issue #244

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

julia: Fix install. #6

julia: Fix install. #6

samueljohn commented Aug 23, 2012

samueljohn commented Aug 23, 2012

StefanKarpinski commented Aug 24, 2012

samueljohn commented Aug 24, 2012

samueljohn commented Aug 24, 2012

StefanKarpinski commented Aug 24, 2012

ViralBShah commented Aug 24, 2012

staticfloat Aug 24, 2012

staticfloat commented Aug 24, 2012

samueljohn commented Aug 24, 2012

samueljohn Aug 24, 2012

staticfloat Aug 24, 2012

staticfloat commented Aug 24, 2012

samueljohn Aug 24, 2012

samueljohn Aug 24, 2012

samueljohn commented Aug 24, 2012

samueljohn commented Aug 24, 2012

staticfloat commented Aug 24, 2012

staticfloat commented Aug 24, 2012

samueljohn commented Aug 24, 2012

samueljohn commented Aug 24, 2012

samueljohn commented Aug 24, 2012

staticfloat commented Aug 24, 2012

samueljohn commented Aug 24, 2012

StefanKarpinski commented Aug 24, 2012

samueljohn commented Aug 24, 2012

StefanKarpinski commented Aug 24, 2012

staticfloat commented Aug 24, 2012

samueljohn commented Aug 24, 2012

staticfloat commented Aug 24, 2012

samueljohn commented Aug 24, 2012

ViralBShah commented Aug 25, 2012

samueljohn commented Aug 25, 2012

staticfloat commented Aug 27, 2012

samueljohn commented Aug 28, 2012

julia: Fix install. #6

julia: Fix install. #6

Conversation

samueljohn commented Aug 23, 2012

samueljohn commented Aug 23, 2012

StefanKarpinski commented Aug 24, 2012

samueljohn commented Aug 24, 2012

samueljohn commented Aug 24, 2012

StefanKarpinski commented Aug 24, 2012

ViralBShah commented Aug 24, 2012

staticfloat Aug 24, 2012

Choose a reason for hiding this comment

staticfloat commented Aug 24, 2012

samueljohn commented Aug 24, 2012

samueljohn Aug 24, 2012

Choose a reason for hiding this comment

staticfloat Aug 24, 2012

Choose a reason for hiding this comment

staticfloat commented Aug 24, 2012

samueljohn Aug 24, 2012

Choose a reason for hiding this comment

samueljohn Aug 24, 2012

Choose a reason for hiding this comment

samueljohn commented Aug 24, 2012

samueljohn commented Aug 24, 2012

staticfloat commented Aug 24, 2012

staticfloat commented Aug 24, 2012

samueljohn commented Aug 24, 2012

samueljohn commented Aug 24, 2012

samueljohn commented Aug 24, 2012

staticfloat commented Aug 24, 2012

samueljohn commented Aug 24, 2012

StefanKarpinski commented Aug 24, 2012

samueljohn commented Aug 24, 2012

StefanKarpinski commented Aug 24, 2012

staticfloat commented Aug 24, 2012

samueljohn commented Aug 24, 2012

staticfloat commented Aug 24, 2012

samueljohn commented Aug 24, 2012

ViralBShah commented Aug 25, 2012

samueljohn commented Aug 25, 2012

staticfloat commented Aug 27, 2012

samueljohn commented Aug 28, 2012