gfortran: 4.7. And now builds from source.

[samueljohn]

A modification of the gcc formula that builds gcc and rips of
everything except gfortran.

We had a small discussion on the homebrew mailing list.
Here is the first version of a gfortran formula that does not suck.
By "suck" I mean download binaries of gcc 4.2 from research.att.com, un-pax, and remove all non-fortran parts.

This formula is based on the gcc formula and removes binaries other than gfortran (you cannot really build gfortran stand-alone). Because long build is loooooooong, we should bottle it.

Please let me know if this works for you. I will do some more tests the next days.

[Sharpie]

Apple's GCC 4.2 is getting long in the tooth and has to disappear sometime. There are only two reservations I have about upgrading GFortran:

1. Non-Apple builds can't generate universal binaries like every other compiler we use.
2. The binaries from r.research.att.com are thrown through one hell of a regression test every day by running through builds of the R package ecosystem.

I don't think point 1 is a practical issue at this time since I am not aware of any Fortran-based formulae that use the --universal flag. However, considering point 2, it would be nice if we had some regression test that helped ensure that we're not screwing the GCC build up in some subtle way.

[samueljohn]

@Sharpie as you, I don't care for the first point. But the second is valid. Not sure though what to do. Gcc comes with it's own regression tests (but not as advanced as building R).
I run the numpy and the scipy test suite. Some tests cover gfortran usage, too. No problems so far.

[MikeMcQuaid]

The GCC regression tests seem good enough for me. Also don't care about universal.

[Sharpie]

I think the real solution to this problem is that we need to finish a CI system for Homebrew that checks packages and does the grunt-work of reviewing pull requests. Running a make test on R and Octave across three flavors of OS X would be a pretty good test of scientific coverage.

I'm for going forward with this as the current Fortran setup makes it trivially easy to roll back to the Apple binaries if something goes horribly wrong.

[MikeMcQuaid]

I've finished a CI system for Homebrew. We don't have any hardware to run it on.

[MikeMcQuaid]

If you make the test method run a test suite then that'll Just Work when the CI gets hardware.

[samueljohn]

@MikeMcQuaid a CI for Homebrew, dude? A dream!
Good you mention that here ... I had in mind to start one myself!

[MikeMcQuaid]

brew test-bot does most of the heavy lifting. Jenkins does the other half. We've had offers of hardware but we really need our own dedicated stuff (but we're working on it).

[staticfloat]

@samueljohn: My installation fails, looks like something's wrong with the install step.

Also, sorry for the window-wrapped output. I was too impatient to recompile with my window sized larger.

[samueljohn]

I think I know the cause. Looks like I do need to pass --enable-plugins (or similar) since install claims a file in plugins could not be found.

[manphiz]

Also, how to handle conflicts with homebrew/dupes/gcc (specifically in #{share})?

EDIT: I have a pull request for homebrew/dupes/gcc at https://github.com/Homebrew/homebrew-dupes/pull/116, as well as some thoughts about solving the conflicts.

[samueljohn]

using opt_prefix now.

[samueljohn]

@staticfloat should build now. Sorry about that. Was a last-minute-fuck-up.

[adamv]

I would be in favor of a Homebrew-gcc or Homebrew-compilers.

[manphiz]

An idea to solve conflicts between multiple GCC installations: manphiz/homebrew-dupes@bd4d0fc

[staticfloat]

@samueljohn; re-tested, compiles and runs OpenBLAS, arpack-ng, Julia, etc.... Seems like a winner to me! Now it just needs to be bottled. :) 👍

[samueljohn]

@adamv depending on Apple's future plans with llvm-gcc, we might want the gcc formula back in homebrew core. For gfortran, I want an executable without version-suffix, so let's have this separate from gcc now.

@manphiz had some good thoughts about the co-existance of gfortran and gcc (given gcc is still version-suffixed).

@staticfloat thanks for testing!

I will continue my tests of this gfortran build for a bit and see how the coexistence with gcc can be assured.

[MikeMcQuaid]

@samueljohn Clang is the future on OSX; I'd rather avoid GCC in core purely to push that.

[samueljohn]

@MikeMcQuaid alright, then lets continue down the road with this separate gfortran formula.

[samueljohn]

@Sharpie there are tests available via make check-fortran (additionally to the checks during the build-stages). However, they need 3 additional libs (all of them are in homebrew-dupes, which is good) and they claim that it is "normal" that some tests are failing (which is to be said to occur quite often). I am not experienced enough with gcc or gfortran to judge.

[Sharpie]

@MikeMcQuaid

Unfortunately, there is no future in Clang for Fortran at the moment. Clang helps with C, C++ and Obj-C only.

Ahh, NVM. I see what you are referring too. I agree, adding a full GCC to the main repo is a bigger can of worms than we want at the moment.

[MikeMcQuaid]

@Sharpie Cool, glad we're agreed. I'm actively for getting gfortran compiling, bottled and in.

[MikeMcQuaid]

@samueljohn This good to bottle?

[samueljohn]

Not yet. Please give me another day for testing.

[MikeMcQuaid]

No rush dude. Ping me when it is.

[samueljohn]

Don't know if that is an issue; there is a little warning:

Comparing stages 2 and 3
warning: gcc/cc1-checksum.o differs
Comparison successful.

Then I added the option (--check) to run make check-fortran.

As I understand gcc, it's quite normal that not all pass and one should check with their website/ML.
I see a warning sometimes WARNING: Couldn't find the global config file.. But it looks pretty nice!

Here is the important part of the long log:

        === gfortran Summary ===

# of expected passes        5792
# of expected failures      6
/private/tmp/gfortran-NfT7/gcc-4.7.2/build/gcc/testsuite/gfortran5/../../gfortran  version 4.7.2 (GCC) 


        === gfortran Summary ===

# of expected passes        5770
# of expected failures      2
# of unsupported tests      8
/private/tmp/gfortran-NfT7/gcc-4.7.2/build/gcc/testsuite/gfortran4/../../gfortran  version 4.7.2 (GCC) 


        === gfortran Summary ===

# of expected passes        5695
# of unsupported tests      19
/private/tmp/gfortran-NfT7/gcc-4.7.2/build/gcc/testsuite/gfortran3/../../gfortran  version 4.7.2 (GCC) 


        === gfortran Summary ===

# of expected passes        5231
# of expected failures      3
# of unsupported tests      25
/private/tmp/gfortran-NfT7/gcc-4.7.2/build/gcc/testsuite/gfortran/../../gfortran  version 4.7.2 (GCC) 


        === gfortran Summary ===

# of expected passes        5903
# of expected failures      25
# of unsupported tests      1
/private/tmp/gfortran-NfT7/gcc-4.7.2/build/gcc/testsuite/gfortran2/../../gfortran  version 4.7.2 (GCC) 


        === gfortran Summary ===

# of expected passes        5865
# of expected failures      4
/private/tmp/gfortran-NfT7/gcc-4.7.2/build/gcc/testsuite/gfortran1/../../gfortran  version 4.7.2 (GCC) 


        === gfortran Summary ===

# of expected passes        6393
# of expected failures      2
# of unsupported tests      16
/private/tmp/gfortran-NfT7/gcc-4.7.2/build/gcc/testsuite/gfortran6/../../gfortran  version 4.7.2 (GCC) 

(On 10.8 with Xcode-only)

As I understand, it's a very good sign that there are no unexpected failures or even errors.

[samueljohn]

So only thing to test now is, if the make bootstrap is actually needed or done anyway?
I commented it out, but it still did a 3 stage build (if that is what bootstrap means).

[samueljohn]

Ah, well, and of course the conflicts with the gcc formula have to be investigated a bit further. I think they are only there if brew install gcc --enable-fortran was done.

[samueljohn]

My gcc was build with --enable-fortran but does not conflict with this gfortran formula any more. Perhaps --enable-nls will add some locale files.

[samueljohn]

Yep, if I brew install gcc ---enable-nls, I get conflicts, so perhaps I just force to disable the translations of the messages for gfortran. Programming in any other language than English is insane.

brew link gcc
Linking /homebrew/Cellar/gcc/4.7.2... Warning: Could not link gcc. Unlinking...

Error: Could not symlink file: /homebrew/Cellar/gcc/4.7.2/share/locale/zh_TW/LC_MESSAGES/gcc.mo
Target /homebrew/share/locale/zh_TW/LC_MESSAGES/gcc.mo already exists. You may need to delete it.
To force the link and delete this file, do:
  brew link --overwrite formula_name

[samueljohn]

There is an old issue open #8539 and I wonder what to do about that. In short, that is about how to use another compiler (clang, llvm-gcc) and link to libgfortran with -lgfortran. This is no problem when just compiling fortran sources, since gfortran knows where to find the stuff.

In theory we could install the libgfortran* into HOMEBREW_PRETIX/lib but I think we should not do that because of all the other files like libgcc_ext.10.4.dylib.

[samueljohn]

There are now left only four "todo"s:

1. Do we prefer #{HOMEBREW_PREFIX} or opt_prefix for the deps that are not keg_only. @Sharpie @MikeMcQuaid?
2. Add a note to the caveat how to link -lgfortran when using gcc, llvm-gcc or even clang ( Library libgfortran not found after installing gfortran #8539 )
3. Do another test compile because I commented in the make bootstrap again. But I am running on battery right now.
4. Does the --disable-nls avoid the locale files (and the conflict with brew install gcc --enable-nls)

[MikeMcQuaid]

I tend to use opt_prefix more now,

[samueljohn]

1. Done. I tend to favor opt_prefix, too.
2. I'd suggest to use -L and link statically against libgfortran.
3. Done that. Works.
4. Done that. Works.

@MikeMcQuaid Unless you want a more specific caveat for 2.) this is ready to pull and test and bottle, I'd say.

[MikeMcQuaid]

Add the caveat if people get confused. Looks good. Should I build the bottles with that optimised bootstrap flag?

[samueljohn]

@MikeMcQuaid Okay, we defer the caveat for now. I would need to play around with that anyway.
For a bottle, I'd refrain from a profiledbootstrap. As I understand it, it is similar to what ATLAS does. It might adapt to the number of cores and cache sizes you have in your brewery. But this is more or less my guess from the name.

But I tested the profiledbootstrap on my machine (10.8, sandybridge, Xcode-only) and with --check. Works like charm.

[Sharpie]

I'd suggest to use -L and link statically against libgfortran.

Like I said in #8539, this is a difficult problem to advise on since there can be multiple sub-folders in lib/gcc corresponding to different architectures. Setting LIBRARY_PATH to the root lib/gcc folder solves the problem for GCC compilers, but may not work for Clang.

[MikeMcQuaid]

Pushed with bottle.

[samueljohn]

Thanks.

Also thanks for the explanation why -L does not work (good) here.

[staticfloat]

Is there a way to force brew to use a bottle? It seems to always be downloading the source and compiling instead of downloading the bottle.

[2bits]

It should not build from source unless you throw some command line option. A normal brew install gfortran should install the bottle on all systems (it did for me on 10.6), though it has to build from source a couple of deps.

[staticfloat]

$ cd $(brew --prefix)
$ git pull origin master
From https://github.com/mxcl/homebrew
 * branch            master     -> FETCH_HEAD
Already up-to-date.
$ brew install gfortran
==> Downloading http://ftpmirror.gnu.org/gcc/gcc-4.7.2/gcc-4.7.2.tar.bz2
^C

If I let it continue, it goes through the entire compilation process.

I don't have the environment variables defined that would screw this up either:

$ env | grep HOMEBREW_BUILD_FROM_SOURCE
$

[jacknagel]

There are other things that disable bottles, for example a non-standard HOMEBREW_PREFIX.

[staticfloat]

Yes, that makes perfect sense, and is indeed the case here. Thanks. I've updated the wiki page to reflect this.