numpy: make numpy find openblas from Homebrew

[mwageringel]

For numpy to find Homebrew's openblas on macOS, this ticket adds an [openblas] section to numpy's site.cfg if Sage is using openblas. This is also what Homebrew's numpy formula does. See also the upstream site.cfg.example.

Before this ticket, numpy would use the macOS Accelerate framework instead of openblas, as can be seen from the logs:

  FOUND:
    extra_compile_args = ['-msse3', '-I/System/Library/Frameworks/vecLib.framework/Headers']
    extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
    define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)]

After:

  FOUND:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/usr/local/Cellar/openblas/0.3.7/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]

This problem was previously discussed on sage-devel.

Depends on #29051
Depends on #29071
Depends on #29084

CC: @dimpase @mkoeppe

Component: packages: standard

Keywords: numpy, scipy

Author: Markus Wageringel

Branch/Commit: eed1d19

Reviewer: Matthias Koeppe

Issue created by migration from https://trac.sagemath.org/ticket/29056

[mwageringel]

Commit: b840c8c

[mwageringel]

Author: Markus Wageringel

[mwageringel]

comment:1

In order to test this, one currently needs to force-link Homebrew's readline to /usr/local (#29000) and configure Sage with

./configure LDFLAGS="-L/usr/local/opt/openblas/lib" CPPFLAGS="-I/usr/local/opt/openblas/include" PKG_CONFIG_PATH="/usr/local/opt/openblas/lib/pkgconfig"

as openblas is keg-only.

---

New commits:

b840c8c

29056: make numpy find openblas from Homebrew

[mwageringel]

Branch: u/gh-mwageringel/29056

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

4d549f2

29056: fix block syntax

[sagetrac-git]

Changed commit from b840c8c to 4d549f2

[dimpase]

comment:3

won't you be getting KeyError in the case of pkg-config not available (and so pkgconfig creating empty dicts)?

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

052c06c

29056: make numpy find openblas from Homebrew

[sagetrac-git]

Changed commit from 4d549f2 to 052c06c

[mwageringel]

comment:5

Replying to @dimpase:

won't you be getting KeyError in the case of pkg-config not available (and so pkgconfig creating empty dicts)?

Is pkg-config not always present, possibly by installing the pkgconf spkg? In any case, I changed the relevant line to account for the possibility of an empty dictionary:

-    if 'openblas' in pc_blas['libraries']:
+    if 'openblas' in pc_blas.get('libraries', []):

I have tested this on 9.1.beta1 both using openblas from Homebrew and from Sage and in each case both numpy and scipy find the correct openblas.

[dimpase]

comment:6

could this be made dependent on #29051 ?

[mwageringel]

comment:7

Of course. Let us postpone this then.

[mwageringel]

Dependencies: #29051

[dimpase]

comment:8

perhaps MacOS with pkg-config and openblas may be handled the same way as the "generic" systems by this script.

[mwageringel]

comment:9

Replying to @dimpase:

perhaps MacOS with pkg-config and openblas may be handled the same way as the "generic" systems by this script.

You mean by removing the Darwin if-case? I tried that first, but for some reason it did not work, possibly because Numpy has the preference: OpenBLAS > Accelerate > BLAS/LAPACK.

This preference can be changed using the variables NPY_BLAS_ORDER/NPY_LAPACK_ORDER, but setting these in Numpy's spkg-install did not seem to help either.

[mkoeppe]

comment:10

I would suggest to also add #29071, #29084 as prereqs

[mkoeppe]

Changed branch from u/gh-mwageringel/29056 to u/mkoeppe/29056

[mkoeppe]

Reviewer: Matthias Koeppe

[mkoeppe]

Changed commit from 052c06c to eed1d19

[mkoeppe]

comment:12

I've merged in the tickets mentioned above and tested using tox -e local-homebrew-macos-standard from #29104.

---

New commits:

06f46eb	Add blas to fflas_ffpack linked libraries so that openblas is picked up on Arch
eed1d19	Merge branch 'u/arojas/make_fflas_ffpack_detect_and_use_system_openblas_on_arch' of git://trac.sagemath.org/sage into t/29056/29056

[mkoeppe]

Changed dependencies from #29051 to #29051, #29071, #29084

[mwageringel]

comment:14

Thank you.

[vbraun]

Changed branch from u/mkoeppe/29056 to eed1d19