BUG: Eigenvectors generated via linalg.eigh are not orthonormal in v1.26.0

[ShinjanM]

Describe the issue:

Hi,

While computing the eigenvectors of Hermitian matrices using numpy.linalg.eigh the eigenvectors are not coming out to be orthonormal. The eigenvectors are orthonormal using the scipy.linalg.eigh routine.

A jupyter notebook with a simple example is attached.

Compare_diag.pdf

Reproduce the code example:

import numpy as np
import scipy as sp

def create_random_hermitian_mat(dim):
    a = np.random.random((dim,dim)) + 1j*np.random.random((dim,dim))
    b = np.conj(a).T + a
    return b

a = create_random_hermitian_mat(4)
u, v = np.linalg.eigh(a)
print("Numpy Results = \n", np.matmul(np.conj(v).T, v))
u_sp, v_sp = sp.linalg.eigh(a)
print("\n Scipy results = \n", np.matmul(np.conj(v_sp).T, v_sp))

Output:

Numpy Results = 
[[ 2.   +0.j    0.   +0.j    -0.   -0.j    1.586+1.586j]
[ 0.   +0.j    2.   +0.j    0.   +0.j    -0.651-0.651j]
[-0.   +0.j    0.   -0.j    2.   +0.j    -1.027-1.027j]
[1.586-1.586j -0.651+0.651j -1.027+1.027j  5.   +0.j   ]]

Scipy Results =
[[ 1.+0.j -0.-0.j -0.-0.j  0.-0.j]
[-0.+0.j  1.+0.j  0.+0.j -0.+0.j]
[-0.+0.j  0.-0.j  1.+0.j  0.+0.j]
[ 0.+0.j -0.-0.j  0.-0.j  1.+0.j]]

Runtime information:

Numpy version.

'1.26.0'

Numpy configuration

Build Dependencies:
  blas:
    detection method: pkgconfig
    found: true
    include directory: /Users/shinjan/Programs/miniforge3/envs/scicomp/include
    lib directory: /Users/shinjan/Programs/miniforge3/envs/scicomp/lib
    name: blas
    pc file directory: /Users/shinjan/Programs/miniforge3/envs/scicomp/lib/pkgconfig
    version: 3.9.0
  lapack:
    detection method: pkgconfig
    found: true
    include directory: /Users/shinjan/Programs/miniforge3/envs/scicomp/include
    lib directory: /Users/shinjan/Programs/miniforge3/envs/scicomp/lib
    name: lapack
    pc file directory: /Users/shinjan/Programs/miniforge3/envs/scicomp/lib/pkgconfig
    version: 3.9.0
Compilers:
  c:
    commands: arm64-apple-darwin20.0.0-clang
    linker: ld64
    name: clang
    version: 15.0.7
  c++:
    commands: arm64-apple-darwin20.0.0-clang++
    linker: ld64
    name: clang
    version: 15.0.7
  cython:
    commands: cython
    linker: cython
    name: cython
    version: 3.0.2
Machine Information:
  build:
    cpu: aarch64
    endian: little
    family: aarch64
    system: darwin
  cross-compiled: true
  host:
    cpu: arm64
    endian: little
    family: aarch64
    system: darwin
Python Information:
  path: /Users/shinjan/Programs/miniforge3/envs/scicomp/bin/python
  version: '3.11'
SIMD Extensions:
  baseline:
  - NEON
  - NEON_FP16
  - NEON_VFPV4
  - ASIMD
  found:
  - ASIMDHP
  not found:
  - ASIMDFHM

Scipy Version

'1.11.3'

Scipy Configuration

Build Dependencies:
  blas:
    detection method: pkgconfig
    found: true
    include directory: /Users/shinjan/Programs/miniforge3/envs/scicomp/include
    lib directory: /Users/shinjan/Programs/miniforge3/envs/scicomp/lib
    name: blas
    openblas configuration: unknown
    pc file directory: /Users/shinjan/Programs/miniforge3/envs/scicomp/lib/pkgconfig
    version: 3.9.0
  lapack:
    detection method: pkgconfig
    found: true
    include directory: /Users/shinjan/Programs/miniforge3/envs/scicomp/include
    lib directory: /Users/shinjan/Programs/miniforge3/envs/scicomp/lib
    name: lapack
    openblas configuration: unknown
    pc file directory: /Users/shinjan/Programs/miniforge3/envs/scicomp/lib/pkgconfig
    version: 3.9.0
  pybind11:
    detection method: pkgconfig
    include directory: /Users/shinjan/Programs/miniforge3/envs/scicomp/include
    name: pybind11
    version: 2.11.1
Compilers:
  c:
    commands: arm64-apple-darwin20.0.0-clang
    linker: ld64
    name: clang
    version: 15.0.7
  c++:
    commands: arm64-apple-darwin20.0.0-clang++
    linker: ld64
    name: clang
    version: 15.0.7
  cython:
    commands: cython
    linker: cython
    name: cython
    version: 0.29.36
  fortran:
    commands: /Users/runner/miniforge3/conda-bld/scipy-split_1696467662374/_build_env/bin/arm64-apple-darwin20.0.0-gfortran
    linker: ld64
    name: gcc
    version: 12.3.0
  pythran:
    include directory: ../../_build_env/venv/lib/python3.11/site-packages/pythran
    version: 0.14.0
Machine Information:
  build:
    cpu: x86_64
    endian: little
    family: x86_64
    system: darwin
  cross-compiled: true
  host:
    cpu: arm64
    endian: little
    family: aarch64
    system: darwin
Python Information:
  path: /Users/shinjan/Programs/miniforge3/envs/scicomp/bin/python
  version: '3.11'

Context for the issue:

Erroneous eigenvectors from numpy.linalg.eigh

[seberg]

How did you install numpy and what does np.show_runtime() say? show_config() is not really useful.
Or just threadpoolctl.threadpool_info() after importing everything (you must install threadpoolctl in both cases to get anything useful).

[ShinjanM]

Output for np.show_runtime()

[{'numpy_version': '1.26.0',
  'python': '3.11.6 | packaged by conda-forge | (main, Oct  3 2023, 10:37:07) '
            '[Clang 15.0.7 ]',
  'uname': uname_result(system='Darwin', node='Shinjans-MacBook-Air.local', release='23.0.0', version='Darwin Kernel Version 23.0.0: Fri Sep 15 14:42:57 PDT 2023; root:xnu-10002.1.13~1/RELEASE_ARM64_T8112', machine='arm64')},
 {'simd_extensions': {'baseline': ['NEON', 'NEON_FP16', 'NEON_VFPV4', 'ASIMD'],
                      'found': ['ASIMDHP'],
                      'not_found': ['ASIMDFHM']}}]
None

Output for threadpoolctl.threadpool_info()

[]

I installed the numpy library via conda using:

conda create -n qtm python=3.11 numpy "libblas=*=*accelerate"

and the scipy library using:

conda install -c conda-forge scipy 

I wanted to use the vecLib framework for Apple, which is supposed to be optimum for Apple Silicon.

[seberg]

Did you import numpy and scipy.linalg before the threadpoolctl.thradpool_info()? Might be that it doesn't show anything for accelerate, so you would have to inspect tihngs closer. In that case it seems like this might be an accelerate issue.

[rgommers]

I cannot reproduce this on arm64 macOS 14, with Accelerate built from source I get:

array([[ 1.00000000e+00+0.00000000e+00j, -3.33066907e-16-6.93889390e-17j,
         2.77555756e-16+2.77555756e-17j,  1.04083409e-16+4.16333634e-17j],
       [-3.33066907e-16+6.93889390e-17j,  1.00000000e+00+0.00000000e+00j,
        -1.31838984e-16+2.08166817e-16j, -1.42247325e-16+1.90819582e-16j],
       [ 2.77555756e-16-2.77555756e-17j, -1.31838984e-16-2.08166817e-16j,
         1.00000000e+00+0.00000000e+00j,  1.04083409e-16+1.52655666e-16j],
       [ 1.04083409e-16-4.16333634e-17j, -1.42247325e-16-1.90819582e-16j,
         1.04083409e-16-1.52655666e-16j,  1.00000000e+00+0.00000000e+00j]])

Surely this is also tested in our test suite.

threadpoolctl indeed doesn't recognize new Accelerate, I'll follow up on that. EDIT: see joblib/threadpoolctl#135

conda create -n qtm python=3.11 numpy "libblas=*=*accelerate"

This is not the new Accelerate, but only old Accelerate BLAS plus Netlib LAPACK routines. That config doesn't get tested even in conda-forge CI, so I would avoid using it.

[ShinjanM]

This is not the new Accelerate, but only old Accelerate BLAS plus Netlib LAPACK routines. That config doesn't get tested even in conda-forge CI, so I would avoid using it.

How do I get the latest Accelerate via conda-forge?

Surely this is also tested in our test suite.

The test suite is also reporting errors. Should I attach the entire error report here?

[rgommers]

You can't - it's an open feature request for conda-forge, I linked to it above. The only way to get new Accelerate right now is to build from source.

The test suite is also reporting errors. Should I attach the entire error report here?

May be useful for future reference. It's unlike we can fix things though.