Tests fails on Mac ARM and packages from macports #162
Comments
|
Interesting, I cannot reproduce on my M1 mac. Out of curiosity, how did you install numpy / openblas?
|
|
Irrespective of this, please feel free to open a PR with your suggested fix. |
|
From your initial report, I suppose you used macports to install numpy and openblas. It's weird that it introduces casing variability but so be it. We might want to make the assertion in the test case-insensitive. threadpoolctl should probably still report the name with the original casing of the underlying library though, just in case it matters for debugging low-level runtime/packaging problems. |
|
Yes, I'm working with MacPorts, where I also co-maintain a couple of GIS related ports (MP lingo for packages). I turns out the command gives the following result: % /opt/local/bin/python3.12 -m threadpoolctl -i numpy
[]I also tried programmatically using examples in the README with the same negative result. What could be the reason to this? |
|
For what it is worth, MPs OpenBLAS in not built with OpenMP support: According to Conda recipe, the build target for Apple ARM is |
I think so and opened #165 |
Hard to tell. Maybe the name of the shared library file doesn't match what we expect or we're not able to find the expected symbols in it. I'd apply this patch before running the command line again to check to check if we're missing the library somehow or if it's just not loaded in the first place. diff --git a/threadpoolctl.py b/threadpoolctl.py
index cfebcb5..d913bc8 100644
--- a/threadpoolctl.py
+++ b/threadpoolctl.py
@@ -986,6 +986,7 @@ class ThreadpoolController:
for i in range(n_dyld):
filepath = ctypes.string_at(libc._dyld_get_image_name(i))
filepath = filepath.decode("utf-8")
+ print(filepath)
# Store the library controller if it is supported and selected
self._make_controller_from_path(filepath)
@@ -1070,6 +1071,7 @@ class ThreadpoolController:
# library. move to next library.
if prefix is None:
continue
+ print("detected prefix", prefix)
# workaround for BLAS libraries packaged by conda-forge on windows, which
# are all renamed "libblas.dll". We thus have to check to which BLAS
@@ -1102,6 +1104,8 @@ class ThreadpoolController:
for func in controller_class.check_symbols
):
self.lib_controllers.append(lib_controller)
+ else:
+ print("no required symbols")
def _check_prefix(self, library_basename, filename_prefixes):
"""Return the prefix library_basename starts with |
|
The patch results in a number of lines with "detected prefix None". |
|
Right, I updated the patch to only show the relevant ones to avoid the noise |
|
Now we have tree lines of "detected prefix libblas". |
|
hum... We explicitely rule out shared libs named It may be time to reconsider supporting |
|
And the fact that you detect 3 libblas and not just one is a hint that there's something fishy. I'm not sure we can reliably handle |
|
Where is this "libblas.dylib" supposed to be? I don't have it in MacPorts directory. |
|
you should have the path just above the "detected prefix libblas" lines. Could you paste the full output here ? |
Or where (in code) is this detected? |
|
Aha, is comes from system: |
|
note that this is not openblas but macos' veclib which we don't support because it doesn't expose an api to control the number of threads which is the purpose of threadpoolctl |
|
Is this something loaded from threadpoolctl or numpy? |
|
the shared lib is first loaded by numpy, then threadpoolctl inspect all loaded libs but doesn't load them itself |
|
Seem numpy isn't involved in that part. I run with a non-existing module: |
|
Or do you mean all loaded libs at all? |
|
Then it means that python itself is linked against veclib (which is kind if odd). threadpoolctl just inspect the list of shared libs loaded by a python program (and importing a package like numpy can trigger the loading of new shared libs like blas) but won't try to load a lib that is not already loaded by the program. |
Note that we will support it partially via flexiblas: you won't be able to inspect / set the number of threads but you should be able to see that it's the active backend of flexiblas. See also the related #135. |
ogrisel
changed the title
|
Thanks. The MP's Numpy seems to be faulty, I filed a ticket for that in https://trac.macports.org/ticket/69326. |
|
otool -L /Users/ogrisel/mambaforge-disabled/envs/dev/lib/libblas.3.dylib |
Running the tests in a MacPorts setting, on an Apple ARM machine, the architecture test fails:
The following patch make the tests succeed:
Would that be a proper fix?
The text was updated successfully, but these errors were encountered: