Skip to content
This repository has been archived by the owner on Jan 28, 2024. It is now read-only.

Installation error on pyfnntw==0.4.0 #5

Open
mushroomfire opened this issue Mar 6, 2023 · 4 comments
Open

Installation error on pyfnntw==0.4.0 #5

mushroomfire opened this issue Mar 6, 2023 · 4 comments

Comments

@mushroomfire
Copy link

I install pyfnntw by:

pip install pyfnntw==0.4.0

I use Conda environment and the PYTHON version is 3.11.0 (as required in pyproject.toml) ans OS is windows10.
The output is as below:

Collecting pyfnntw==0.4.0
  Downloading pyfnntw-0.4.0.tar.gz (71 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 71.1/71.1 kB 647.0 kB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: pyfnntw
  Building wheel for pyfnntw (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for pyfnntw (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [84 lines of output]
      Running `maturin pep517 build-wheel -i D:\Anaconda\envs\py311\python.exe --compatibility off`
      馃敆 Found pyo3 bindings
      馃悕 Found CPython 3.11 at D:\Anaconda\envs\py311\python.exe
         Compiling autocfg v1.1.0
         Compiling proc-macro2 v1.0.51
         Compiling quote v1.0.21
         Compiling unicode-ident v1.0.3
         Compiling syn v1.0.109
         Compiling thiserror v1.0.39
         Compiling cfg-if v1.0.0
         Compiling target-lexicon v0.12.4
         Compiling scopeguard v1.1.0
         Compiling crossbeam-utils v0.8.15
         Compiling version_check v0.9.4
         Compiling once_cell v1.13.1
         Compiling ucd-trie v0.1.5
         Compiling windows_x86_64_msvc v0.36.1
         Compiling rayon-core v1.11.0
         Compiling num_cpus v1.15.0
         Compiling libc v0.2.132
         Compiling parking_lot_core v0.9.3
         Compiling const_fn v0.4.9
         Compiling tcmalloc-sys v0.3.0
         Compiling either v1.8.1
         Compiling arrayvec v0.4.12
         Compiling smallvec v1.9.0
         Compiling Inflector v0.11.4
         Compiling rawpointer v0.2.1
         Compiling nodrop v0.1.14
         Compiling itoa v0.4.8
         Compiling unindent v0.1.10
         Compiling aliasable v0.1.3
         Compiling indoc v1.0.7
         Compiling permutation v0.4.1
         Compiling sync-unsafe-cell v0.1.0
         Compiling matrixmultiply v0.3.2
         Compiling memoffset v0.8.0
         Compiling num-traits v0.2.15
         Compiling crossbeam-epoch v0.9.14
         Compiling lock_api v0.4.7
         Compiling num-integer v0.1.45
         Compiling proc-macro-error-attr v1.0.4
         Compiling proc-macro-error v1.0.4
         Compiling windows-sys v0.36.1
         Compiling tcmalloc v0.3.0
         Compiling num-format v0.4.0
         Compiling pyo3-build-config v0.16.5
         Compiling crossbeam-channel v0.5.7
         Compiling crossbeam-deque v0.8.3
         Compiling num-complex v0.4.3
         Compiling ordered-float v3.0.0
         Compiling rayon v1.7.0
         Compiling parking_lot v0.12.1
         Compiling pyo3-ffi v0.16.5
         Compiling pyo3 v0.16.5
         Compiling ndarray v0.15.6
         Compiling pyo3-macros-backend v0.16.5
         Compiling thiserror-impl v1.0.39
         Compiling ouroboros_macro v0.15.6
         Compiling concat-idents v1.1.4
         Compiling pest v2.5.6
         Compiling ouroboros v0.15.6
         Compiling pyo3-macros v0.16.5
         Compiling semver-parser v0.10.2
         Compiling semver v0.11.0
         Compiling rustc_version v0.3.3
         Compiling likely_stable v0.1.2
         Compiling fnntw v0.4.0 (C:\Users\Administrator\AppData\Local\Temp\pip-install-uv8o5oyw\pyfnntw_89a06f849e7240068ea92b3e9ceaee94\local_dependencies\fnntw)
         Compiling numpy v0.16.2
         Compiling pyfnntw v0.4.0 (C:\Users\Administrator\AppData\Local\Temp\pip-install-uv8o5oyw\pyfnntw_89a06f849e7240068ea92b3e9ceaee94)
      error: could not compile `pyfnntw` due to 2 previous errors
      馃挜 maturin failed
        Caused by: Failed to build a native library through cargo
        Caused by: Cargo build finished with "exit code: 101": `"cargo" "rustc" "--release" "--manifest-path" "C:\\Users\\Administrator\\AppData\\Local\\Temp\\pip-install-uv8o5oyw\\pyfnntw_89a06f849e7240068ea92b3e9ceaee94\\Cargo.toml" "--message-format" "json" "--lib"`
      error: linking with `link.exe` failed: exit code: 1181
        |
        = note: "C:\\Program Files (x86)\\Microsoft Visual Studio\\2022\\BuildTools\\VC\\Tools\\MSVC\\14.33.31629\\bin\\HostX64\\x64\\link.exe" "/DEF:C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\rustc25OP3V\\lib.def" "/NOLOGO" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\rustc25OP3V\\symbols.o" "C:\\Users\\Administrator\\AppData\\Local\\Temp\\pip-install-uv8o5oyw\\pyfnntw_89a06f849e7240068ea92b3e9ceaee94\\target\\release\\deps\\pyfnntw.pyfnntw.5efcba88-cgu.0.rcgu.o" "/LIBPATH:C:\\Users\\Administrator\\AppData\\Local\\Temp\\pip-install-uv8o5oyw\\pyfnntw_89a06f849e7240068ea92b3e9ceaee94\\target\\release\\deps" "/LIBPATH:C:\\Users\\Administrator\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\windows_x86_64_msvc-0.36.1\\lib" "/LIBPATH:D:\\Anaconda\\envs\\py311\\libs" "/LIBPATH:C:\\Users\\Administrator\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib" "C:\\Users\\Administrator\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libcompiler_builtins-3f0eed38bcc54ffc.rlib" "windows.lib" "python311.lib" "legacy_stdio_definitions.lib" "tcmalloc.lib" "kernel32.lib" "advapi32.lib" "userenv.lib" "kernel32.lib" "ws2_32.lib" "bcrypt.lib" "msvcrt.lib" "legacy_stdio_definitions.lib" "/NXCOMPAT" "/LIBPATH:C:\\Users\\Administrator\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib" "/OUT:C:\\Users\\Administrator\\AppData\\Local\\Temp\\pip-install-uv8o5oyw\\pyfnntw_89a06f849e7240068ea92b3e9ceaee94\\target\\release\\deps\\pyfnntw.dll" "/OPT:REF,ICF" "/DLL" "/IMPLIB:C:\\Users\\Administrator\\AppData\\Local\\Temp\\pip-install-uv8o5oyw\\pyfnntw_89a06f849e7240068ea92b3e9ceaee94\\target\\release\\deps\\pyfnntw.dll.lib" "/DEBUG" "/NATVIS:C:\\Users\\Administrator\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\intrinsic.natvis" "/NATVIS:C:\\Users\\Administrator\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\liballoc.natvis" "/NATVIS:C:\\Users\\Administrator\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\libcore.natvis" "/NATVIS:C:\\Users\\Administrator\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\libstd.natvis"
        = note: Non-UTF-8 output: LINK : fatal error LNK1181: \xce\xde\xb7\xa8\xb4\xf2\xbf\xaa\xca\xe4\xc8\xeb\xce\xc4\xbc\xfe\xa1\xb0tcmalloc.lib\xa1\xb1\r\n


      error: aborting due to previous error


      Error: command ['maturin', 'pep517', 'build-wheel', '-i', 'D:\\Anaconda\\envs\\py311\\python.exe', '--compatibility', 'off'] returned non-zero exit status 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pyfnntw
Failed to build pyfnntw
ERROR: Could not build wheels for pyfnntw, which is required to install pyproject.toml-based projects
NativeCommandExitException: Program "pip.exe" ended with non-zero exit code: 1.

In my WSL, the error seems to be related to tcmalloc-sys v0.3.0.
Can you tell me how to solve it?
BTW., why the PYTHON version is required larger than 3.11.0? It would be helpful to change it to >=3.7.0.

@cavemanloverboy
Copy link
Owner

I will switch it back to >3.7. I believe I also accidentally left out a feature... I will fix these

@cavemanloverboy
Copy link
Owner

go ahead and try again. If it doesn't work... try disabling tcmallocator feature and let me know if it works

@cavemanloverboy
Copy link
Owner

cavemanloverboy commented Mar 6, 2023

4.1 is out with >3.7 and no tcmallocator

@mushroomfire
Copy link
Author

4.1 is out with >3.7 and no tcmallocator

I have successfully installed pytnntw==0.4.1 and everything works well. My test script is as below:

import numpy as np
from time import time
from scipy.spatial import KDTree
import pyfnntw

N = 10**7
k = 12
pos = np.random.random((N, 3))

start = time()
kdt = KDTree(pos, 32, boxsize=[1.0, 1.0, 1.0])
print(f"Scipy build tree time: {time()-start} s.")
start1 = time()
dis, indices = kdt.query(pos, k, workers=-1)
end = time()
print(f"Scipy query time: {end-start1} s.")
print(f"Scipy total time: {end-start} s.")
print(indices[0])

start = time()
kdt = pyfnntw.Treef64(
    pos, leafsize=32, par_split_level=4, boxsize=np.array([1.0, 1.0, 1.0])
)
print(f"Pyfnntw build tree time: {time()-start} s.")
start1 = time()
dis, indices = kdt.query(pos, k)
end = time()
print(f"Pyfnntw query time: {end-start1} s.")
print(f"Pyfnntw total time: {end-start} s.")
print(indices[0])

In my computer, Intel Xeon E5-2630 @2.2GHz (20 cores), the output is as below:

Scipy build tree time: 17.172202825546265 s.
Scipy query time: 10.185096025466919 s.
Scipy total time: 27.357298851013184 s.
[      0  635729 4951542 2569924 7108333 5208854 3171473 9350673 3314987
 1842148 5125748 2983723]
Parallelism activated: 40 threads
Pyfnntw build tree time: 1.5725889205932617 s.
Pyfnntw query time: 10.762544393539429 s.
Pyfnntw total time: 12.336133241653442 s.
[      0  635729 4951542 2569924 7108333 5208854 3171473 9350673 3314987
 1842148 5125748 2983723]

The pyfnntw shows excellent performance on building KDtree aginst that in Scipy. Thanks for your significant work, and I have add pyfnntw to my library to accelerate the NearestNeighbor Search.
BTW., one suggestion:
Provide pre-compiled wheel for different OS system, wchich makes installation easier for Python users.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants