ngt: 1.8.4 -> 1.12.3-alpha

[tomberek]

Motivation for this change

Version bump

Things done

hash update

• Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Ensured that relevant documentation is up to date
• Fits CONTRIBUTING.md.

[SuperSandro2000]

Build errors on darwin:

-- The C compiler identification is Clang 7.1.0
-- The CXX compiler identification is Clang 7.1.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /nix/store/gkcr1p2x0psdc31bavmqbfb2nwq0d6jn-clang-wrapper-7.1.0/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /nix/store/gkcr1p2x0psdc31bavmqbfb2nwq0d6jn-clang-wrapper-7.1.0/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- VERSION: 1.12.2
-- CMAKE_BUILD_TYPE: Release
-- CMAKE_BUILD_TYPE_LOWER: release
CMake Error at /nix/store/z7qvx6wbvmjvs767g06h03pkzr5aqn6i-cmake-3.18.2/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:165 (message):
  Could NOT find OpenMP_C (missing: OpenMP_C_FLAGS OpenMP_C_LIB_NAMES)
Call Stack (most recent call first):
  /nix/store/z7qvx6wbvmjvs767g06h03pkzr5aqn6i-cmake-3.18.2/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:458 (_FPHSA_FAILURE_MESSAGE)
  /nix/store/z7qvx6wbvmjvs767g06h03pkzr5aqn6i-cmake-3.18.2/share/cmake-3.18/Modules/FindOpenMP.cmake:529 (find_package_handle_standard_args)
  CMakeLists.txt:70 (find_package)


-- Configuring incomplete, errors occurred!
See also "/private/var/folders/r4/k85s52jx3555ckj0wty615s00000gp/T/nix-build-NGT-v1.12.2.drv-0/source/build/CMakeFiles/CMakeOutput.log".
See also "/private/var/folders/r4/k85s52jx3555ckj0wty615s00000gp/T/nix-build-NGT-v1.12.2.drv-0/source/build/CMakeFiles/CMakeError.log".

[tomberek]

@SuperSandro2000 i have some notes with this addition:

buildInputs = [ openmpi ];
NIX_ENFORCE_NO_NATIVE = false;

The NIX var may not be needed. Can you test something like that out?

[SuperSandro2000]

Still fails with my changes but much further down compiling:

7 warnings generated.
[ 87%] Linking CXX executable l2-uint8-range-search
7 warnings generated.
7 warnings generated.
[ 89%] Linking CXX executable ngt
[ 92%] Linking CXX executable cosine-float
[ 92%] Built target l2-uint8-range-search
[ 92%] Built target cosine-float
[ 92%] Built target ngt_exe
make: *** [Makefile:149: all] Error 2
builder for '/nix/store/9ynaf0k66xjk59phgxmn7zf3nhz41nzk-NGT-v1.12.3-alpha.drv' failed with exit code 2
error: build of '/nix/store/9ynaf0k66xjk59phgxmn7zf3nhz41nzk-NGT-v1.12.3-alpha.drv' failed

[tomberek]

Incorporated your suggestions. I'm getting a hold of a Darwin machine to resolve your build issue.

[tomberek]

Result of nixpkgs-review pr 101790 1

1 package built:

- ngt

[tomberek]

OSX build works on my end

[SuperSandro2000]

Run on OS X Catalina:

Result of nixpkgs-review pr 101790 1

1 package failed to build:

• ngt

/private/var/folders/r4/k85s52jx3555ckj0wty615s00000gp/T/nix-build-NGT-v1.12.3-alpha.drv-0/source/lib/NGT/Index.h:1589:10: warning: 'NGT::GraphAndTreeIndex::verify' hides overloaded virtual function [-Woverloaded-virtual]                                                                                                                                                               bool verify(std::vector<uint8_t> &status, bool info, char mode);                                                                                                                                 ^                                                                                                                                                                                  /private/var/folders/r4/k85s52jx3555ckj0wty615s00000gp/T/nix-build-NGT-v1.12.3-alpha.drv-0/source/lib/NGT/Index.h:931:18: note: hidden overloaded virtual function 'NGT::GraphIndex::verify' declared here: different number of parameters (2 vs 3)                                                                                                                                         virtual bool verify(std::vector<uint8_t> &status, bool info)                                                                                                                                             ^                                                                                                                                                                          In file included from /private/var/folders/r4/k85s52jx3555ckj0wty615s00000gp/T/nix-build-NGT-v1.12.3-alpha.drv-0/source/bin/ngtq/ngtq.cpp:17:                                               In file included from /private/var/folders/r4/k85s52jx3555ckj0wty615s00000gp/T/nix-build-NGT-v1.12.3-alpha.drv-0/source/lib/NGT/NGTQ/Command.h:17:                                          /private/var/folders/r4/k85s52jx3555ckj0wty615s00000gp/T/nix-build-NGT-v1.12.3-alpha.drv-0/source/lib/NGT/NGTQ/Quantizer.h:534:7: error: unknown type name '__m128'
      __m128 sum = _mm_setzero_ps();
      ^
/private/var/folders/r4/k85s52jx3555ckj0wty615s00000gp/T/nix-build-NGT-v1.12.3-alpha.drv-0/source/lib/NGT/NGTQ/Quantizer.h:534:20: error: use of undeclared identifier '_mm_setzero_ps'
      __m128 sum = _mm_setzero_ps();
                   ^
/private/var/folders/r4/k85s52jx3555ckj0wty615s00000gp/T/nix-build-NGT-v1.12.3-alpha.drv-0/source/lib/NGT/NGTQ/Quantizer.h:536:2: error: unknown type name '__m128i'
        __m128i x1 = _mm_cvtepu8_epi32(*(__m128i const*)optr);
        ^
/private/var/folders/r4/k85s52jx3555ckj0wty615s00000gp/T/nix-build-NGT-v1.12.3-alpha.drv-0/source/lib/NGT/NGTQ/Quantizer.h:536:35: error: use of undeclared identifier '__m128i'
        __m128i x1 = _mm_cvtepu8_epi32(*(__m128i const*)optr);
                                         ^
/private/var/folders/r4/k85s52jx3555ckj0wty615s00000gp/T/nix-build-NGT-v1.12.3-alpha.drv-0/source/lib/NGT/NGTQ/Quantizer.h:537:2: error: unknown type name '__m128i'
        __m128i x2 = _mm_cvtepu8_epi32(*(__m128i const*)gcptr);
        ^
/private/var/folders/r4/k85s52jx3555ckj0wty615s00000gp/T/nix-build-NGT-v1.12.3-alpha.drv-0/source/lib/NGT/NGTQ/Quantizer.h:537:35: error: use of undeclared identifier '__m128i'
        __m128i x2 = _mm_cvtepu8_epi32(*(__m128i const*)gcptr);
                                         ^
/private/var/folders/r4/k85s52jx3555ckj0wty615s00000gp/T/nix-build-NGT-v1.12.3-alpha.drv-0/source/lib/NGT/NGTQ/Quantizer.h:539:2: error: unknown type name '__m128'
        __m128 sub = _mm_sub_ps(_mm_cvtepi32_ps(x1), *(__m128 const*)lcptr);
        ^
/private/var/folders/r4/k85s52jx3555ckj0wty615s00000gp/T/nix-build-NGT-v1.12.3-alpha.drv-0/source/lib/NGT/NGTQ/Quantizer.h:539:49: error: use of undeclared identifier '__m128'
        __m128 sub = _mm_sub_ps(_mm_cvtepi32_ps(x1), *(__m128 const*)lcptr);
                                                       ^
7 warnings generated.
[ 76%] Linking CXX executable l2-uint8
8 warnings and 8 errors generated.
make[2]: *** [bin/ngtq/CMakeFiles/ngtq_exe.dir/build.make:82: bin/ngtq/CMakeFiles/ngtq_exe.dir/ngtq.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:425: bin/ngtq/CMakeFiles/ngtq_exe.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

[tomberek]

@SuperSandro2000 It seems those are AVX instructions failing. Perhaps we've got some hardware differences. Does it work if you remove the AVX?

[SuperSandro2000]

@SuperSandro2000 It seems those are AVX instructions failing. Perhaps we've got some hardware differences. Does it work if you remove the AVX?

Yes it builds without avx enabled.

[tomberek]

Disabled AVX by default.

[SuperSandro2000]

Result of nixpkgs-review pr 101790 run on x86_64-darwin 1

1 package built:

• ngt

[tomberek]

Result of nixpkgs-review pr 101790 1

1 package built:

• ngt

[SuperSandro2000]

Result of nixpkgs-review pr 101790 run on x86_64-linux 1

1 package built:

• ngt