Getting Segfault while compiling the Faiss (faiss::{anonymous}::kernel_accumulate_block) on ARM

[navneet1v]

Summary

While compiling the Faiss lib on the ARM from source I am getting the below SegFault. This is happening only for gcc version 8.3.1. For gcc version 7.xx it is working correctly.

[ 70%] Building CXX object external/faiss/faiss/CMakeFiles/faiss.dir/impl/pq4_fast_scan_search_1.cpp.o
during RTL pass: expand
/usr/share/opensearch/k-NN/jni/external/faiss/faiss/impl/pq4_fast_scan_search_1.cpp: In function 'void faiss::{anonymous}::kernel_accumulate_block(int, const uint8_t*, const uint8_t*, ResultHandler&, const Scaler&) [with int NQ = 2; int BB = 1; ResultHandler = faiss::simd_result_handlers::FixedStorageHandler<2, 2>; Scaler = faiss::DummyScaler]':
/usr/share/opensearch/k-NN/jni/external/faiss/faiss/impl/pq4_fast_scan_search_1.cpp:60:65: internal compiler error: Segmentation fault
             simd32uint8 chi = simd32uint8(simd16uint16(c) >> 4) & mask;
                               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~

Platform

OS: Linux 178893520d2b 6.1.34-59.116.amzn2023.aarch64 aarch64 aarch64 aarch64 GNU/Linux

gcc version: gcc (GCC) 8.3.1 20190311

Faiss version:

3219e3d

Installed from:
Complied from Source Code

Faiss compilation options:

set(FAISS_ENABLE_GPU OFF)
set(FAISS_ENABLE_PYTHON OFF)
target_link_libraries(${TARGET_LIB_FAISS} faiss ${TARGET_LIB_COMMON} OpenMP::OpenMP_CXX)

-- CXX flags release: -Wall -Wunreachable-code -Wcast-align -Ofast -lm -DNDEBUG -std=c++11 -pthread  -DHAVE_CXX0X  -march=native -Wl,--no-as-needed -fpic

Running on:

• CPU
• GPU

Interface:

• C++
• Python

Reproduction instructions

Compile the Faiss lib from source code on arm machine with GCC version 8.3.1.

ERROR:

[ 70%] Building CXX object external/faiss/faiss/CMakeFiles/faiss.dir/impl/pq4_fast_scan_search_1.cpp.o
during RTL pass: expand
/usr/share/opensearch/k-NN/jni/external/faiss/faiss/impl/pq4_fast_scan_search_1.cpp: In function 'void faiss::{anonymous}::kernel_accumulate_block(int, const uint8_t*, const uint8_t*, ResultHandler&, const Scaler&) [with int NQ = 2; int BB = 1; ResultHandler = faiss::simd_result_handlers::FixedStorageHandler<2, 2>; Scaler = faiss::DummyScaler]':
/usr/share/opensearch/k-NN/jni/external/faiss/faiss/impl/pq4_fast_scan_search_1.cpp:60:65: internal compiler error: Segmentation fault
             simd32uint8 chi = simd32uint8(simd16uint16(c) >> 4) & mask;
                               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~

[mdouze]

As a matter of fact, this is a gcc bug. Please report to GCC and feel free to link the bug report from here.

[peterzhuamazon]

I have confirmed on the arm64 server if we switch to Rockylinux8 and use gcc 8.5.0 the compilation goes to success.

This seems like an isolated issue on CentOS7 arm64 with gcc 8.3.1 version specifically.

Thanks.

[peterzhuamazon]

[100%] Linking CXX shared library release/libopensearchknn_faiss.so
[100%] Built target opensearchknn_faiss

......


Rocky Linux release 8.8 (Green Obsidian)
NAME="Rocky Linux"
VERSION="8.8 (Green Obsidian)"
ID="rocky"
ID_LIKE="rhel centos fedora"
VERSION_ID="8.8"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Rocky Linux 8.8 (Green Obsidian)"
ANSI_COLOR="0;32"
LOGO="fedora-logo-icon"
CPE_NAME="cpe:/o:rocky:rocky:8:GA"
HOME_URL="https://rockylinux.org/"
BUG_REPORT_URL="https://bugs.rockylinux.org/"
SUPPORT_END="2029-05-31"
ROCKY_SUPPORT_PRODUCT="Rocky-Linux-8"
ROCKY_SUPPORT_PRODUCT_VERSION="8.8"
REDHAT_SUPPORT_PRODUCT="Rocky Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="8.8"
Rocky Linux release 8.8 (Green Obsidian)
Rocky Linux release 8.8 (Green Obsidian)
Rocky Linux release 8.8 (Green Obsidian)

......


gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-18)
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.