Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Arch Migrator #58

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .azure-pipelines/azure-pipelines-linux.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 5 additions & 1 deletion .azure-pipelines/azure-pipelines-win.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ cuda_compiler_version:
cxx_compiler:
- gxx
cxx_compiler_version:
- '10'
- '11'
docker_image:
- quay.io/condaforge/linux-anvil-cos7-x86_64
libblas:
Expand Down
47 changes: 47 additions & 0 deletions .ci_support/linux_aarch64_.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
BUILD:
- aarch64-conda_cos7-linux-gnu
cdt_arch:
- aarch64
cdt_name:
- cos7
channel_sources:
- conda-forge
channel_targets:
- conda-forge main
cuda_compiler:
- nvcc
cuda_compiler_version:
- None
cxx_compiler:
- gxx
cxx_compiler_version:
- '11'
docker_image:
- quay.io/condaforge/linux-anvil-aarch64
libblas:
- 3.9 *netlib
liblapack:
- 3.9 *netlib
numpy:
- '1.21'
- '1.23'
- '1.20'
- '1.20'
pin_run_as_build:
python:
min_pin: x.x
max_pin: x.x
python:
- 3.10.* *_cpython
- 3.11.* *_cpython
- 3.8.* *_cpython
- 3.9.* *_cpython
target_platform:
- linux-aarch64
zip_keys:
- - cxx_compiler_version
- cuda_compiler_version
- cdt_name
- docker_image
- - python
- numpy
43 changes: 43 additions & 0 deletions .ci_support/linux_ppc64le_.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
cdt_name:
- cos7
channel_sources:
- conda-forge
channel_targets:
- conda-forge main
cuda_compiler:
- nvcc
cuda_compiler_version:
- None
cxx_compiler:
- gxx
cxx_compiler_version:
- '11'
docker_image:
- quay.io/condaforge/linux-anvil-ppc64le
libblas:
- 3.9 *netlib
liblapack:
- 3.9 *netlib
numpy:
- '1.21'
- '1.23'
- '1.20'
- '1.20'
pin_run_as_build:
python:
min_pin: x.x
max_pin: x.x
python:
- 3.10.* *_cpython
- 3.11.* *_cpython
- 3.8.* *_cpython
- 3.9.* *_cpython
target_platform:
- linux-ppc64le
zip_keys:
- - cxx_compiler_version
- cuda_compiler_version
- cdt_name
- docker_image
- - python
- numpy
2 changes: 1 addition & 1 deletion .circleci/config.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions .scripts/build_steps.sh

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 2 additions & 3 deletions .scripts/run_osx_build.sh

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

27 changes: 27 additions & 0 deletions .travis.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

41 changes: 31 additions & 10 deletions README.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 5 additions & 2 deletions conda-forge.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
conda_build:
pkg_format: '2'
conda_forge_output_validation: true
github:
branch_name: main
tooling_branch_name: main
conda_build:
pkg_format: '2'
provider:
linux_aarch64: default
linux_ppc64le: default
31 changes: 16 additions & 15 deletions recipe/build-pkg.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,25 +19,26 @@ cmake ${CMAKE_ARGS} \
cmake --build _build_python_generic --target swigfaiss -j $CPU_COUNT

# Build version with avx2 support, see build-lib.sh
cmake ${CMAKE_ARGS} \
-Dfaiss_ROOT=_libfaiss_avx2_stage/ \
-DFAISS_OPT_LEVEL=avx2 \
-DFAISS_ENABLE_GPU=${FAISS_ENABLE_GPU} \
-DCMAKE_BUILD_TYPE=Release \
-DPython_EXECUTABLE="${PYTHON}" \
-B _build_python_avx2 \
faiss/python
cmake --build _build_python_avx2 --target swigfaiss_avx2 -j $CPU_COUNT
if [[ "${target_platform}" == *-64 ]]; then
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this supposed to match win-64 and linux-64 but not linux-aarch64 or linux-ppc64le? What about osx-64? I saw a comment below, the linux & windows CI agents support AVX2 (OSX doesn't yet). I'd prefer to check explicitly for specific matching strings.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't used on Windows. There are .bat build files for that.

AFAICT this was already building on macOS. So this doesn't change that.

All this does is exclude non-x86_64 targets.

cmake ${CMAKE_ARGS} \
-Dfaiss_ROOT=_libfaiss_avx2_stage/ \
-DFAISS_OPT_LEVEL=avx2 \
-DFAISS_ENABLE_GPU=${FAISS_ENABLE_GPU} \
-DCMAKE_BUILD_TYPE=Release \
-DPython_EXECUTABLE="${PYTHON}" \
-B _build_python_avx2 \
faiss/python
cmake --build _build_python_avx2 --target swigfaiss_avx2 -j $CPU_COUNT

# copy generated swig module with avx2-support to specifically named file, cf.
# https://github.com/facebookresearch/faiss/blob/v1.7.1/faiss/python/setup.py#L37-L40
cp _build_python_avx2/swigfaiss_avx2.py _build_python_generic/swigfaiss_avx2.py
cp _build_python_avx2/_swigfaiss_avx2.so _build_python_generic/_swigfaiss_avx2.so
# copy generated swig module with avx2-support to specifically named file, cf.
# https://github.com/facebookresearch/faiss/blob/v1.7.1/faiss/python/setup.py#L37-L40
cp _build_python_avx2/swigfaiss_avx2.py _build_python_generic/swigfaiss_avx2.py
cp _build_python_avx2/_swigfaiss_avx2.so _build_python_generic/_swigfaiss_avx2.so
fi

# Build actual python module.
pushd _build_python_generic
$PYTHON setup.py install --single-version-externally-managed --record=record.txt --prefix=$PREFIX
popd
# clean up cmake-cache between builds
rm -r _build_python_generic
rm -r _build_python_avx2
rm -r _build_python_*
Loading