Skip to content

Commit

Permalink
Update developer guide to include M1 Setup (opensearch-project#1222)
Browse files Browse the repository at this point in the history
* Update Developer Guide to include M1 setup

Signed-off-by: Ryan Bogan <[email protected]>
Co-authored-by: Heemin Kim <[email protected]>
  • Loading branch information
ryanbogan and heemin32 authored Oct 20, 2023
1 parent 0862a53 commit 318cf48
Show file tree
Hide file tree
Showing 2 changed files with 51 additions and 1 deletion.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,5 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
### Infrastructure
### Documentation
### Maintenance
* Update developer guide to include M1 Setup [#1222](https://github.com/opensearch-project/k-NN/pull/1222)
### Refactoring
51 changes: 50 additions & 1 deletion DEVELOPER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,11 @@ One easy way to install on mac or linux is to use pip:
pip install cmake==3.23.3
```

On Mac M1 machines, install cmake using:
```bash
brew install cmake
```

#### Faiss Dependencies

To build the *faiss* JNI library, you need to have openmp, lapack and blas installed. For more information on *faiss*
Expand All @@ -78,6 +83,50 @@ Additionally, the `gcc` toolchain needs to be installed on Mac. To install, run:
brew install gcc
```

#### Extra setup for Mac M1 Machines

The following commands enable running/building k-NN on M1 machines:

```bash
// Go to k-NN folder
cd k-NN

// Build to generate the necessary files to be modified below (will fail)
./gradlew build

//Go to jni folder
cd jni

// File changes required
sed -i -e 's/\/usr\/local\/opt\/libomp\//\/opt\/homebrew\/opt\/llvm\//g' CMakeLists.txt
sed -i -e 's/-march=native/-mcpu=apple-m1/g' external/nmslib/similarity_search/CMakeLists.txt
sed -i -e 's/pragma message WARN/pragma message /g' external/nmslib/similarity_search/src/distcomp_scalar.cc
sed -i -e 's/-mcpu=apple-a14/-mcpu=apple-m1/g' external/nmslib/python_bindings/setup.py
sed -i -e 's/__aarch64__/__undefine_aarch64__/g' external/faiss/faiss/utils/distances_simd.cpp

// Install llvm
brew install llvm
echo 'export PATH="/opt/homebrew/opt/llvm/bin:$PATH"' >> ~/.zshrc
source ~/.zshrc

// Set compiler path for CMAKE
export CC=/opt/homebrew/opt/llvm/bin/clang
export CXX=/opt/homebrew/opt/llvm/bin/clang++

// Build
cmake . --fresh
make
```

Next, obtain a minimum distribution tarball of the k-NN version you want to build:

1. Fork the [OpenSearch Repo](https://github.com/opensearch-project/OpenSearch) into your github account.
2. Clone the repository locally
3. Run the following commands:
```cd OpenSearch && ./gradlew -p distribution/archives/darwin-tar assemble```
4. You should see a opensearch-min-<version>-SNAPSHOT-darwin-x64.tar.gz file present in distribution/archives/darwin-tar/build/distributions/
5. Build k-NN by passing the OpenSearch distribution path in `./gradlew <integTest/run> -PcustomDistributionUrl="<Full path to .tar.gz file you noted above>"`

#### Environment

Currently, the plugin only supports Linux on x64 and arm platforms.
Expand Down Expand Up @@ -114,7 +163,7 @@ Please follow these formatting guidelines:
* Wildcard imports (`import foo.bar.baz.*`) are forbidden and will cause the build to fail.
* If *absolutely* necessary, you can disable formatting for regions of code with the `// tag::NAME` and `// end::NAME` directives, but note that these are intended for use in documentation, so please make it clear what you have done, and only do this where the benefit clearly outweighs the decrease in consistency.
* Note that JavaDoc and block comments i.e. `/* ... */` are not formatted, but line comments i.e `// ...` are.
* There is an implicit rule that negative boolean expressions should use the form `foo == false` instead of `!foo` for better readability of the code. While this isn't strictly enforced, if might get called out in PR reviews as something to change.
* There is an implicit rule that negative boolean expressions should use the form `foo == false` instead of `!foo` for better readability of the code. While this isn't strictly enforced, it might get called out in PR reviews as something to change.

## Build

Expand Down

0 comments on commit 318cf48

Please sign in to comment.