v23.01.00

nv-legate · Jan 30, 2023 · 2455b55 · 2455b55
2 parents 81ad156 + 1817dc7
commit 2455b55
Show file tree

Hide file tree

Showing 237 changed files with 7,019 additions and 6,840 deletions.
diff --git a/.github/release.yml b/.github/release.yml
@@ -0,0 +1,17 @@
+changelog:
+  exclude:
+    labels:
+      - category:task
+  categories:
+    - title: 🐛 Bug Fixes
+      labels:
+        - category:bug-fix
+    - title: 🚀 New Features
+      labels:
+        - category:new-feature
+    - title: 🛠️ Improvements
+      labels:
+        - category:improvement
+    - title: 📖 Documentation
+      labels:
+        - category:documentation
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -6,6 +6,9 @@ on:
   pull_request:
     branches-ignore:
       - gh-pages  # deployment target branch (this workflow should not exist on that branch anyway)
+  schedule:
+    # * is a special character in YAML so you have to quote this string
+    - cron:  '0 */6 * * *'
 env:
   COMMIT: ${{ github.event.pull_request.head.sha || github.sha }}
   PROJECT: github-cunumeric-ci
@@ -57,7 +60,7 @@ jobs:
         if: always()
       - name: Upload Build Log
         if: always()
-        uses: actions/upload-artifact@v2
+        uses: actions/upload-artifact@v3
         with:
           name: build-log
           path: ./**/${{ env.COMMIT }}-build.log.gpg
@@ -74,9 +77,10 @@ jobs:
           - {name: GPU test, options: --use cuda --gpus 1 --debug, log: gpu}
           - {name: 2 GPUs test, options: --use cuda --gpus 2 --debug, log: gpus}
           - {name: OpenMP test, options: --use openmp --omps 1 --ompthreads 2 --debug, log: omp}
-          - {name: 2 OpenMPs test, options: --use openmp --omps 2 --ompthreads 2 --debug, log: omps}
+          - {name: 2 NUMA OpenMPs test, options: --use openmp --omps 2 --ompthreads 2 --numamem 2048 --debug, log: omps}
           - {name: Eager execution test, options: --use eager --debug, log: eager}
           - {name: mypy, options: mypy, log: mypy}
+          - {name: documentation, options: docs, log: docs}
     name: ${{ matrix.name }}
     steps:
       - name: Dump GitHub context
@@ -125,7 +129,7 @@ jobs:
           cat *artifacts/*/*
       - name: Upload Log
         if: always()
-        uses: actions/upload-artifact@v2
+        uses: actions/upload-artifact@v3
         with:
           name: test-${{ matrix.log }}-log
           path: ./**/${{ env.COMMIT }}-test-${{ matrix.log }}.log.gpg
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -1,18 +1,25 @@
 repos:
+    - repo: https://github.com/pre-commit/mirrors-mypy
+      rev: 'v0.991'
+      hooks:
+        - id: mypy
+          language: system
+          pass_filenames: false
+          args: ['cunumeric']
     - repo: https://github.com/PyCQA/isort
-      rev: 5.10.1
+      rev: 5.11.4
       hooks:
             - id: isort
     - repo: https://github.com/psf/black
-      rev: 22.8.0
+      rev: 22.12.0
       hooks:
             - id: black
     - repo: https://github.com/PyCQA/flake8
-      rev: 5.0.4
+      rev: 6.0.0
       hooks:
             - id: flake8
     - repo: https://github.com/pre-commit/mirrors-clang-format
-      rev: 'v14.0.6'  # Use the sha / tag you want to point at
+      rev: 'v15.0.7'  # Use the sha / tag you want to point at
       hooks:
         - id: clang-format
           files: \.(cu|cuh|h|cc|inl)$
@@ -25,5 +32,9 @@ repos:
           entry: python scripts/hooks/enforce_pytest_main.py
           language: python
           pass_filenames: false
+
+ci:
+  skip: [mypy]
+
 default_language_version:
     python: python3
diff --git a/BUILD.md b/BUILD.md
@@ -15,89 +15,36 @@ limitations under the License.
 
 -->
 
-# Overview
+# Basic build
 
-The build system is designed to enable two different modes of use:
-1. Simple `pip install` for users
-2. Highly customizable incremental builds for developers
+Users must have a working installation of the
+[Legate Core](https://github.com/nv-legate/legate.core)
+library prior to installing cuNumeric. **Installing cuNumeric by itself will not
+automatically install Legate Core.**
 
-We review each of these modes with examples.
+As for other dependencies, the Dependencies section on the
+[Legate Core build instructions](https://github.com/nv-legate/legate.core/blob/HEAD/BUILD.md)
+also covers cuNumeric, so no additional packages are required.
 
+Once Legate Core is installed, you can simply invoke `./install.py` from the
+cuNumeric top-level directory. The build will automatically pick up the
+configuration used when building Legate Core (e.g. the CUDA Toolkit directory).
 
-# Building for Users
+# Advanced topics
 
-## Using install.py
+## Building through pip & cmake
 
-For releases <= 22.07, the main method for building cuNumeric was the `install.py` script.
-Although the underlying implementation has significantly changed, `install.py` still supports the
-same usage and same set of flags. For a full list of flags, users can run:
+cuNumeric uses the same cmake/scikit-build-based build workflow as Legate Core.
+See the
+[Legate Core build instructions](https://github.com/nv-legate/legate.core/blob/HEAD/BUILD.md)
+for an overview.
 
-```
-$ ./install.py --help
-```
-
-## Using Conda
-
-cuNumeric can be installed using Conda by pointing to the required channels (`-c`):
-
-```
-conda install -c nvidia -c conda-forge -c legate legate-core
-```
-
-## Using pip
-
-cuNumeric is not yet registered in a standard pip repository. However, users can still use the 
-pip installer to build and install cuNumeric. After downloading or cloning the cunumeric source,
-users can run the following in the cunumeric folder:
-
-```
-$ pip install .
-```
-or
-```
-$ python3 -m pip install .
-```
-
-This will install cuNumeric in the standard packages directory for the environment Python.
-Note: This is currently not sufficient for running cuNumeric programs. cuNumeric relies
-on the `legate` launcher from Legate core, which must be installed separately.
-For details on installing Legate, consult the [Legate repository](https://github.com/nv-legate/legate.core).
-
-### Advanced Customization
+There are several examples in the `scripts` folder. We walk through the steps in
+`build-with-legate-separately-no-install.sh` here.
 
-If users need to customize details of the underlying CMake build, they can pass
-CMake flags through the `SKBUILD_CONFIGURE_OPTIONS` environment variable:
+We assume a pre-existing Legate Core build. For details on building Legate Core,
+consult the [Legate Core repository](https://github.com/nv-legate/legate.core).
 
-```
-$ SKBUILD_CONFIGURE_OPTIONS="-D Legion_USE_CUDA:BOOL=ON" \
-  pip install .
-```
-An alternative syntax using `setup.py` with `scikit-build` is
-```
-$ python setup.py install -- -DLegion_USE_CUDA:BOOL=ON
-```
-
-# Building for Developers
-
-## Overview
-
-pip uses [scikit-build](https://scikit-build.readthedocs.io/en/latest/)
-in `setup.py` to drive the build and installation.  A `pip install` will trigger three general actions:
-
-1. CMake build and installation of C++ libraries
-2. CMake generation of configuration files and build-dependent Python files
-3. pip installation of Python files
-
-The CMake build can be configured independently of `pip`, allowing incremental C++ builds directly through CMake.
-This simplifies rebuilding `libcunumeric.so` either via command-line or via IDE.
-After building the C++ libraries, the `pip install` can be done in "editable" mode using the `-e` flag.
-This configures the Python site packages to import the Python source tree directly.
-The Python source can then be edited and used directly for testing without requiring a `pip install`.
-
-## Example
-
-There are several examples in the `scripts` folder. We walk through the steps in the `build-with-legate-separately-no-install.sh` here.
-We assume a pre-existing Legate CUDA build. For details on building Legate, consult the [Legate repository](https://github.com/nv-legate/legate.core).
 First, the CMake build needs to be configured:
 
 ```
@@ -106,6 +53,7 @@ $ cmake -S . -B build -GNinja -D legate_core_ROOT:STRING=path/to/legate/build
 
 We point cuNumeric to the Legate *build* tree, not an installation.
 This generates all build-dependent headers and Python files.
+
 Once configured, we can build the C++ libraries:
 
 ```
@@ -118,14 +66,12 @@ Once the C++ libraries are available, we can do an editable (development) pip in
 ```
 $ SKBUILD_BUILD_OPTIONS="-D FIND_CUNUMERIC_CPP=ON -D cunumeric_ROOT=$(pwd)/build" \
   python3 -m pip install \
-  --root / --no-deps --no-build-isolation 
+  --root / --no-deps --no-build-isolation
   --editable .
 ```
 
-The Python source tree and CMake build tree are now available with the environment Python 
-for running cuNumeric programs. The diagram below illustrates the 
+The Python source tree and CMake build tree are now available with the environment Python
+for running cuNumeric programs. The diagram below illustrates the
 complete workflow for building both Legate core and cuNumeric.
 
 <img src="docs/figures/developer-build.png" alt="drawing" width="600"/>
-
-
diff --git a/CMakeLists.txt b/CMakeLists.txt
@@ -47,7 +47,7 @@ include(rapids-cuda)
 include(rapids-export)
 include(rapids-find)
 
-set(cunumeric_version 22.10.00)
+set(cunumeric_version 22.12.00)
 
 # For now we want the optimization flags to match on both normal make and cmake
 # builds so we override the cmake defaults here for release, this changes
@@ -79,24 +79,23 @@ endif()
 if(CMAKE_GENERATOR STREQUAL "Ninja")
   function(add_touch_cunumeric_ninja_build_target)
     set(_suf )
-    set(_depends )
     if(SKBUILD)
       set(_suf "_python")
     endif()
+    add_custom_target("touch_cunumeric${_suf}_ninja_build" ALL
+      COMMAND ${CMAKE_COMMAND} -E touch_nocreate "${CMAKE_CURRENT_BINARY_DIR}/build.ninja"
+      COMMENT "touch build.ninja so ninja doesn't re-run CMake on rebuild"
+      VERBATIM
+    )
     foreach(_dep IN ITEMS cunumeric cunumeric_python
                           legion_core legion_core_python
                           Legion LegionRuntime
                           Realm RealmRuntime
                           Regent)
       if(TARGET ${_dep})
-        list(APPEND _depends ${_dep})
+        add_dependencies("touch_cunumeric${_suf}_ninja_build" ${_dep})
       endif()
     endforeach()
-    add_custom_target("touch_cunumeric${_suf}_ninja_build" ALL
-      COMMAND ${CMAKE_COMMAND} -E touch_nocreate "${CMAKE_CURRENT_BINARY_DIR}/build.ninja"
-      COMMENT "touch build.ninja so ninja doesn't re-run CMake on rebuild"
-      VERBATIM DEPENDS ${_depends}
-    )
   endfunction()
   add_touch_cunumeric_ninja_build_target()
 endif()
diff --git a/README.md b/README.md
@@ -35,94 +35,18 @@ canonical NumPy implementation.
 
 If you have questions, please contact us at legate(at)nvidia.com.
 
-1. [Installation](#installation)
-1. [Dependencies](#dependencies)
-1. [Building from Source](#building-from-source)
-1. [Usage and Execution](#usage-and-execution)
-1. [Supported and Planned Features](#supported-and-planned-features)
-1. [Supported Types and Dimensions](#supported-types-and-dimensions)
-1. [Documentation](#documentation)
-1. [Future Directions](#future-directions)
-1. [Contributing](#contributing)
-1. [Known Bugs](#known-bugs)
-
 ## Installation
 
 cuNumeric is available [on conda](https://anaconda.org/legate/cunumeric):
 
 ```
 conda install -c nvidia -c conda-forge -c legate cunumeric
 ```
+
 The conda package is compatible with CUDA >= 11.4 (CUDA driver version >= r470),
 and Volta or later GPU architectures.
 
-Docker image build scripts, as well as specialized install scripts for
-supported clusters are available on the
-[quickstart](https://github.com/nv-legate/quickstart) repo.
-
-Read on for general instructions on building cuNumeric from source.
-
-## Dependencies
-
-Users must have a working installation of the
-[Legate Core](https://github.com/nv-legate/legate.core)
-library prior to installing cuNumeric.
-
-cuNumeric requires the following:
-
-  - Python >= 3.8
-  - [CUDA](https://developer.nvidia.com/cuda-downloads) >= 10.2
-  - GNU Make
-  - C++17 compatible compiler (g++, clang, or nvc++)
-  - Fortran compiler (for building OpenBLAS; not necessary if you provide a pre-built version of OpenBLAS)
-  - the Python packages listed in any one of the conda environment files:
-    - `conda/environment-test-3.8.yml`
-    - `conda/environment-test-3.9.yml`
-    - `conda/environment-test-3.10.yml`
-
-See the [corresponding section](https://github.com/nv-legate/legate.core#dependencies)
-on the Legate Core instructions for help on installing the required Python packages
-using conda.
-
-cuNumeric is tested and guaranteed to be compatible with Volta and later GPU
-architectures. You can use cuNumeric with Pascal GPUs as well, but there could
-be issues due to lack of independent thread scheduling. Please report any such
-issues on GitHub.
-
-## Building from Source
-
-Installation can be done the `install.py` script.
-For releases >= 22.10, `pip install` is now available.
-The most common installation command is:
-
-```
-./install.py --with-core <path-to-legate-core-installation>
-```
-
-This will build cuNumeric against the Legate Core installation and then
-install cuNumeric into the same location.
-
-If Legate Core has been installed with CUDA support, a working cuTENSOR
-installation must also be provided to the installation command with the
-`--with-cutensor` option:
-```
-./install.py --with-core <path-to-legate-core-installation> --with-cutensor <path-to-cutensor-installation>
-```
-
-You can also specify an installation of [OpenBLAS](https://www.openblas.net/)
-to use for the build. If you already have an installation of OpenBLAS on your
-machine, you can inform the installation script using the `--with-openblas`
-option:
-
-```
-./install.py --with-openblas <path-to-OpenBLAS>
-```
-
-Advanced users can also invoke `install.py --help` to see options for
-configuring cuNumeric by invoking the `install.py` script directly.
-More information on building - including development workflows - can be found
-in the [build instructions](BUILD.md)
-
+See [BUILD.md](BUILD.md) for instructions on building cuNumeric from source.
 
 ## Usage and Execution