Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor EVE's exported CMake target and installation #1336

Merged
merged 17 commits into from
Jul 29, 2022
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/actions/run_docker/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# Container image that runs your code
FROM jfalcou/compilers:v3


Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's be pedantic for 0.5ns.
Can we remove that noisy commit? I guess this new line is accidental.

Copy link
Contributor Author

@justend29 justend29 Jul 26, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Being pedantic is good.

I actually placed that line there purposefully to change a Dockerfile, causing rebuilding of the docker image. I did this in an attempt to fix those mysterious CI errors. It didn't work.

I will clean up!

# Copies your code file from your action repository to the filesystem path `/` of the container
COPY entrypoint.sh /entrypoint.sh

Expand Down
14 changes: 9 additions & 5 deletions cmake/config/eve-install.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,10 @@
## Copyright : EVE Contributors & Maintainers
## SPDX-License-Identifier: MIT
##==================================================================================================
set(MAIN_DEST "lib")
set(INSTALL_DEST "include")
include(GNUInstallDirs)
set(MAIN_DEST "${CMAKE_INSTALL_LIBDIR}/eve-${PROJECT_VERSION}")
set(INSTALL_DEST "${CMAKE_INSTALL_INCLUDEDIR}/eve-${PROJECT_VERSION}")
set(DOC_DEST "${CMAKE_INSTALL_DOCDIR}-${PROJECT_VERSION}")

## =================================================================================================
## Exporting target for external use
Expand All @@ -15,13 +17,15 @@ target_include_directories( eve_lib INTERFACE
$<INSTALL_INTERFACE:${INSTALL_DEST}>
)
target_compile_features(eve_lib INTERFACE cxx_std_20)
set_target_properties(eve_lib PROPERTIES EXPORT_NAME eve)
add_library(eve::eve ALIAS eve_lib)

## =================================================================================================
## Install target with versionned folder
## Install target with versioned folder
## =================================================================================================
install(TARGETS eve_lib EXPORT eve_lib DESTINATION "${MAIN_DEST}")
install(TARGETS eve_lib EXPORT eve-targets DESTINATION "${MAIN_DEST}")
install(DIRECTORY ${PROJECT_SOURCE_DIR}/include/eve DESTINATION "${INSTALL_DEST}" )
install(FILES ${PROJECT_SOURCE_DIR}/cmake/eve-config.cmake DESTINATION "${MAIN_DEST}" )
install(FILES ${PROJECT_SOURCE_DIR}/cmake/eve-multiarch.cmake DESTINATION "${MAIN_DEST}" )
install(EXPORT eve_lib DESTINATION "${MAIN_DEST}")
install(FILES ${PROJECT_SOURCE_DIR}/LICENSE.md DESTINATION "${DOC_DEST}" )
install(EXPORT eve-targets NAMESPACE "eve::" DESTINATION "${MAIN_DEST}")
7 changes: 3 additions & 4 deletions cmake/eve-config.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@
##==================================================================================================
## Reuse install.cmake to preapre package properly
##==================================================================================================
get_filename_component(SELF_DIR "${CMAKE_CURRENT_LIST_FILE}" PATH)
include(${SELF_DIR}/eve_lib.cmake)
include(${SELF_DIR}/eve-multiarch.cmake)
set(EVE_LIBRARIES eve_lib)
include("${CMAKE_CURRENT_LIST_DIR}/eve-targets.cmake")
include("${CMAKE_CURRENT_LIST_DIR}/eve-multiarch.cmake")
set(EVE_LIBRARIES eve::eve)
98 changes: 61 additions & 37 deletions doc/page01_info.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,14 @@ Installation & Quick Start {#install}

In term of SIMD extension sets, we actively supports (ie code is optimized and regularly tested) the following:

- **Intel**
- SSE2, SSSE3, SSE3, SSE4.1, SSE4.2
- AVX, AVX2, FMA3
- AVX512 in SKylake-AVX512 mode (F, CD, VL, DQ, BW)
- **ARM**
- NEON A32 (64 & 128 bits)
- NEON A64 (64 & 128 bits)
- ASIMD
- **Intel**
- SSE2, SSSE3, SSE3, SSE4.1, SSE4.2
- AVX, AVX2, FMA3
- AVX512 in SKylake-AVX512 mode (F, CD, VL, DQ, BW)
- **ARM**
- NEON A32 (64 & 128 bits)
- NEON A64 (64 & 128 bits)
- ASIMD

The following instructions are tentatively supported (ie code is incomplete and not tested in depth):

Expand All @@ -32,17 +32,19 @@ The following instructions are tentatively supported (ie code is incomplete and
# Retrieving the source

## Github

**EVE** is available on GitHub and can be retrieved via the following command:
<br/>

@verbatim
$ git clone https://github.com/jfalcou/eve.git
$ git clone https://github.com/jfalcou/eve.git
@endverbatim

This retrieves the `main` branch which contains the latest stable version. Development happens
live on `main` so if you need stability, use a tagged versions.

## CPM

You can install **EVE** directly via [CPM](https://github.com/cpm-cmake/CPM.cmake). After
[adding CPM to your CMake setup](https://github.com/cpm-cmake/CPM.cmake#adding-cpm), just
add the following commands:
Expand All @@ -61,24 +63,27 @@ CPMAddPackage(
```

## SPACK

**EVE** is [**available**](https://spack.readthedocs.io/en/latest/package_list.html#eve
) via [**SPACK**](https://spack.readthedocs.io/en/latest/getting_started.html):

```bash
$ spack install eve
spack install eve
```

Once installed, you can use **EVE** headers directly from your
[**SPACK**](https://spack.readthedocs.io/en/latest/getting_started.html) installation.

## AUR

**EVE** is available on Arch-Linux via an [**AUR package**](https://aur.archlinux.org/packages/eve-git/).

```bash
$ yay -S eve-git
yay -S eve-git
```

## Conan

[**Conan Center**](https://conan.io/center/) hosts **EVE** as the
[`jfalcou-eve` package](https://conan.io/center/jfalcou-eve).

Expand All @@ -94,55 +99,56 @@ cmake
```

## VCPKG

**EVE** can be fetched from [VCPKG](https://vcpkgx.com/details.html?package=eve). Note that, as of
now, we still don't support MSVC.

```bash
$ vcpkg install eve
vcpkg install eve
```

# Installation from Source

If you didn't fetched **EVE** from a package manager, you'll need to install it via our CMake
system.

## Setting up the Library
Create a `build` directory here and enter it. Once in the `build` directory, you can use **CMake**
to generate the build system for **EVE**.

We recommend using [Ninja](https://ninja-build.org/) but any build system is fine.
With **CMake**, generate the build system for **EVE**. We recommend using
[Ninja](https://ninja-build.org/) but any build system is fine.

@verbatim
$ mkdir build
$ cd build
$ cmake .. -G Ninja
cmake -B build -G Ninja
@endverbatim

Once **CMake** completes, you can use the `install` target to build and install **EVE**. By default,
the library will be installed in the `/usr/local`directory, thus requiring root privileges on Linux.
Since **EVE** is header only, once **CMake** completes, you can install **EVE** without building. By
default, the library will be installed globally to your system, in `/usr/local` for Linux, thus
requiring root privileges.

@verbatim
$ ninja install
@endverbatim
```bash
cmake --install build
```

You can select an alternative installation path by specifying the `CMAKE_INSTALL_PREFIX` option at configuration time.
An alternative installation prefix can be specified with the '--prefix' flag.

@verbatim
$ cmake .. -G Ninja -DCMAKE_INSTALL_PREFIX=path/to/install
$ ninja install
@endverbatim
```bash
cmake --install build --prefix path/to/install
```

## Building the Documentation

You can rebuild **EVE** documentation if you have the latest version of Doxygen installed
using the `doxygen` target:
<br/>

@code
ninja doxygen
cmake --build build --target doxygen
@endcode

The resulting HTML files will be available in the `doc` folder.

# Using the library

## Compilation

Once installed, you can compile the following code to check if everything is alright.
Expand All @@ -152,7 +158,7 @@ Once installed, you can compile the following code to check if everything is alr
To do so, use your C++20 aware favorite compiler, for example g++.

@verbatim
$ g++ test.cpp -std=c++20 -march=native -O3 -DNDEBUG -I/path/to/install/include -o output
$ g++ test.cpp -std=c++20 -march=native -O3 -DNDEBUG -I/path/to/install/include/eve-<version> -o output
@endverbatim

Don't forget the `--std=c++20` option to be sure to activate C++20 support. If you use a different compiler, check your compiler user's manual to use the proper option.
Expand All @@ -168,7 +174,7 @@ The SIMD instruction sets that **EVE** uses is decided at compile time. You can
You can also select a specific instructions set by using the proper option(s) from your compiler. For example, let's compile for exactly SSE4.1.

@verbatim
$ g++ test.cpp -std=c++20 -msse4.1 -O3 -DNDEBUG -I/path/to/install/include -o output
$ g++ test.cpp -std=c++20 -msse4.1 -O3 -DNDEBUG -I/path/to/install/include/eve-<version> -o output
@endverbatim

## Execution
Expand All @@ -185,19 +191,37 @@ $ ./output

That's it, **EVE** is properly installed and ready to use.

## Use in CMake

Once installed, **EVE** may be consumed though its config-file **CMake** package. Simply find and
link against **EVE's** **CMake** target, as you would any other **CMake** library, and then
configure and build your **CMake** project.

```cmake
add_executable(use-eve main.cpp)
find_package(eve CONFIG REQUIRED)
target_link_libraries(use-eve PRIVATE eve::eve)
```

> If a custom installation prefix was used, ensure your **EVE** installation is within **CMake's**
search path with the use of the **CMake** variables **eve_ROOT** (CMake 3.12), **eve_DIR**, or
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we remove the 3.12 remark ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we are now saying 3.19 is the minimum, yes. Thanks!

**CMAKE_PREFIX_PATH**.

# Advanced options

If you want to dig a bit further, you can pass additional options to `cmake` to
activate additional features.

| Option | Effect | Target |
|--------|:-------------|:---------------------|
| `EVE_BUILD_TEST` | Enable unit tests for **EVE** (`ON` by default). | `ninja unit` |
| `EVE_BUILD_BENCHMARKS`| Enable benchmark tests for **EVE** (`OFF` by default). | `ninja benchmarks` |
| `EVE_BUILD_RANDOM` | Enable random tests for **EVE** (`OFF` by default). | `ninja random` |

There is currently over 2000 tests, so compiling all unit tests may require a large machine or
some time. We recommend compiling in parallel using `-j`.
| `EVE_BUILD_TEST` | Enable unit tests for **EVE** (`ON` by default). | `unit` |
| `EVE_BUILD_BENCHMARKS`| Enable benchmark tests for **EVE** (`OFF` by default). | `benchmarks` |
| `EVE_BUILD_RANDOM` | Enable random tests for **EVE** (`OFF` by default). | `random` |
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't we had a EVE_BUILD_INTEGRATION entry at some point or did I dreamt too deep ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that option was added as part of #1336 to enable the integration tests that are now registered with CTest. Thanks for catching my not updating this documentation section to match! I'll do that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added these docs and made an aggregated integration target to run the integration tests. This target was really just to have a target to put in the table, though ;)


There is currently over 2000 tests, so compiling all unit tests may require a large machine or some
time. We recommend compiling in parallel using `-j`.
All available **CMake** targets may be listed via `cmake --build build --target help`, each of which
may be built individually.

Some options are also available to control some other aspects

Expand Down
63 changes: 34 additions & 29 deletions doc/tutorial/mutli_arch.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,13 @@
# The Issue
The **EVE** library uses compile-time code generation process to ensure a high-level of performance.
However, there can be many reasons why your application or library might run on unknown systems.
Thus, you may want to be able to write your SIMD kernel once and yet be able to adapt to your end-user's
hardware.
Thus, you may want to be able to write your SIMD kernel once and yet be able to adapt to your
end-user's hardware.

A first solution could be to just recompile a given kernel with different architecture settings and
link everything. Then, at runtime, use eve::is_supported to choose the correct implementation
between the available ones. However, in attempting to solve the problem in this way, you'll face issues as the
core of **EVE** is based on templated callable object, you may end up violating the
between the available ones. However, in attempting to solve the problem in this way, you'll face
issues as the core of **EVE** is based on templated callable object, you may end up violating the
[One Definition Rule](https://en.cppreference.com/w/cpp/language/definition) and end up with a
binary containing the incorrect symbols and implementations.

Expand Down Expand Up @@ -45,7 +45,9 @@ The code breaks down this way:
Nothing special is required except for the `extern "C"` attribute. If we want to get around this
limitation and have a function taking arbitrary C++ types as parameters, there are different
strategies. One such strategy is to use the mangled name to export a function returning an array or
[structure containing all the pre-computed functions pointers](http://community.qnx.com/sf/docman/do/downloadDocument/projects.toolchain/docman.root.articles/doc1154) from the `.so` library.
[structure containing all the pre-computed functions
pointers](http://community.qnx.com/sf/docman/do/downloadDocument/projects.toolchain/docman.root.articles/doc1154)
from the `.so` library.

## Writing the dynamic function hub
We now need to write an actual `compute` function that our users will call. This is the place where
Expand All @@ -63,14 +65,15 @@ What is going on inside `compute` ?
runtime via eve::is_supported. Notice the names of the .so files, we will go back to those
afterward but understand that they are arbitrary. To do so, we use *Immediately Invoked Lambda
Expressions* to perform this initialisation.
+ Next, we load the `compute_kernel` symbol from the handle we just loaded. As opening and fetching
pointer to symbol can be costly, we will make it so those tasks are done once by storing both
results in static variables.
+ Once everything is loaded and setup, we call the function pointer with the appropriate parameters.
+ Next, we load the `compute_kernel` symbol from the handle we just loaded. As opening and
fetching pointer to symbol can be costly, we will make it so those tasks are done once by
storing both results in static variables.
+ Once everything is loaded and setup, we call the function pointer with the appropriate
parameters.

Obviously, in realistic settings you would actually care about runtime issues, check that every pointer
are non-null, and use functions like `dlerror` to find out what would have caused an issue. This is
left as an exercice for the reader.
Obviously, in realistic settings you would actually care about runtime issues, check that every
pointer are non-null, and use functions like `dlerror` to find out what would have caused an issue.
This is left as an exercice for the reader.

## Compiling and Using Dynamic Kernels
The next part is much more down-to-earth. We need to compile the kernel function inside multiple
Expand All @@ -97,20 +100,21 @@ suffix. For each suffixes, the corresponding options will be used from [OPTIONS]
Optionally, you can pass an `INTERFACE` name to be used for compiling each of the targets.
Optionally, the `QUIET` flag will remove error messages.

Once called, `eve_build_variants`sets up a variable in the current scope named `eve_${NAME}_variants`
that contains the list of target building the shared libraries. It can be used to setup dependencies.
Once called, `eve_build_variants`sets up a variable in the current scope named
`eve_${NAME}_variants` that contains the list of target building the shared libraries. It can be
used to setup dependencies.

Let's say we want to compile our kernel for SSE2, SSE4.1 and AVX2.
The CMake code will be looking like:

```cmake
cmake_minimum_required(VERSION 3.19)
cmake_minimum_required(VERSION 3.18)
project(eve-multi-arch LANGUAGES CXX)

find_package(eve)
find_package(eve CONFIG REQIURED)

add_library(setup INTERFACE)
target_link_libraries(setup INTERFACE ${EVE_LIBRARIES})
target_link_libraries(setup INTERFACE eve::eve)
target_compile_options(setup INTERFACE "-O3")
justend29 marked this conversation as resolved.
Show resolved Hide resolved

eve_build_variants( NAME compute
Expand All @@ -130,18 +134,19 @@ Let's get into the details:
+ After the classical CMake basics, we use `find_package` to grab **EVE**. Depending on your
**EVE** installation path, you may have to specify `CMAKE_PREFIX_PATH` for CMake to find it.
+ We define an INTERFACE library that use the main **EVE** interface and add some customisation.
+ We call `eve_build_variants` to build a set of targets. The base name will be `compute` and there
are three different target suffixes: `basic`, `advanced`, `perfect`. For each of those, the following
options will be used: `-msse2`, `-msse4.1`, `-march=haswell`. Each target is compiled with its corresponding
options. In the end, we expect three libraries to be compiled: `libcompute_basic.so`, `libcompute_advanced.so`,
and `libcompute_perfect.so`. Notice how the suffixes are arbitrary. They just need to correspond
to any naming scheme you see fit and those names will have to be used in the dynamic loading function.
+ We call `eve_build_variants` to build a set of targets. The base name will be `compute` and
there are three different target suffixes: `basic`, `advanced`, `perfect`. For each of those,
the following options will be used: `-msse2`, `-msse4.1`, `-march=haswell`. Each target is
compiled with its corresponding options. In the end, we expect three libraries to be compiled:
`libcompute_basic.so`, `libcompute_advanced.so`, and `libcompute_perfect.so`. Notice how the
suffixes are arbitrary. They just need to correspond to any naming scheme you see fit and those
names will have to be used in the dynamic loading function.
+ To perform this compilation, we use the `setup` interface we customized earlier.
+ We add our executable target that just compiles a main file and the compute function. To be sure
we don't forget to compile the libraries when we compile the main executable, we use the exported
`eve_compute_variants` macro that contains the list of targets created by `eve_build_variants`.
Because we use `dlopen`, we need to link with the appropriate library which is conveniently
provided by `CMAKE_DL_LIBS`.
we don't forget to compile the libraries when we compile the main executable, we use the
exported `eve_compute_variants` macro that contains the list of targets created by
`eve_build_variants`. Because we use `dlopen`, we need to link with the appropriate library
which is conveniently provided by `CMAKE_DL_LIBS`.

That's all. Once generated, this CMake file will let you compile the `multi-arch` target that will
trigger the compilation of the three libraries. Once compiled, the execution of the multi-arch
Expand All @@ -160,7 +165,7 @@ The complete project is available as in the `examples/multi-arch` folder. As an
try to modify the code to handle AVX512 and check everything still works.

# Conclusion
Handling multiple architecture within a single application is not trivial. It requires some scaffolding,
much of those being provided by **EVE** itself at both the CMake and C++ level.
Handling multiple architecture within a single application is not trivial. It requires some
scaffolding, much of those being provided by **EVE** itself at both the CMake and C++ level.

**/
6 changes: 2 additions & 4 deletions examples/multi-arch/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,10 @@
cmake_minimum_required(VERSION 3.18)
project(eve-multi-arch LANGUAGES CXX)

set(CMAKE_PREFIX_PATH "/usr/local/lib")
find_package(eve)
find_package(eve CONFIG REQUIRED)

add_library(setup INTERFACE)
target_link_libraries(setup INTERFACE ${EVE_LIBRARIES})
target_link_libraries(setup INTERFACE eve::eve)
target_compile_options(setup INTERFACE "-O3")

eve_build_variants( NAME compute
Expand All @@ -23,4 +22,3 @@ eve_build_variants( NAME compute
add_executable(multi-arch main.cpp compute.cpp)
add_dependencies(multi-arch ${eve_compute_variants})
target_link_libraries(multi-arch PUBLIC setup ${CMAKE_DL_LIBS})

Loading