-
Notifications
You must be signed in to change notification settings - Fork 902
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add description of the cuIO GDS integration (#8293)
Adds a document to describe cuIO behavior with respect to the GDS library use. Also includes a disclaimer about the current state of the integration. Authors: - Vukasin Milovanovic (https://github.com/vuule) Approvers: - GALI PREM SAGAR (https://github.com/galipremsagar) - Keith Kraus (https://github.com/kkraus14) URL: #8293
- Loading branch information
Showing
2 changed files
with
24 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
GPUDirect Storage Integration | ||
============================= | ||
|
||
Many IO APIs can use GPUDirect Storage (GDS) library to optimize IO operations. | ||
GDS enables a direct data path for direct memory access (DMA) transfers between GPU memory and storage, which avoids a bounce buffer through the CPU. | ||
GDS also has a compatibility mode that allows the library to fall back to copying through a CPU bounce buffer. | ||
The SDK is available for download `here <https://developer.nvidia.com/gpudirect-storage>`_. | ||
|
||
Use of GPUDirect Storage in cuDF is disabled by default, and can be enabled through environment variable ``LIBCUDF_CUFILE_POLICY``. | ||
This variable also controls the GDS compatibility mode. There are two special values for the environment variable: | ||
|
||
- "GDS": Use of GDS is enabled; GDS compatibility mode is *off*. | ||
- "ALWAYS": Use of GDS is enabled; GDS compatibility mode is *on*. | ||
|
||
Any other value (or no value set) will keep the GDS disabled for use in cuDF and IO will be done using cuDF's CPU bounce buffers. | ||
|
||
This environment variable also affects how cuDF treats GDS errors. | ||
When ``LIBCUDF_CUFILE_POLICY`` is set to "GDS" and a GDS API call fails for any reason, cuDF falls back to the internal implementation with bounce buffers. | ||
When ``LIBCUDF_CUFILE_POLICY`` is set to "ALWAYS" and a GDS API call fails for any reason (unlikely, given that the compatibility mode is on), | ||
cuDF throws an exception to propagate the error to te user. | ||
|
||
NOTE: current GDS integration is not fully optimized and enabling GDS will not lead to performance improvements in all cases. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters