Skip to content

Commit

Permalink
Add description of the cuIO GDS integration (#8293)
Browse files Browse the repository at this point in the history
Adds a document to describe cuIO behavior with respect to the GDS library use.
Also includes a disclaimer about the current state of the integration.

Authors:
  - Vukasin Milovanovic (https://github.com/vuule)

Approvers:
  - GALI PREM SAGAR (https://github.com/galipremsagar)
  - Keith Kraus (https://github.com/kkraus14)

URL: #8293
  • Loading branch information
vuule authored May 24, 2021
1 parent b9588d1 commit 936b02d
Show file tree
Hide file tree
Showing 2 changed files with 24 additions and 1 deletion.
22 changes: 22 additions & 0 deletions docs/cudf/source/io-gds-integration.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
GPUDirect Storage Integration
=============================

Many IO APIs can use GPUDirect Storage (GDS) library to optimize IO operations.
GDS enables a direct data path for direct memory access (DMA) transfers between GPU memory and storage, which avoids a bounce buffer through the CPU.
GDS also has a compatibility mode that allows the library to fall back to copying through a CPU bounce buffer.
The SDK is available for download `here <https://developer.nvidia.com/gpudirect-storage>`_.

Use of GPUDirect Storage in cuDF is disabled by default, and can be enabled through environment variable ``LIBCUDF_CUFILE_POLICY``.
This variable also controls the GDS compatibility mode. There are two special values for the environment variable:

- "GDS": Use of GDS is enabled; GDS compatibility mode is *off*.
- "ALWAYS": Use of GDS is enabled; GDS compatibility mode is *on*.

Any other value (or no value set) will keep the GDS disabled for use in cuDF and IO will be done using cuDF's CPU bounce buffers.

This environment variable also affects how cuDF treats GDS errors.
When ``LIBCUDF_CUFILE_POLICY`` is set to "GDS" and a GDS API call fails for any reason, cuDF falls back to the internal implementation with bounce buffers.
When ``LIBCUDF_CUFILE_POLICY`` is set to "ALWAYS" and a GDS API call fails for any reason (unlikely, given that the compatibility mode is on),
cuDF throws an exception to propagate the error to te user.

NOTE: current GDS integration is not fully optimized and enabling GDS will not lead to performance improvements in all cases.
3 changes: 2 additions & 1 deletion docs/cudf/source/io.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,5 @@ This page contains Input / Output related APIs in cuDF.
:maxdepth: 2
:caption: Contents:

io-supported-types.rst
io-supported-types.rst
io-gds-integration.rst

0 comments on commit 936b02d

Please sign in to comment.