Skip to content

Virtual Object Layer

Hyo-Kyung Lee edited this page Dec 18, 2023 · 2 revisions

List of VOL Connectors Registered with The HDF Group

Connector Connector Identifier Search Name* Short Description URL Contacts
Asynchronous I/O 512 async Provides support for asynchronous operations to HDF5 https://github.com/hpc-io/vol-async Suren Byna (sbyna at lbl dot gov)
Cache 513 cache Provides support for multi-level, multi-location data caching to dataset I/O operations https://github.com/hpc-io/vol-cache Suren Byna (sbyna at lbl dot gov)
Log-based 514 LOG The log-based VOL plugin stores HDF5 datasets in a log-based storage layout.
In this layout, data of multiple write requests made by an MPI process are appended one after another in the file. Such I/O strategy can avoid the expensive inter-process communication and I/O serialization due to file lock contentions when storing data in the canonical order. Through the log-based VOL, existing HDF5 programs can achieve a better parallel write performance with minimal changes to their codes.
https://github.com/DataLib-ECP/vol-log-based/blob/master/README.md Kai Yuan Hou
(khl7265 at ece dot northwestern dot edu)
DAOS 4004 daos Designed to utilize the DAOS object storage system by use of the DAOS API
https://doi.org/10.1109/TPDS.2021.3097884
https://github.com/HDFGroup/vol-daos
HDF5 DAOS VOL Connector Design
HDF5 DAOS VOL Connector User's Guide
help at hdfgroup dot org
native 0 native help at hdfgroup dot org
pass-through 517 pass_through_ext Provides a simple example of a pass-through VOL connector https://github.com/hpc-io/vol-external-passthrough Suren Byna (sbyna at lbl dot gov)
dset-split 518 dset-split Creates separate sub files for each dataset created and mounts these sub-files as external links in the main file. It enables versioning of HDF5 files at a dataset boundary. https://github.com/hpc-io/vol-dset-split Annmary Justine (annmary dot roy at hpe dot com)
PDC-VOL 519 PDC-VOL It is a terminal VOL that reads and writes HDF5 objects to the PDC system https://github.com/hpc-io/pdc https://github.com/hpc-io/vol-pdc Houjun Tang (htang4 at lbl dot gov)
REST 520 REST Designed to utilize web-based storage systems by use of the HDF5 REST APIs https://github.com/HDFGroup/vol-rest Matthew Larson (mlarson at hdfgroup dot org)
LowFive 521 LowFive A new data transport layer based on the HDF5 data model, for in situ workflows. Executables using LowFive can communicate in situ (using in-memory data and MPI message passing), reading and writing traditional HDF5 files to physical storage, and combining the two modes. https://github.com/diatomic/LowFive Tom Peterka (tpeterka at mcs dot anl dot gov)
Dmitriy Morozov (dmorozov at lbl dot gov)

*The Search Name provides a mechanism for searching for a VOL.

List of Prototype VOL Connectors

Connector Connector Identifier Search Name* Short Description URL Contacts
rados unassigned rados Prototype VOL connector to access data in RADOS https://github.com/HDFGroup/vol-rados help at hdfgroup dot org

*The Search Name provides a mechanism for searching for a VOL.

Clone this wiki locally