forked from HDFGroup/hdf5
-
Notifications
You must be signed in to change notification settings - Fork 0
Virtual Object Layer
Hyo-Kyung Lee edited this page Dec 18, 2023
·
2 revisions
Connector | Connector Identifier | Search Name* | Short Description | URL | Contacts |
---|---|---|---|---|---|
Asynchronous I/O | 512 | async | Provides support for asynchronous operations to HDF5 | https://github.com/hpc-io/vol-async | Suren Byna (sbyna at lbl dot gov) |
Cache | 513 | cache | Provides support for multi-level, multi-location data caching to dataset I/O operations | https://github.com/hpc-io/vol-cache | Suren Byna (sbyna at lbl dot gov) |
Log-based | 514 | LOG | The log-based VOL plugin stores HDF5 datasets in a log-based storage layout. In this layout, data of multiple write requests made by an MPI process are appended one after another in the file. Such I/O strategy can avoid the expensive inter-process communication and I/O serialization due to file lock contentions when storing data in the canonical order. Through the log-based VOL, existing HDF5 programs can achieve a better parallel write performance with minimal changes to their codes. |
https://github.com/DataLib-ECP/vol-log-based/blob/master/README.md | Kai Yuan Hou (khl7265 at ece dot northwestern dot edu) |
DAOS | 4004 | daos | Designed to utilize the DAOS object storage system by use of the DAOS API https://doi.org/10.1109/TPDS.2021.3097884 |
https://github.com/HDFGroup/vol-daos HDF5 DAOS VOL Connector Design HDF5 DAOS VOL Connector User's Guide |
help at hdfgroup dot org |
native | 0 | native | help at hdfgroup dot org | ||
pass-through | 517 | pass_through_ext | Provides a simple example of a pass-through VOL connector | https://github.com/hpc-io/vol-external-passthrough | Suren Byna (sbyna at lbl dot gov) |
dset-split | 518 | dset-split | Creates separate sub files for each dataset created and mounts these sub-files as external links in the main file. It enables versioning of HDF5 files at a dataset boundary. | https://github.com/hpc-io/vol-dset-split | Annmary Justine (annmary dot roy at hpe dot com) |
PDC-VOL | 519 | PDC-VOL | It is a terminal VOL that reads and writes HDF5 objects to the PDC system | https://github.com/hpc-io/pdc https://github.com/hpc-io/vol-pdc | Houjun Tang (htang4 at lbl dot gov) |
REST | 520 | REST | Designed to utilize web-based storage systems by use of the HDF5 REST APIs | https://github.com/HDFGroup/vol-rest | Matthew Larson (mlarson at hdfgroup dot org) |
LowFive | 521 | LowFive | A new data transport layer based on the HDF5 data model, for in situ workflows. Executables using LowFive can communicate in situ (using in-memory data and MPI message passing), reading and writing traditional HDF5 files to physical storage, and combining the two modes. | https://github.com/diatomic/LowFive | Tom Peterka (tpeterka at mcs dot anl dot gov) Dmitriy Morozov (dmorozov at lbl dot gov) |
*The Search Name provides a mechanism for searching for a VOL.
Connector | Connector Identifier | Search Name* | Short Description | URL | Contacts |
---|---|---|---|---|---|
rados | unassigned | rados | Prototype VOL connector to access data in RADOS | https://github.com/HDFGroup/vol-rados | help at hdfgroup dot org |
*The Search Name provides a mechanism for searching for a VOL.