unnecessary breakage with "HDF5" v1.12

[conradsnicta]

@steven-varga Hi Steve, if you have the HDF group's ear (maybe @gheber?), perhaps you could kindly inform them that HDF5 v1.12 should probably be better called HDF6 at this stage?

The breaking API changes between HDF5 releases are rather disconcerting, even taking into the various brittle pre-processor tricks used as "workarounds". Stuff like H5_USE_110_API is in the end just a hack, papering over the unwarranted breakage.

The point of HDF is to be a boring storage abstraction, where by "boring" I mean stable. HDF5 is definitely not boring, which decreases its value and utility.

The use of semantic versioning to label HDF releases would help in this regard immensely. Minor releases such as v1.12 should not be introducing any kind of API breaks. Such breaks are better communicated by bumping the major version number, for example HDF6, HDF7, etc.

The current approach of API breakage between minor releases is causing all sorts of issues. Examples:

• https://gitlab.com/conradsnicta/armadillo-code/-/issues/155
• latest versions of armadillo (9.850.1_1) and hdf5 (1.12.0) available from Homebrew do not interact together correctly, result in runtime error Homebrew/homebrew-core#52003
• Alternate Homebrew HDF5 fix mlpack/mlpack#2328

TLDR: I would suggest that the HDF5 developers be made explicitly aware that public API breaks are not exactly conducive to productivity. It would be useful to properly communicate such changes via an increase in the major version number, for example HDF6.

[steven-varga]

@conradsnicta Hi Conrad, being a consumer for both the HDF5 and Armadillo C++ library I only can confirm the importance of your statement (and library). As an independent researcher I have no control of the HDF5 development cycle, and as of now h5cpp doesn't support the 1.12 API yet. -- OTOH h5cpp code base can keep up with the changes, and I put effort into supporting Armadillo C++. In the next few months, I will leverage the difference such that h5::read | h5::write will work with HDF5 v1.12, -- but probably it would be too much to ship h5cpp with Armadillo C++. (if you did, I could add CSV and your binary format...)

In my understanding HDF6 is less likely to happen, this document suggests that 5 within HDF5 is not a versioning but part of a trademarked name. Then again, I can't speak for The HDFGroup, as they are a different entity. Did you want to take your question to the hdf5 public forum?

best wishes: steven

[conradsnicta]

Thanks for the suggestion. Let's see how the HDF group responds.

• https://forum.hdfgroup.org/t/too-much-api-breakage-between-releases-mandatory-use-of-semantic-versioning/7200
• too much API breakage - mandatory use of semantic versioning HDFGroup/hdf5#2

[eudoxos]

Hi Steven, is the 1.12 support going to land in foreseeable time? I don't know how much work it entails for you, and not being that familiar with HDF5, I am just the BFU who says: it won't compile with 1.12. Thanks!

[steven-varga]

Thank you for the interest, there is a next generation of code base on my laptop waiting to be released, which takes C++ persistence to another level. In the past few weeks I am sleeplessly working on the matching H5CLUSTER, a rental supercomputing service, which happens to be a small but functional building block of the HDFGroup HUG-2020 event. In this event series I also will do a short presentation on the upcoming code base, and how H5CPP is making progress towards providing scalable persistence for C++ arbitrary objects.
Please join us, and feel free to ask questions on the event!

best wishes: steven

[steven-varga]

@eudoxos I had a talk with the FNAL last week, and provided a fix for v.1.12.0 by conditionally disabling the custom pipeline, which depends on custom DAPL; a feature that may be faulty in libhdf5 1.12.0 see: 8c3b5b0

in any event I checked H5CPP against the spack version of 1.12.0 with the following flags:
spack install [email protected]~cxx~debug~fortran~hl~java~szip~threadsafe+mpi+shared+pic api=v112 and it checked out for me.