Ext: MeshRefinement

[ax3l]

Adds a simple convention for mesh refinement.

Description

This adds a naming convention to efficiently identify levels of a refined mesh.

Affected Components

• base (cosmetic)
• EXT: MeshRefinement

Logic Changes

Adds a naming convention through record suffixes for meshes that can be used to identify refined meshes.

Allows two ways to store a refined mesh: through one record per refinement level (e.g. for ADIOS) or through one record per refinement level and patch (for non-contiguous refinements).

Writer Changes

A writer can suffix the meshes to express levels.

• openPMD-api: https://github.com/openPMD/openPMD-api
• WarpX: openPMD: Support for mesh refinement ECP-WarpX/WarpX#1842

Reader Changes

A reader can group mesh-refined records visually or build an MR structure.

Good candidates:

• yt: https://github.com/yt-project/yt
• ParaView: ...

Unaffected for now:

• openPMD-validator: https://github.com/openPMD/openPMD-validator
• openPMD-viewer: https://github.com/openPMD/openPMD-viewer
• VisIt: https://github.com/openPMD/openPMD-visit-plugin
• converter: https://github.com/openPMD/openPMD-converter
• openPMD-api: https://github.com/openPMD/openPMD-api

Data Updater

New functionality. No data update needed.

[franzpoeschel]

I see two major issues with the current state of mesh refinement:

1. The use of suffixes _lvl<N> and _<P> seems inconsistent in an otherwise hierarchical data standard, as it reimplements hierarchies at the naming level. Should we consider developing this in conjunction with Allow user to store non-openPMD information #115 (custom hierarchies)?
2. The feature description in the standard should be more explicit on the required capabilities of the storage formats and maybe even reconsider them. Without mesh refinement, openPMD requires (roughly) a storage format that supports attributes, hierarchies and n-dimensional datasets. With mesh refinement, the backend is additionally required to allow querying written regions (in the API, the information returned by .available_chunks() is purely backend-specific and completely transparent to the actual openPMD standard).
   In my understanding, HDF5 chunking allows storing a sparsely written dataset without using disk space corresponding with the entire domain (rather, it will use disk space corresponding with the covered chunks). Explicitly encoding the written chunks in either the standard or the HDF5 schema would then enable mesh refinement in HDF5, too.
   To me, this seems like too small a detail for making mesh-refinement ADIOS-only over it.

[ax3l]

Potential directions for HDF5 support of MR in openPMD:

• Add support for sparse data and enhance support for variable-length data in HDF5 HDFGroup/hdf5#3257
• https://github.com/LifeboatLLC/SparseHDF5

[ursg]

This is clearly motivated specifically with a patch-based refinement strategy in mind, and as such probably works very well.

However, I would like to point out that this structure would seem quite alien to other refined mesh approaches (tree-based or individuall-cell-based ones). Deliberately grouping mesh data by refinement level does not come natural to them!

Sorry about this nonproductive comment, as I do not have a suggestion for a suitable, more general approach. There certainly is merit in the proposal as it is specified here, but you might want to adapt a more specific name ("PatchBasedMeshRefinement"), as to not encumber future applications with different approaches.