Reading/writing an array of strings

[pbartholomew08]

I am trying to read an array of strings from an HDF5 file. I've tried both fixed/variable size strings using the following Python code

import h5py
import numpy as np

a = ["left", "right", "bottom", "top"]
f = h5py.File("test.h5", "w")
f.create_dataset("foo", data=np.array(a, dtype="S6")) # Drop dtype for variable strings
f.close()

Using h5dump indicates this file is valid

> h5dump test.h5 
HDF5 "test.h5" {
GROUP "/" {
   DATASET "foo" {
      DATATYPE  H5T_STRING {
         STRSIZE 6;
         STRPAD H5T_STR_NULLPAD;
         CSET H5T_CSET_ASCII;
         CTYPE H5T_C_S1;
      }
      DATASPACE  SIMPLE { ( 4 ) / ( 4 ) }
      DATA {
      (0): "left\000\000", "right\000", "bottom", "top\000\000\000"
      }
   }
}
}

however bpls rejects the variable

> $ADIOS2/bin/bpls test.h5 
WARNING: IO is not accepting definition of variable: /foo. Skipping.

I was going to try writing the file instead from ADIOS2/Fortran so I could inspect and compare the output from h5dump, however when I try to use the adios2_type_string_array which I think I need it is unavailable to the compiler. Additionally the generic resolution fails for adios2_put passing a variable of type

character(len=8), dimension(4) :: strings

I am using adios v2.8.3, however I think the adios2_type_string_array is also not available in the latest Fortran bindings when I searched on Github.

[guj]

Thank you for the feedback!

In general, it would be good to use the most recent release, in case features or issues are already covered.
The most recent ADIOS release is 2.9.

Regarding to array of strings, ADIOS variable supports a scalar string, but not an array of strings like HDF5 does. (https://adios2.readthedocs.io/en/latest/components/components.html#variable)

ADIOS attributes, however, can take in an array of strings.
So, please consider storing the array of string "a" in your example as an attribute instead.

It looks like h5py uses variable length string instead of a basic string. I will add a pull request in ADIOS to read in variable length string from HDF5.

I have not used the fortran interface for awhile. Will give it a try tomorrow and let you know.

[guj]

Some update:

1. A pull request is submitted that will read in HDF5 variable strings correctly added support to read back from H5T_STRING VARIABLES #4060
2. The current fortran module does not have adios2_type_string_array anymore (I didn't track at which version it is dropped). So looks like we need to update the document ( https://adios2.readthedocs.io/en/latest/api_full/api_full.html @pnorbert)
3. one can write out string array through attributes with fortran as well.
4. you can also use ADIOS python api to write h5 files.

Happy to answer questions.