-
Notifications
You must be signed in to change notification settings - Fork 45
/
Copy pathusing_h5import.txt
42 lines (33 loc) · 1.69 KB
/
using_h5import.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
Say you want to use h5import to import two binary data files, infile1.bin and
infile2.bin, into a single hdf5 file.
First it is necessary to create two configuration files (we'll call them
config1.txt and config2.txt) which tell h5import how to interpret the data and
where to put it. In this case let's assume the input files contain a sequence of
8-bit integer samples. We set PATH to tell the data where to live in the hdf5
file. We specify that the RANK is 1, since it is a single-dimensional dataset.
We set INPUT_CLASS to IN and INPUT-SIZE to 8 to specify the data type. Finally,
we need to specify DIMENSION-SIZES which gives the length of the array we are
importing. You can determine this from the file size: for example, if the size
of infile1 is exactly 1 MB, since each element is one byte the value of
DIMENSION-SIZES should be 1000000 (or if the file contained 4-byte floating
point samples but was still 1MB, DIMENSION-SIZES would be set to 250000).
So for our example, assuming both infile1.dat and infile2.dat are 1MB in size,
the contents of config1.txt would be:
PATH /data1
RANK 1
DIMENSION-SIZES 1000000
INPUT-CLASS IN
INPUT-SIZE 8
and the contents of config2.txt would be:
PATH /data2
RANK 1
DIMENSION-SIZES 1000000
INPUT-CLASS IN
INPUT-SIZE 8
then you can use the command line argument below to do the import:
h5import infile1.bin -c config1.txt infile2.bin -c config2.txt -o outfile.hdf5
Note that there are a bunch of additional settings on how to chunk the data in
memory but these are not really useful for a single dimensional dataset so I
usually ignore them.
The full documentation for the h5import tool can be found at
https://support.hdfgroup.org/HDF5/doc1.6/Tools.html#Tools-Import.