Skip to content

Commit

Permalink
Merge PRs #143, 144 to hdf5_1_12 (#630)
Browse files Browse the repository at this point in the history
* Snapshot version 1.12 release 1-3.  Update  version to 1.12.1-4.

* First cut of the H5 public API documentation. (#80)

* First cut of the H5 public API documentation.

* Added H5Z "bonus track."

* Applied Quincey's patch.

* Added the missing patches from Quincey's original patch.

* H5PL (complete) and basic H5VL API documentation.

* Added H5I API docs.

* Added H5L API docs.

* First installment from Elena's H5T batch.

* Second installment of Elena's H5T batch.

* Final installment of Elena's H5T batch.

* Full set of current H5F documentation. (#105)

* First cut of the H5 public API documentation.

* Added H5Z "bonus track."

* Applied Quincey's patch.

* Added the missing patches from Quincey's original patch.

* H5PL (complete) and basic H5VL API documentation.

* Added H5I API docs.

* Added H5L API docs.

* First installment from Elena's H5T batch.

* Second installment of Elena's H5T batch.

* Final installment of Elena's H5T batch.

* Migrated documentation for SWMR functions.

* Catching up on MDC functions.

* Integrated the H5F MDC function documentation.

* Added MDC and parallel H5F functions.

* Slightly updated main page.

* Added doxygen/dox/H5AC_cache_config_t.dox to MANIFEST.

* Doxygen - added (mostly) beginner functions (#112)

* Doxygen - added (mostly) beginner functions

* Removed duplicate H5Pset_szip function

* Add src/H5module.h to MANIFEST.

* close #195. (#196)

* Update HDF5PluginMacros.cmake

* Update HDF5PluginMacros.cmake

* Avoid aligned access for references by decoding into temporary buffer and then copying the result into the actual buffer.   Update test to be more thorough with using compound datatype fields everywhere. (#206)

* Modify temporary rpath for testing in java example scripts. (#230)

* Fix undefined left shifting of negative numbers (#338)

Undefined Bahavior Sanitizer errored here about left shifting negative numbers.

* Fixes various warnings noticed on Windows (#425)

* Fixes various warnings noticed on Windows

- Adds a prototype for our implementation of vasprintf
- Return type of H5_get_utf16_str() is now non-const
- Fixes possible uninitialized return type in Wremove_utf8
- Better isolation of fork() code in accum.c:test_swmr_write_big()
- Better isolation of non-zlib code in dsets.c:test_filter_delete()
- Removed unused variable in trefer.c:test_reference_cmpnd_obj()

* Fixes clang-format issues

* Applied clang-tidy readability-non-const-parameter warning fixes auto… (#429)

* Automatically applied clang-tidy readability-avoid-const-params-in-decls fixes

Removes useless const declarations.

* Fixed most readability-non-const-parameter warnings

These changes were made automatically by clang-tidy, but I manually reverted the changes related to the H5Z_func_t signature.

* Reformat source with clang v10.0.1.

Co-authored-by: Larry Knox <[email protected]>

* Added C++11 override keyword where appropriate (#433)

Added H5_OVERRIDE macro for compatibility with both C++11 and older.

* Various clang tidy warning fixes (#448)

* Fixed clang-tidy bugprone-reserved-identifier warnings

* Fixed clang-tidy bugprone-assert-side-effect warnings

* Fixed clang-tidy bugprone-copy-constructor-init warning

* Fixed clang-tidy readability-redundant-preprocessor warning

For error_test.c the removed code was already dead, because it was in the else of an `#if H5_USE_16_API` block.

Based on H5Location.h, I think p_get_ref_obj_type was meant to be in `#ifndef DOXYGEN_SHOULD_SKIP_THIS` and an `#endif` was missing.  Similarly, in the header, getObjTypeByIdx is only in H5_NO_DEPRECATED_SYMBOLS, not DOXYGEN_SHOULD_SKIP_THIS.

* Fixed clang-tidy readability-redundant-string-init warnings

* Fixed some clang-tidy performance-type-promotion-in-math-fn warnings

* Fixed clang-tidy performance-unnecessary-value-param warnings

* Reformat source with clang v10.0.1.

Co-authored-by: Larry Knox <[email protected]>

* Removed checks/workarounds for pre-C++89 compatibility (#449)

After 30+ years, just assume that the following exist:
- extension-less includes
- namespaces
- std::
- static_cast
- bool

* Fixed all clang-tidy bugprone-suspicious-string-compare warnings (#451)

* Fixed all clang-tidy bugprone-suspicious-string-compare warnings

This change was generated entirely by clang-tidy itself.

* Reformat code with clang v10.0.1.

Co-authored-by: Larry Knox <[email protected]>

* Remove 2 functions incorrectly merged from develop in a cherry-pick merge of PR #451.

* Minor parallel improvements (#519)

* Improve MPI error reporting, handled failed operations in parallel tests more nicely, and clean up MPI_Allreduce for determining whether to break collective I/O

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Clean up MPI-IO VFD tracing support (#520)

* Clean up tracing support

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Brings the native implementation of H5Fdelete() from Bitbucket (#524)

* Committing clang-format changes

* Brings the native VFD H5Fdelete() implementation from Bitbucket

Only brings the 'del' callbacks, not the 'open/close' scheme.

* Formatter changes

* Committing clang-format changes

* Fixes direct VFD callback name

* Removes UNUSED macro from family API call

* Adds barrier and rank 0 check to MPI-I/O VFD delete

* Revert "Adds barrier and rank 0 check to MPI-I/O VFD delete"

This reverts commit 909765f.

* Revert "Revert "Adds barrier and rank 0 check to MPI-I/O VFD delete""

This reverts commit 9b04bef.

* Adds a second barrier after the delete in MPI-I/O VFD

* Only delete files in the core VFD when the backing store flag is set

* Fixes string issues in multi VFD

Also, h5test.c cleanup code now uses H5Fdelete().

* Formatted source

* Rework fapl checks for MPI-I/O VFD delete callback

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Minor warning fixes in develop (#526)

* Committing clang-format changes

* Minor warning fixes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Removes implementation of my_strdup() from the multi VFD (#527)

* Committing clang-format changes

* Removes my_strdup() from the multi VFD

* Use strdup directly when memory sanity checks are off

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Removes dead H5ST package from the library (#528)

* Committing clang-format changes

* Removes the unused H5ST package from the library

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Fix HDFFV-11232 (#530)

* fixed missed closing of a dataset

* fixed missed closing of a dataset

* fixed typo in error return

* Committing clang-format changes

* minor edits

* code format

* Committing clang-format changes

* code format

* minor edit

* added H5fortkit dependency for H5VLff.F90, HDFFV-11232

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Fixes incorrect usage of H5I_BADID (#554)

* Committing clang-format changes

* Fixes incorrect use of H5I_BADID

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Fixes a segfault when H5Pset_mdc_log_options is called multiple times on a fapl (#601)

* Committing clang-format changes

* Fixes a segfault when H5Pset_mdc_log_options() is called multiple times

An internal string is incorrectly freed when the API call is invoked
multiple times on a property list, which will usually cause a segfault
to occur. On the first call the log location is NULL so the problem
doesn't occur.

Fixes HDFFV-11239

* Fixes typos

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Fix for a segfault when H5Pset_fapl_log is passed an invalid fapl ID (#607)

* Committing clang-format changes

* Fixes an issue where H5Pset_fapl_log sefaults when passed an invalid
fapl ID

This was due to a pointer-containing struct being memset after the first
internal API call. If the first call failed, the error condition would
check if the pointer was not NULL and then attempt to free it if not.
This would lead to the freeing of a wild pointer if an invalid fapl ID
were passed in.

This was fixed by reordering the memset and adding a test to ensure the
problem stays fixed.

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Fixes crashes when size_hint > UINT32_MAX is passed to H5Gcreate1 (#611)

* Committing clang-format changes

* Fixes incorrect size_hint handling in H5Gcreate1

* Updates the size hint type for group creation

* Updates the RELEASE.txt note

* Revert "Updates the RELEASE.txt note"

This reverts commit 3df386a.

* Reverts previous behavior to use a uint32_t struct field

* Updates RELEASE.txt

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Revert "Brings the native implementation of H5Fdelete() from Bitbucket (#524)"

This reverts commit 38d1b12.

* Removed mentions of Wdeclaration-after-statement now that C99 is requ… (#447)

* Removed mentions of Wdeclaration-after-statement now that C99 is required

* Remove -Werror=declaration-after-statement from error-general file.

Co-authored-by: Larry Knox <[email protected]>

* Made private my_yyinput function static (#618)

This prevents it being exported as a public symbol.

* Adds const to a few global variables (#623)

* Committing clang-format changes

* Adds consts to a few global variables

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* (fix) Segmentation fault when using a compound type. (#143)

* (fix) Segmentation fault when using a compound type.

In the case when a compounded attribute is written to dataset
followed by writing the data with a data transform function
to the dataset will result in a segmentation fault. It turns out
the data is classified as compounded while it is not. Now, the
state is always reset first to not compounded followed by the
existing check if the variable is compounded.

* (fix) Removed undesired comment lines.

* (fix) Segmentation fault when using a compound type: added test.

* (fix) Added the missing cmpd_transform.c file to MANIFEST.

* (fix) cmpd_dtransform test: autotools and source header.

Added the cmp_dtransform test to the autotools configuration and
updated the HDF Group copyright header.

Co-authored-by: Jan-Willem Blokland <[email protected]>

* (fix) H5Z_xform_create function and scientific notation (#144)

* (fix) H5Z_xform_create function and scientific notation

Implemented a more sophisticated check to support scientific notation
in the expression of the H5Zset_data_transform function.

* (fix) H5Z_xform_create and scientific notation: Added test.

Added a test to demonstrate that the parsing of expression
which includes scientific notation works correctly. Improved
inline comment.

Co-authored-by: Jan-Willem Blokland <[email protected]>

* Committing clang-format changes

Co-authored-by: Gerd Heber <[email protected]>
Co-authored-by: bljhdf <[email protected]>
Co-authored-by: H. Joe Lee <[email protected]>
Co-authored-by: Quincey Koziol <[email protected]>
Co-authored-by: Sean McBride <[email protected]>
Co-authored-by: Dana Robinson <[email protected]>
Co-authored-by: Quincey Koziol <[email protected]>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Scot Breitenfeld <[email protected]>
Co-authored-by: Jan-Willem Blokland <[email protected]>
Co-authored-by: Jan-Willem Blokland <[email protected]>
  • Loading branch information
12 people authored May 6, 2021
1 parent f573889 commit 6ff6504
Show file tree
Hide file tree
Showing 7 changed files with 189 additions and 14 deletions.
1 change: 1 addition & 0 deletions MANIFEST
Original file line number Diff line number Diff line change
Expand Up @@ -1139,6 +1139,7 @@
./test/cache_tagging.c
./test/chunk_info.c
./test/cmpd_dset.c
./test/cmpd_dtransform.c
./test/cork.c
./test/corrupt_stab_msg.h5
./test/cross_read.c
Expand Down
25 changes: 14 additions & 11 deletions src/H5T.c
Original file line number Diff line number Diff line change
Expand Up @@ -3408,8 +3408,8 @@ H5T__create(H5T_class_t type, size_t size)
* Note: Common code for both H5T_copy and H5T_copy_reopen, as part of
* the const-correct datatype copying routines.
*
* Programmer: David Young
* January 18, 2020
* Programmer: David Young
* January 18, 2020
*
*-------------------------------------------------------------------------
*/
Expand Down Expand Up @@ -3462,8 +3462,8 @@ H5T__initiate_copy(const H5T_t *old_dt)
* Return: Success: Pointer to a new copy of the OLD_DT argument.
* Failure: NULL
*
* Programmer: David Young
* January 18, 2020
* Programmer: David Young
* January 18, 2020
*
*-------------------------------------------------------------------------
*/
Expand All @@ -3490,8 +3490,8 @@ H5T__copy_transient(H5T_t *old_dt)
* Return: Success: Pointer to a new copy of the OLD_DT argument.
* Failure: NULL
*
* Programmer: David Young
* January 18, 2020
* Programmer: David Young
* January 18, 2020
*
*-------------------------------------------------------------------------
*/
Expand Down Expand Up @@ -3521,8 +3521,8 @@ H5T__copy_all(H5T_t *old_dt)
*
* Note: Common code for both H5T_copy and H5T_copy_reopen.
*
* Programmer: David Young
* January 18, 2020
* Programmer: David Young
* January 18, 2020
*
*-------------------------------------------------------------------------
*/
Expand Down Expand Up @@ -3794,8 +3794,8 @@ H5T_copy(const H5T_t *old_dt, H5T_copy_t method)
* Return: Success: Pointer to a new copy of the OLD_DT argument.
* Failure: NULL
*
* Programmer: David Young
* January 18, 2020
* Programmer: David Young
* January 18, 2020
*
*-------------------------------------------------------------------------
*/
Expand Down Expand Up @@ -5270,7 +5270,10 @@ H5T__path_find_real(const H5T_t *src, const H5T_t *dst, const char *name, H5T_co
} /* end else-if */

/* Set the flag to indicate both source and destination types are compound types
* for the optimization of data reading (in H5Dio.c). */
* for the optimization of data reading (in H5Dio.c).
* Make sure that path->are_compounds is only TRUE for compound types.
*/
path->are_compounds = FALSE;
if (H5T_COMPOUND == H5T_get_class(src, TRUE) && H5T_COMPOUND == H5T_get_class(dst, TRUE))
path->are_compounds = TRUE;

Expand Down
14 changes: 12 additions & 2 deletions src/H5Ztrans.c
Original file line number Diff line number Diff line change
Expand Up @@ -1538,10 +1538,20 @@ H5Z_xform_create(const char *expr)
"unable to allocate memory for data transform expression")

/* Find the number of times "x" is used in this equation, and allocate room for storing that many points
* A more sophisticated check is needed to support scientific notation.
*/
for (i = 0; i < HDstrlen(expr); i++)
if (HDisalpha(expr[i]))
for (i = 0; i < HDstrlen(expr); i++) {
if (HDisalpha(expr[i])) {
if ((i > 0) && (i < (HDstrlen(expr) - 1))) {
if (((expr[i] == 'E') || (expr[i] == 'e')) &&
(HDisdigit(expr[i - 1]) || (expr[i - 1] == '.')) &&
(HDisdigit(expr[i + 1]) || (expr[i + 1] == '-') || (expr[i + 1] == '+')))
continue;
}

count++;
}
}

/* When there are no "x"'s in the equation (ie, simple transform case),
* we don't need to allocate any space since no array will have to be
Expand Down
1 change: 1 addition & 0 deletions test/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -331,6 +331,7 @@ set (H5_TESTS
thread_id # special link
vol
timer
cmpd_dtransform
)

macro (ADD_H5_EXE file)
Expand Down
2 changes: 1 addition & 1 deletion test/Makefile.am
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ TEST_PROG= testhdf5 \
cache cache_api cache_image cache_tagging lheap ohdr \
stab gheap evict_on_close farray earray btree2 fheap \
pool accum hyperslab istore bittests dt_arith page_buffer \
dtypes dsets chunk_info cmpd_dset filter_fail extend direct_chunk \
dtypes dsets chunk_info cmpd_dset cmpd_dtransform filter_fail extend direct_chunk \
external efc objcopy objcopy_ref links unlink twriteorder big mtime \
fillval mount \
flush1 flush2 app_ref enum set_extent ttsafe enc_dec_plist \
Expand Down
136 changes: 136 additions & 0 deletions test/cmpd_dtransform.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
* Copyright by The HDF Group. *
* All rights reserved. *
* *
* This file is part of HDF5. The full HDF5 copyright notice, including *
* terms governing use, modification, and redistribution, is contained in *
* the COPYING file, which can be found at the root of the source code *
* distribution tree, or in https://www.hdfgroup.org/licenses. *
* If you do not have access to either file, you may request a copy from *
* [email protected]. *
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * */

/*
* Programmer: Jan-Willem Blokland
* December 1, 2020
*
* Purpose: Test writing compounded attribute followed by
* writing data with a data transform function.
*/

#include "h5test.h"

#define FILENAME "cmpd_dtransform.h5"
#define LENGTH 11

typedef struct {
char name[64];
char unit[64];
} att_t;

int
main(void)
{
hsize_t dima[] = {1};
hsize_t dims[] = {LENGTH};
hid_t str_dtyp_id, att_dtyp_id, file_id, fspace_id, dset_id, att_dspc_id, att_attr_id, dxpl_id;

/* Compound datatype */
att_t *atts = HDmalloc(sizeof(att_t));
HDstrcpy(atts[0].name, "Name");
HDstrcpy(atts[0].unit, "Unit");

/* String type */
if ((str_dtyp_id = H5Tcopy(H5T_C_S1)) < 0)
TEST_ERROR;
H5Tset_size(str_dtyp_id, 64);

/* Attribute type */
if ((att_dtyp_id = H5Tcreate(H5T_COMPOUND, sizeof(att_t))) < 0)
TEST_ERROR;
H5Tinsert(att_dtyp_id, "NAME", HOFFSET(att_t, name), str_dtyp_id);
H5Tinsert(att_dtyp_id, "UNIT", HOFFSET(att_t, unit), str_dtyp_id);

/* Create file. */
if ((file_id = H5Fcreate(FILENAME, H5F_ACC_TRUNC, H5P_DEFAULT, H5P_DEFAULT)) < 0)
TEST_ERROR;

/* Create file dataspace. */
if ((fspace_id = H5Screate_simple(1, dims, NULL)) < 0)
TEST_ERROR;

/* Create dataset. */
if ((dset_id = H5Dcreate2(file_id, "test_dset", H5T_NATIVE_INT, fspace_id, H5P_DEFAULT, H5P_DEFAULT,
H5P_DEFAULT)) < 0)
TEST_ERROR;

/* Write the attribute (compound) to the dataset */
if ((att_dspc_id = H5Screate_simple(1, dima, NULL)) < 0)
TEST_ERROR;
if ((att_attr_id =
H5Acreate2(dset_id, "ATTRIBUTES", att_dtyp_id, att_dspc_id, H5P_DEFAULT, H5P_DEFAULT)) < 0)
TEST_ERROR;
if (H5Awrite(att_attr_id, att_dtyp_id, atts) < 0)
TEST_ERROR;

/* Create dataset transfer property list */
const char *expr = "2*x";
if ((dxpl_id = H5Pcreate(H5P_DATASET_XFER)) < 0)
TEST_ERROR;
if (H5Pset_data_transform(dxpl_id, expr) < 0) {
HDprintf("**** ERROR: H5Pset_data_transform (expression: %s) ****\n", expr);
TEST_ERROR;
}

int *data = HDmalloc(LENGTH * sizeof(int));
int *data_res = HDmalloc(LENGTH * sizeof(int));
for (unsigned i = 0; i < LENGTH; i++) {
data[i] = 10;
data_res[i] = 2 * data[i];
}

/* Write the data */
if (H5Dwrite(dset_id, H5T_NATIVE_INT, H5S_ALL, H5S_ALL, dxpl_id, data) < 0)
TEST_ERROR;

/* Read attribute */
att_t *atts_res = HDmalloc(sizeof(att_t));
if (H5Aread(att_attr_id, att_dtyp_id, atts_res) < 0)
TEST_ERROR;

/* Verify attribute */
if (HDstrcmp(atts_res[0].name, atts[0].name) != 0)
TEST_ERROR;
if (HDstrcmp(atts_res[0].unit, atts[0].unit) != 0)
TEST_ERROR;

/* Read the data */
if (H5Dread(dset_id, H5T_NATIVE_INT, H5S_ALL, H5S_ALL, H5P_DEFAULT, data) < 0)
TEST_ERROR;

/* Verify data */
for (unsigned idx = 0; idx < LENGTH; idx++) {
if (data[idx] != data_res[idx])
TEST_ERROR;
}

HDfree(atts);
HDfree(atts_res);
HDfree(data);
HDfree(data_res);

/* Close all identifiers. */
H5Pclose(dxpl_id);
H5Aclose(att_attr_id);
H5Sclose(att_dspc_id);
H5Dclose(dset_id);
H5Sclose(fspace_id);
H5Fclose(file_id);
H5Tclose(att_dtyp_id);
H5Tclose(str_dtyp_id);

return 0;

error:
return 1;
}
24 changes: 24 additions & 0 deletions test/dtransform.c
Original file line number Diff line number Diff line change
Expand Up @@ -581,6 +581,7 @@ test_specials(hid_t file)
const char *special3 = "1000/x";
const char *special4 = "-x";
const char *special5 = "+x";
const char *special6 = "2e+1*x";

TESTING("data transform of some special cases")

Expand Down Expand Up @@ -702,6 +703,29 @@ test_specials(hid_t file)

COMPARE_INT(read_buf, data_res)

if (H5Dclose(dset_id) < 0)
TEST_ERROR

/*-----------------------------
* Operation 6: 2e+1*x
*----------------------------*/
if (H5Pset_data_transform(dxpl_id, special6) < 0)
TEST_ERROR;

for (row = 0; row < ROWS; row++)
for (col = 0; col < COLS; col++)
data_res[row][col] = transformData[row][col] * 20;

if ((dset_id = H5Dcreate2(file, "/special6", H5T_NATIVE_INT, dataspace, H5P_DEFAULT, H5P_DEFAULT,
H5P_DEFAULT)) < 0)
TEST_ERROR
if (H5Dwrite(dset_id, H5T_NATIVE_INT, H5S_ALL, H5S_ALL, dxpl_id, transformData) < 0)
TEST_ERROR
if (H5Dread(dset_id, H5T_NATIVE_INT, H5S_ALL, H5S_ALL, H5P_DEFAULT, read_buf) < 0)
TEST_ERROR

COMPARE_INT(read_buf, data_res)

if (H5Dclose(dset_id) < 0)
TEST_ERROR

Expand Down

0 comments on commit 6ff6504

Please sign in to comment.