Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge PRs in issue #642 to hdf5_1_12 #643

Merged
merged 60 commits into from
May 15, 2021
Merged
Changes from 1 commit
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
bcaf211
Snapshot version 1.12 release 1-3. Update version to 1.12.1-4.
lrknox Nov 23, 2020
9b9ef60
Merge branch 'hdf5_1_12' of https://github.com/HDFGroup/hdf5 into hdf…
lrknox Nov 26, 2020
7a78fd1
First cut of the H5 public API documentation. (#80)
gheber Nov 18, 2020
ba77da0
Full set of current H5F documentation. (#105)
gheber Nov 21, 2020
3c0394b
Doxygen - added (mostly) beginner functions (#112)
bljhdf Nov 23, 2020
3360ebd
Add src/H5module.h to MANIFEST.
lrknox Nov 26, 2020
5be0fe8
Merge branch 'hdf5_1_12' of https://github.com/HDFGroup/hdf5 into hdf…
lrknox Dec 11, 2020
a1ec23d
Merge branch 'hdf5_1_12' of https://github.com/HDFGroup/hdf5 into hdf…
lrknox Dec 14, 2020
0937643
Merge branch 'hdf5_1_12' of https://github.com/HDFGroup/hdf5 into hdf…
lrknox Dec 17, 2020
93f4ce1
close #195. (#196)
hyoklee Dec 17, 2020
d9fcbec
Merge branch 'hdf5_1_12' of https://github.com/HDFGroup/hdf5 into hdf…
lrknox Dec 26, 2020
16ca6d1
Avoid aligned access for references by decoding into temporary buffer…
qkoziol Dec 19, 2020
acfe570
Merge branch 'hdf5_1_12' of https://github.com/HDFGroup/hdf5 into hdf…
lrknox Jan 6, 2021
edf801c
Modify temporary rpath for testing in java example scripts. (#230)
lrknox Dec 29, 2020
3736bd1
Merge branch 'hdf5_1_12' of https://github.com/HDFGroup/hdf5 into hdf…
lrknox Jan 9, 2021
dbc345b
Merge branch 'hdf5_1_12' of https://github.com/HDFGroup/hdf5 into hdf…
lrknox Feb 23, 2021
3af66dd
Fix undefined left shifting of negative numbers (#338)
seanm Feb 19, 2021
67d2ada
Merge branch 'hdf5_1_12' of https://github.com/HDFGroup/hdf5 into hdf…
lrknox Mar 10, 2021
febde7c
Fixes various warnings noticed on Windows (#425)
derobins Mar 5, 2021
ac550e1
Merge branch 'hdf5_1_12' of https://github.com/HDFGroup/hdf5 into hdf…
lrknox Mar 10, 2021
9840892
Applied clang-tidy readability-non-const-parameter warning fixes auto…
seanm Mar 9, 2021
e391ca0
Added C++11 override keyword where appropriate (#433)
seanm Mar 10, 2021
aaa4929
Various clang tidy warning fixes (#448)
seanm Mar 10, 2021
c7ae947
Removed checks/workarounds for pre-C++89 compatibility (#449)
seanm Mar 10, 2021
eb95b11
Fixed all clang-tidy bugprone-suspicious-string-compare warnings (#451)
seanm Mar 10, 2021
80d2a1d
Remove 2 functions incorrectly merged from develop in a cherry-pick m…
lrknox Mar 11, 2021
fb6f838
Merge branch 'hdf5_1_12' of https://github.com/HDFGroup/hdf5 into hdf…
lrknox Mar 21, 2021
afccbcd
Merge branch 'hdf5_1_12' of https://github.com/HDFGroup/hdf5 into hdf…
lrknox Apr 1, 2021
0ae7773
Merge branch 'hdf5_1_12' of https://github.com/HDFGroup/hdf5 into hdf…
lrknox Apr 2, 2021
86e5ad4
Merge branch 'hdf5_1_12' of https://github.com/HDFGroup/hdf5 into hdf…
lrknox Apr 5, 2021
90d87c7
Merge branch 'hdf5_1_12' of https://github.com/HDFGroup/hdf5 into hdf…
lrknox May 3, 2021
7bd5f8c
Minor parallel improvements (#519)
qkoziol Mar 26, 2021
42d2a48
Clean up MPI-IO VFD tracing support (#520)
qkoziol Mar 26, 2021
38d1b12
Brings the native implementation of H5Fdelete() from Bitbucket (#524)
derobins Apr 16, 2021
7a61a4a
Minor warning fixes in develop (#526)
derobins Mar 31, 2021
e49e86f
Removes implementation of my_strdup() from the multi VFD (#527)
derobins Mar 31, 2021
c776b52
Removes dead H5ST package from the library (#528)
derobins Mar 30, 2021
292388a
Fix HDFFV-11232 (#530)
brtnfld Mar 31, 2021
e9c2cfb
Fixes incorrect usage of H5I_BADID (#554)
derobins Apr 19, 2021
81f0d6f
Fixes a segfault when H5Pset_mdc_log_options is called multiple times…
derobins Apr 29, 2021
1931062
Fix for a segfault when H5Pset_fapl_log is passed an invalid fapl ID …
derobins Apr 29, 2021
4685a83
Fixes crashes when size_hint > UINT32_MAX is passed to H5Gcreate1 (#611)
derobins Apr 30, 2021
e91845a
Revert "Brings the native implementation of H5Fdelete() from Bitbucke…
lrknox May 4, 2021
d52bb2d
Removed mentions of Wdeclaration-after-statement now that C99 is requ…
seanm May 3, 2021
7fcc80e
Made private my_yyinput function static (#618)
seanm May 4, 2021
546ee14
Adds const to a few global variables (#623)
derobins May 4, 2021
0cb8c35
Merge branch 'hdf5_1_12' of https://github.com/HDFGroup/hdf5 into hdf…
lrknox May 6, 2021
92a6cf7
(fix) Segmentation fault when using a compound type. (#143)
jwsblokland Dec 2, 2020
6978fa4
(fix) H5Z_xform_create function and scientific notation (#144)
jwsblokland Dec 2, 2020
719dc3c
Committing clang-format changes
github-actions[bot] May 6, 2021
8a6e98b
Merge branch 'hdf5_1_12' of https://github.com/HDFGroup/hdf5 into hdf…
lrknox May 10, 2021
60ce40c
Merge branch 'hdf5_1_12' of https://github.com/HDFGroup/hdf5 into hdf…
lrknox May 15, 2021
7a68286
Fix H5Eget_auto2/H5Eauto_is_v2 to not clear error stack (#625)
jhendersonHDF May 12, 2021
d14681c
Cleanup tools debug build warnings (#627)
byrnHDF May 5, 2021
37b13a9
h5diff subset indexing (#628)
byrnHDF May 7, 2021
7b6148d
Removes gratuitous (double)x.yF casts (#632)
derobins May 7, 2021
4d7289b
Cleans up a const warning left over from previous constification (#633)
derobins May 7, 2021
1376e7a
Purges UFAIL from the library (#637)
derobins May 11, 2021
f22e24a
Bmr dev hdffv 11223 (#640)
bmribler May 12, 2021
e4dda7a
Revert "Fix H5Eget_auto2/H5Eauto_is_v2 to not clear error stack (#625)"
lrknox May 15, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Bmr dev hdffv 11223 (#640)
* Fixed HDFFV-11223 (CVE-2018-14460)

Description
    - Added checks against buffer size to prevent segfault, in case of data
      corruption, for sdim->size and sdim->max.
    - Renamed data files in an existing test to shorten their length
      as agreed with other developers previously.
Platforms tested:
    Linux/64 (jelly)

* Committing clang-format changes

* Updated for test files

* Updated for HDFFV-11223

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2 people authored and lrknox committed May 15, 2021
commit f22e24a04d9be37627e8b62eaccb9eb7dfa220f9
3 changes: 2 additions & 1 deletion MANIFEST
Original file line number Diff line number Diff line change
@@ -2826,7 +2826,8 @@
./tools/test/h5repack/testfiles/h5repack_layout.h5-plugin_test.ddl
./tools/test/h5repack/testfiles/h5repack_layout.h5-plugin_version_test.ddl
./tools/test/h5repack/testfiles/h5repack_layout.h5-plugin_zero.ddl
./tools/test/h5repack/testfiles/h5repack_HDFFV-10590_CVE-2018-17432.h5
./tools/test/h5repack/testfiles/h5repack_CVE-2018-17432.h5
./tools/test/h5repack/testfiles/h5repack_CVE-2018-14460.h5
./tools/test/h5repack/testfiles/GS.h5repack_paged_nopersist.h5.ddl
./tools/test/h5repack/testfiles/S.h5repack_fsm_aggr_persist.h5.ddl
./tools/test/h5repack/testfiles/SP.h5repack_fsm_aggr_nopersist.h5.ddl
13 changes: 12 additions & 1 deletion release_docs/RELEASE.txt
Original file line number Diff line number Diff line change
@@ -469,7 +469,18 @@ Bug Fixes since HDF5-1.12.0 release
===================================
Library
-------
- Fixed CVE-2018-17435
- Fixed CVE-2018-14460

The tool h5repack produced a segfault when the rank in dataspace
message was corrupted, causing invalid read while decoding the
dimension sizes.

The problem was fixed by ensuring that decoding the dimension sizes
and max values will not go beyong the end of the buffer.

(BMR - 2021/05/12, HDFFV-11223)

- Fixed CVE-2018-11206

The tool h5dump produced a segfault when the size of a fill value
message was corrupted and caused a buffer overflow.
23 changes: 18 additions & 5 deletions src/H5Osdspace.c
Original file line number Diff line number Diff line change
@@ -106,12 +106,13 @@ H5FL_ARR_EXTERN(hsize_t);
--------------------------------------------------------------------------*/
static void *
H5O__sdspace_decode(H5F_t *f, H5O_t H5_ATTR_UNUSED *open_oh, unsigned H5_ATTR_UNUSED mesg_flags,
unsigned H5_ATTR_UNUSED *ioflags, size_t H5_ATTR_UNUSED p_size, const uint8_t *p)
unsigned H5_ATTR_UNUSED *ioflags, size_t p_size, const uint8_t *p)
{
H5S_extent_t *sdim = NULL; /* New extent dimensionality structure */
unsigned flags, version;
unsigned i; /* Local counting variable */
void * ret_value = NULL; /* Return value */
H5S_extent_t * sdim = NULL; /* New extent dimensionality structure */
unsigned flags, version;
unsigned i; /* Local counting variable */
const uint8_t *p_end = p + p_size - 1; /* End of the p buffer */
void * ret_value = NULL; /* Return value */

FUNC_ENTER_STATIC

@@ -161,6 +162,13 @@ H5O__sdspace_decode(H5F_t *f, H5O_t H5_ATTR_UNUSED *open_oh, unsigned H5_ATTR_UN

/* Decode dimension sizes */
if (sdim->rank > 0) {
/* Ensure that rank doesn't cause reading passed buffer's end,
due to possible data corruption */
uint8_t sizeof_size = H5F_SIZEOF_SIZE(f);
if (p + (sizeof_size * sdim->rank - 1) > p_end) {
HGOTO_ERROR(H5E_OHDR, H5E_OVERFLOW, NULL, "rank might cause reading passed buffer's end")
}

if (NULL == (sdim->size = (hsize_t *)H5FL_ARR_MALLOC(hsize_t, (size_t)sdim->rank)))
HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, NULL, "memory allocation failed")

@@ -170,6 +178,11 @@ H5O__sdspace_decode(H5F_t *f, H5O_t H5_ATTR_UNUSED *open_oh, unsigned H5_ATTR_UN
if (flags & H5S_VALID_MAX) {
if (NULL == (sdim->max = (hsize_t *)H5FL_ARR_MALLOC(hsize_t, (size_t)sdim->rank)))
HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, NULL, "memory allocation failed")

/* Ensure that rank doesn't cause reading passed buffer's end */
if (p + (sizeof_size * sdim->rank - 1) > p_end)
HGOTO_ERROR(H5E_OHDR, H5E_OVERFLOW, NULL, "rank might cause reading passed buffer's end")

for (i = 0; i < sdim->rank; i++)
H5F_DECODE_LENGTH(f, p, sdim->max[i]);
} /* end if */
10 changes: 8 additions & 2 deletions tools/test/h5repack/CMakeTests.cmake
Original file line number Diff line number Diff line change
@@ -51,7 +51,8 @@
${HDF5_TOOLS_TEST_H5REPACK_SOURCE_DIR}/testfiles/h5repack_named_dtypes.h5
${HDF5_TOOLS_TEST_H5REPACK_SOURCE_DIR}/testfiles/h5repack_nested_8bit_enum.h5
${HDF5_TOOLS_TEST_H5REPACK_SOURCE_DIR}/testfiles/h5repack_nested_8bit_enum_deflated.h5
${HDF5_TOOLS_TEST_H5REPACK_SOURCE_DIR}/testfiles/h5repack_HDFFV-10590_CVE-2018-17432.h5
${HDF5_TOOLS_TEST_H5REPACK_SOURCE_DIR}/testfiles/h5repack_CVE-2018-17432.h5
${HDF5_TOOLS_TEST_H5REPACK_SOURCE_DIR}/testfiles/h5repack_CVE-2018-14460.h5
${HDF5_TOOLS_TEST_H5REPACK_SOURCE_DIR}/testfiles/h5repack_nbit.h5
${HDF5_TOOLS_TEST_H5REPACK_SOURCE_DIR}/testfiles/h5repack_objs.h5
${HDF5_TOOLS_TEST_H5REPACK_SOURCE_DIR}/testfiles/h5repack_refs.h5
@@ -1551,10 +1552,15 @@
ADD_H5_TEST (HDFFV-7840 "TEST" h5diff_attr1.h5)

# test CVE-2018-17432 fix
set (arg h5repack_HDFFV-10590_CVE-2018-17432.h5 h5repack_HDFFV-10590_CVE-2018-17432_out.h5 --low=1 --high=2 -f GZIP=8 -l dset1:CHUNK=5x6)
set (arg h5repack_CVE-2018-17432.h5 h5repack__CVE-2018-17432_out.h5 --low=1 --high=2 -f GZIP=8 -l dset1:CHUNK=5x6)
set (TESTTYPE "TEST")
ADD_H5_FILTER_TEST (HDFFV-10590 "" ${TESTTYPE} 1 ${arg})

# test CVE-2018-14460 fix
set (arg h5repack_CVE-2018-14460.h5 h5repack_CVE-2018-14460_out.h5)
set (TESTTYPE "TEST")
ADD_H5_FILTER_TEST (HDFFV-11223 "" ${TESTTYPE} 1 ${arg})

# tests for metadata block size option ('-M')
ADD_H5_TEST_META (meta_short h5repack_layout.h5 -M 8192)
ADD_H5_TEST_META (meta_long h5repack_layout.h5 --metadata_block_size=8192)
9 changes: 7 additions & 2 deletions tools/test/h5repack/h5repack.sh.in
Original file line number Diff line number Diff line change
@@ -129,7 +129,8 @@ $SRC_H5REPACK_TESTFILES/h5repack_paged_persist.h5
########h5diff/testfile########
$SRC_H5DIFF_TESTFILES/h5diff_attr1.h5
########test#HDFFV-10590########
$SRC_H5REPACK_TESTFILES/h5repack_HDFFV-10590_CVE-2018-17432.h5
$SRC_H5REPACK_TESTFILES/h5repack_CVE-2018-17432.h5
$SRC_H5REPACK_TESTFILES/h5repack_CVE-2018-14460.h5
########tools/testfiles#for#external#links########
$SRC_TOOLS_TESTFILES/tsoftlinks.h5
$SRC_TOOLS_TESTFILES/textlinkfar.h5
@@ -1712,7 +1713,11 @@ TOOLTEST HDFFV-5932 h5repack_attr_refs.h5
TOOLTEST HDFFV-7840 h5diff_attr1.h5

# test HDFFV-10590
arg="h5repack_HDFFV-10590_CVE-2018-17432.h5 h5repack_HDFFV-10590_CVE-2018-17432_out.h5 --low=1 --high=2 -f GZIP=8 -l dset1:CHUNK=5x6"
arg="h5repack_CVE-2018-17432.h5 h5repack_CVE-2018-17432_out.h5 --low=1 --high=2 -f GZIP=8 -l dset1:CHUNK=5x6"
TOOLTEST_FAIL $arg

# test HDFFV-11223
arg="h5repack_CVE-2018-14460.h5 h5repack_CVE-2018-14460_out.h5"
TOOLTEST_FAIL $arg

# tests for metadata block size option
Binary file not shown.