Rework Dynamic Analysis and sanitize testing #4681
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
byrnHDF
added
Merge - To 1.14
Priority - 2. Medium ⏹
byrnHDF
requested review from
lrknox,
derobins,
fortnern,
jhendersonHDF,
qkoziol,
vchoi-hdfgroup,
bmribler,
glennsong09,
mattjala and
brtnfld
as code owners
tools/test/h5ls/CMakeTests.cmake
Outdated
| @@ -320,22 +320,28 @@ | |||
| # rather than "IEEE 16-bit little-endian float". | |||
| if (H5_WORDS_BIGENDIAN) | |||
| ADD_H5_TEST (tfloat16_be 0 -w80 -v tfloat16_be.h5) | |||
| ADD_H5_TEST (tfloat16_be_nosupport 0 -w80 -v tfloat16_be.h5) | |||
| set_tests_properties (H5LS-tfloat16_be_nosupport PROPERTIES WILL_FAIL "true") | |||
| if (NOT HDF5_USING_ANALYSIS_TOOL) | |||
There was a problem hiding this comment.
Why do we need to be checking whether an analysis tool is used when tests that are expected to fail are being tested? Surely there's a better way to do this.
There was a problem hiding this comment.
The trouble with these tests are that the sanitzer will avoid the runtest.cmake macro and therefore the test will not fail. I could execute the test and skip the property setting.
There was a problem hiding this comment.
What is it about the sanitizers that causes them to not run through the normal testing macros/framework? Do they do more than just add in the -fsanitize=address flag for example?
There was a problem hiding this comment.
I am investigating that now - it might just be valgrind that has the issue.
There was a problem hiding this comment.
Looks like the sanitizers do not have a problem with the macros - so will convert the checks to MEMCHECKER only.
There was a problem hiding this comment.
Checking for a specific analysis tool doesn't seem much better either. I don't necessarily want to suggest adding another test macro for this purpose, but it's not great that anybody adding new tests needs to know that tests which are expected to fail have to be wrapped in if (NOT HDF5_USING_ANALYSIS_TOOL) or similar. It should be simpler than that.
There was a problem hiding this comment.
Reworked this whole analysis as sanitisers do not have the problems that valgrind has.
In fact, I think the sanitizers negate the need to support valgrind limitations and even the MEMCHECKER checks should just be eliminated.
byrnHDF
changed the title
|
The sanitizers do not need any special handling for the CMake macros like valgrind. The variety of sanitizers make the issues with valgrind moot, as the sanitizers provide more information during a test run on failures. |
lrknox
approved these changes
Aug 5, 2024
jhendersonHDF
approved these changes
Aug 5, 2024
lrknox
pushed a commit
to lrknox/hdf5
that referenced
this pull request
Aug 21, 2024
* Ignore predetermined failing test and check pointer before use * Rework Analysis process
lrknox
added a commit
that referenced
this pull request
Aug 22, 2024
* Warning fix (#4682) * warning fix * warning fix * CMake link line needs to use new HDF5_ENABLE_THREADS (#4685) * Correct the properties for using THREADS library (#4690) * Bump the github-actions group with 5 updates (#4688) Bumps the github-actions group with 5 updates: | Package | From | To | | --- | --- | --- | | [actions/download-artifact](https://github.com/actions/download-artifact) | `4.1.7` | `4.1.8` | | [DoozyX/clang-format-lint-action](https://github.com/doozyx/clang-format-lint-action) | `0.13` | `0.17` | | [softprops/action-gh-release](https://github.com/softprops/action-gh-release) | `2.0.6` | `2.0.8` | | [ossf/scorecard-action](https://github.com/ossf/scorecard-action) | `2.3.3` | `2.4.0` | | [github/codeql-action](https://github.com/github/codeql-action) | `3.25.11` | `3.25.15` | Updates `actions/download-artifact` from 4.1.7 to 4.1.8 - [Release notes](https://github.com/actions/download-artifact/releases) - [Commits](actions/download-artifact@65a9edc...fa0a91b) Updates `DoozyX/clang-format-lint-action` from 0.13 to 0.17 - [Release notes](https://github.com/doozyx/clang-format-lint-action/releases) - [Commits](DoozyX/clang-format-lint-action@v0.13...v0.17) Updates `softprops/action-gh-release` from 2.0.6 to 2.0.8 - [Release notes](https://github.com/softprops/action-gh-release/releases) - [Changelog](https://github.com/softprops/action-gh-release/blob/master/CHANGELOG.md) - [Commits](softprops/action-gh-release@a74c6b7...c062e08) Updates `ossf/scorecard-action` from 2.3.3 to 2.4.0 - [Release notes](https://github.com/ossf/scorecard-action/releases) - [Changelog](https://github.com/ossf/scorecard-action/blob/main/RELEASE.md) - [Commits](ossf/scorecard-action@dc50aa9...62b2cac) Updates `github/codeql-action` from 3.25.11 to 3.25.15 - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@b611370...afb54ba) --- updated-dependencies: - dependency-name: actions/download-artifact dependency-type: direct:production update-type: version-update:semver-patch dependency-group: github-actions - dependency-name: DoozyX/clang-format-lint-action dependency-type: direct:production update-type: version-update:semver-minor dependency-group: github-actions - dependency-name: softprops/action-gh-release dependency-type: direct:production update-type: version-update:semver-patch dependency-group: github-actions - dependency-name: ossf/scorecard-action dependency-type: direct:production update-type: version-update:semver-minor dependency-group: github-actions - dependency-name: github/codeql-action dependency-type: direct:production update-type: version-update:semver-patch dependency-group: github-actions ... * Fix segfault when closing datatype during failure in H5Topen2 (#4683) * Rework Dynamic Analysis and sanitize testing (#4681) * Ignore predetermined failing test and check pointer before use * Rework Analysis process * Remove another H5E_BEGIN/END_TRY within the library (#4675) * Update logic for (deprecated) H5Gget_objinfo() call to eliminate H5E_BEGIN_TRY * Handle case for '.' at the end of a path * Drop H5E_BEGIN/END_TRY and just check the error return from H5I_clear_types() (#4694) Original case that the change in commit 2dc738a no longer applies. * Add check of returned value from API calls. (#4702) These were found while investigating GH-4672, but they were not related to GH-4672. * Add mac dmg binary and remove old macos-13 workflows (#4699) * Add Windows SHLWAPI lib to public interface (#4701) * Use local variable in btree2 and print value (#4679) * Correct logic * Technically, level 1 Express could skip tests * Add windows signing (#4703) * Add tests for H5R get name APIs (#4657) Added functionality tests for the following APIs: H5Rget_file_name H5Rget_obj_name H5Rget_attr_name Also removed "+1" when returning a name length in H5R__get_attr_name(). The exter "+1" gave an incorrect value for the length of the referenced object's attribute name. Fixed GH-4447 * Fix Fortran test The C API H5Rget_attr_name incorrectly added 1 to the length of the referenced object's attribute name, so the Fortran API h5rget_attr_name_f removed 1 from the returned value to accommodate the incorrectness. This PR fixes H5Rget_attr_name so this workaround in h5rget_attr_name_f is no longer needed. * Add test H5Aget_name against H5Rget_attr_name * Replace Visual Studio ???? with 2022 in MSI README file (#4709) * Change logic for checking secrets exists (#4711) * Change osx refs to macos (#4707) * Replace alias \Code with \TText (#4714) Fixed GH-2151 * Correct signing names and variables (#4713) * Add secrets to release workflow (#4719) * Add missing blosc2 info (#4717) * Fix error return types in H5Rdeprec.c (#4722) Copy-pasted code from elsewhere used FAIL instead of H5G_UNKNOWN and H5I_INVALID_HID. * Fix the release reference name (#4721) * Test creating unseekable file (#4720) * Cleanup up tests (#4724) * Add arch name to dmg file name (#4732) The binaries in snapshot dmg file do not work on x86_64. * Fix snapshot CI failure by adding arch name to dmg file (#4734) See also #4732. * Fix incorrect VOL vs. non-VOL calls partially (#4733) * Fix incorrect VOL vs. non-VOL calls H5Lget_info2() called H5I_object() instead of H5VL_vol_object() crashed user application. This is a wide-spread issue (GH-4730) but this PR only addresses GH-4705. * Remove an incorrect change * Fix segfault in ROS3 credential parsing (#4736) * Fix segfault in s3 credential parsing * Fix AWS cred parsing when >1 profile provided * Revert gh-pages action hash to fix daily build (#4735) * Revert gh-pages action hash to fix daily build See also #4734 * Revert gh-pages action hash to fix daily build * Eliminate another use of H5E_clear_stack() within the library (#4726) * Remove call to H5E_clear_stack() Also clean up a bunch of error macros and the return value from H5B_valid()
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Sanitizers do not exhibit the same issues as valgrind did with the CMake test macros. Therefore allow sanitizers to use the same testing processes as regular testing. Also consideration should be given to removing the workarounds used for valgrind testing as sanitizers seem to do much better reporting of issues.