Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replaced last sprintf with snprintf #4007

Merged
merged 2 commits into from
Feb 20, 2024

Conversation

seanm
Copy link
Contributor

@seanm seanm commented Feb 12, 2024

To replace the only remaining sprintf with safer snprintf the size of the buffer needs to be known.

But in this case will require non-trivial changes that need thought/discussion.

This is just to show the issue...

@seanm
Copy link
Contributor Author

seanm commented Feb 12, 2024

@byrnHDF @jhendersonHDF @hyoklee this last one is non-trivial... thoughts?

@seanm seanm marked this pull request as draft February 12, 2024 18:28
@seanm seanm force-pushed the sprintf-API-change branch from 1248819 to b92d0a8 Compare February 13, 2024 17:36
@jhendersonHDF
Copy link
Collaborator

This approach seems reasonable and in line with other changes we made to H5O code when trying to clear up buffer overflow issues due to not knowing the size of the message buffers. Of course if you update the H5O_msg_class_t structure there will be several other encode callbacks that need to be updated, but it doesn't seem like a particularly bad idea to me and will be trivial unless you want to add new checks in all the encode routines to make sure buffers aren't overflowed. https://github.com/HDFGroup/hdf5/blob/develop/src/H5Omessage.c#L1586 and https://github.com/HDFGroup/hdf5/blob/develop/src/H5Omessage.c#L1975-L1976 are where the encode callback gets made and, at least at first glance, mesg->raw_size should contain the size of the allocated buffer for the message in the latter case. You'd need to calculate the remaining size in the buffer based on the current position in the message buffer of the uint8_t *p; variable in that code, but it seems perfectly doable.

For the former case, it looks like that function is currently called in 7 places and each of those places appear to have some calculated size that could be used, though in several places you will also have to make adjustments to the size value passed based on a pointer into the message buffer. In short, seems reasonable but will definitely be a decent bit of work to have a reasonable value to pass for p_size in every case.

@seanm
Copy link
Contributor Author

seanm commented Feb 13, 2024

This approach seems reasonable and in line with other changes we made to H5O code when trying to clear up buffer overflow issues due to not knowing the size of the message buffers.

Is this changing public API? Any need for some kind of API version bump or anything like that?

...unless you want to add new checks in all the encode routines to make sure buffers aren't overflowed.

I don't want to do that now. But it's probably a good idea, and will become possible with this groundwork.

My motivation now is just to fix the compiler warnings about sprintf.

@jhendersonHDF
Copy link
Collaborator

Is this changing public API? Any need for some kind of API version bump or anything like that?

It doesn't look like any sort of public API changes are needed here. It looks to all be in private code.

I don't want to do that now. But it's probably a good idea, and will become possible with this groundwork.

As sort of a half-measure, you could consider passing SIZE_MAX for the other places with the new p_size parameter, similar to what we do in https://github.com/HDFGroup/hdf5/blob/develop/src/H5Odtype.c#L1328-L1333, to signal that the buffer size checks should be skipped. Not ideal, but it's a quick solution and doesn't really change the library's current status in regards to checking buffer sizes on encode. Then you could just pass the correct size for p_size to H5O__mtime_encode.

@seanm
Copy link
Contributor Author

seanm commented Feb 13, 2024

It doesn't look like any sort of public API changes are needed here. It looks to all be in private code.

Oh ok great! I thought this was more of a breaking change.

As sort of a half-measure, you could consider passing SIZE_MAX for the other places with the new p_size parameter

Yeah, where I can't trivially determine the size, I'll do that. As you say, it's no worse than today. :)

@seanm seanm force-pushed the sprintf-API-change branch from 0bc302d to c76e88a Compare February 13, 2024 21:50
@seanm
Copy link
Contributor Author

seanm commented Feb 13, 2024

So, I had to use SIZE_MAX in 3 of 4 call sites. :( There's a cascading effect now, where those call sites themselves need their function signature changed so their callers pass the buffer length.

But that can be some future PR...

@qkoziol
Copy link
Contributor

qkoziol commented Feb 14, 2024

LGTM

To have the size of the buffer, it was required to change a function signature, and change all users of it.

In most cases, determining the buffer size wasn't  trivial and so SIZE_MAX is passed. But at least this improves the infrastructure. Someone can later figure out the correct sizes.
@seanm seanm force-pushed the sprintf-API-change branch from 5c5115b to ac86ce8 Compare February 14, 2024 23:38
@seanm seanm changed the title WIP: discussion point for last sprintf Replaced last sprintf with snprintf Feb 14, 2024
@seanm seanm marked this pull request as ready for review February 14, 2024 23:39
@seanm
Copy link
Contributor Author

seanm commented Feb 14, 2024

@qkoziol thanks. I've updated the commit message and removed the Draft flag.

@derobins derobins added Priority - 1. High 🔼 These are important issues that should be resolved in the next release Component - C Library Core C library issues (usually in the src directory) labels Feb 16, 2024
@ajelenak ajelenak merged commit 7aed6ab into HDFGroup:develop Feb 20, 2024
lrknox pushed a commit to lrknox/hdf5 that referenced this pull request Mar 4, 2024
* Replaced last sprintf with snprintf

To have the size of the buffer, it was required to change a function signature, and change all users of it.

In most cases, determining the buffer size wasn't  trivial and so SIZE_MAX is passed. But at least this improves the infrastructure. Someone can later figure out the correct sizes.
lrknox added a commit that referenced this pull request Mar 5, 2024
* Remove oneapi return value warning. (#4028)

* Replaced last sprintf with snprintf (#4007)

* Replaced last sprintf with snprintf

To have the size of the buffer, it was required to change a function signature, and change all users of it.

In most cases, determining the buffer size wasn't  trivial and so SIZE_MAX is passed. But at least this improves the infrastructure. Someone can later figure out the correct sizes.

* Test vlen sequence IO in API tests (#4027)

* Check argument for CMake REGEX FCMangle.h. (#4029)

* Replace deprecated Fortran 'include mpif.h' with 'USE mpi' (#4031)

With MPI 4.1 the use of the mpif.h include file has been deprecated. Codes
should transition to USE mpi or USE mpi_f08.

Signed-off-by: Christoph Niethammer <[email protected]>

* Fix H5F_get_access_plist to copy file locking settings (#4030)

H5F_get_access_plist previously did not copy over the file locking settings
from a file into the new File Access Property List that it creates. This would
make it difficult to match the file locking settings between an external file
and its parent file.

* Fix missing NOT from if check in HL folder (#4036)

* Fix the datatype passed to H5*exists_async APIs in tests. (#4033)

Add a new testing function to verify C_BOOL values.

* Add deb and rpm binaries to snapshots (#4035)

* Update and Add general INSTALL (#4016)

* Improve performance of flushing single objects (#4017)

Improve performance of flushing a single object, and remove metadata
cache flush markers

* Fix memory leak in H5LTopen_file_image when H5LT_FILE_IMAGE_DONT_COPY flag is used (#4021)

When the H5LT_FILE_IMAGE_DONT_COPY flag is passed to H5LTopen_file_image, the internally-allocated
udata structure gets leaked as the core file driver doesn't have a way to determine when or if it
needs to call the 'udata_free' callback. This has been fixed by freeing the udata structure when
the 'image_free' callback gets made during file close, where the file is holding the last reference
to the udata structure.

* Fix allocating too much memory in dset API test (#4041)

* Don't try to load general-19 warnings file for icc (#4042)

The Autools Classic Intel compiler configuration attempts to load a file
named `general-19` from the intel-warnings/classic directory, which does
not exist.

This removes the attempted load of the file.

* Remove unused AIX cross-compile cache overrides (#4043)

The ibm-aix Autotools config file had some unmaintained and unnecessary
Autoconf cache overrides. These have been removed.

* Consolidate Autotools linux files (#4044)

There are many architecture-specific linux files in the config
directory, all of which simply redirect to linux-gnulibc1.

This change renames linux-gnulibc1 to linux-gnu and deletes the more
specific files.

* Remove check for gettimeofday + tz in CMake (#4045)

This is not used in the library

* Remove limitations on preset generators (#4051)

* Fix issue with FAPL file locking setting inheriting test (#4053)

Fixes an issue where the HDF5_USE_FILE_LOCKING environment variable being
set can interfere with the file locking setting that the test expects to
be returned.

* Bump the github-actions group with 2 updates (#4054)

Bumps the github-actions group with 2 updates: [actions/download-artifact](https://github.com/actions/download-artifact) and [github/codeql-action](https://github.com/github/codeql-action).

* Fix VOL-compatibility issues in External Link API test  (#4039)

Fix link API tests with incorrect filename

* Add upddated cmake tools from source location (#4040)

* Add options to allow tools type selection and naming (#4046)

* Improve error messages when tools attempt to use non-enabled S3 and HDFS VFDs. (#4047)

* Correct several 1.15/1.15.0 references to 1.14/1.14.4.

* Ignore HDF5Examples/CMakeUserPresets.json
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component - C Library Core C library issues (usually in the src directory) Priority - 1. High 🔼 These are important issues that should be resolved in the next release
Projects
Status: Needs Merged
Development

Successfully merging this pull request may close these issues.

6 participants