Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds write_pod() to tiered storage's ByteBlockWriter #34409

Closed
wants to merge 1 commit into from

Conversation

brooksprumo
Copy link
Contributor

Problem

In Tiered Storage's ByteBlockWriter, it is possible to invoke undefined behavior with write_type(), if called with a value that contains uninitialized bytes. This happens with structs that contain implicit padding, e.g. HotAccountMeta.

As an incremental improvement, if we know when have a value with no uninitialized bytes, we can ensure no undefined behavior by using bytemuck::bytes_from().

Summary of Changes

Leverage bytemuck to provide an undefined-behavior-free version of write_type(), write_pod().

@brooksprumo brooksprumo self-assigned this Dec 12, 2023
@brooksprumo brooksprumo marked this pull request as ready for review December 12, 2023 01:26
Copy link

codecov bot commented Dec 12, 2023

Codecov Report

Merging #34409 (85cbb0f) into master (39a3566) will decrease coverage by 0.1%.
The diff coverage is 100.0%.

Additional details and impacted files
@@            Coverage Diff            @@
##           master   #34409     +/-   ##
=========================================
- Coverage    81.9%    81.8%   -0.1%     
=========================================
  Files         819      819             
  Lines      220984   220989      +5     
=========================================
- Hits       181015   180971     -44     
- Misses      39969    40018     +49     

Copy link
Contributor

@yhchiang-sol yhchiang-sol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! The PR looks good.

Btw, do you think we should deprecate write_type?

/// Write the specified typed instance to the internal buffer of
/// the ByteBlockWriter instance.
///
/// Prefer `write_pod()` when possible, because `write_type()` may cause
/// undefined behavior if `value` contains uninitized bytes.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for also improving the comment. Do you think we should deprecate write_type? Or we just make write_pod write_type?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We still need write_type() for some situations, so cannot remove it entirely.

However, I do not intend to merge this PR, as I've since put up #34415, which handles these concerns better.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. Let me convert it to a draft. Feel free to convert it back when you think it's needed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is ready for review, and is not a draft. But I'd rather solve this problem by #34415 instead. Please let's review that one first. If that one is merged, I'll close this PR.

@yhchiang-sol yhchiang-sol marked this pull request as draft December 13, 2023 20:37
@brooksprumo brooksprumo marked this pull request as ready for review December 13, 2023 20:58
@brooksprumo
Copy link
Contributor Author

Closing. This problem has been resolved by #34415.

@brooksprumo brooksprumo deleted the ts/ub/write_pod branch December 14, 2023 18:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants