[Merged by Bors] - Ensure Ptr/PtrMut/OwningPtr are aligned when casting in debug builds #7117

james7132 · 2023-01-07T04:09:22Z

Objective

Improve safety testing when using bevy_ptr types. This is a follow-up to #7113.

Solution

Add a debug-only assertion that pointers are aligned when casting to a concrete type. This should very quickly catch any unsoundness from unaligned pointers, even without miri. However, this can have a large negative perf impact on debug builds.

Changelog

Added: Ptr::deref will now panic in debug builds if the pointer is not aligned.
Added: PtrMut::deref_mut will now panic in debug builds if the pointer is not aligned.
Added: OwningPtr::read will now panic in debug builds if the pointer is not aligned.
Added: OwningPtr::drop_as will now panic in debug builds if the pointer is not aligned.

crates/bevy_ptr/src/lib.rs

Co-authored-by: JoJoJet <[email protected]>

JoJoJet

LGTM. I believe the CI failure is a pre-existing bug.

crates/bevy_ptr/src/lib.rs

BoxyUwU · 2023-01-08T06:45:30Z

the CI failure is caused by the fact that BlobVec::new does not create an aligned pointer for non-zsts:

            let mut blob_vec = BlobVec {
                data: NonNull::dangling(),
                capacity: 0,
                len: 0,
                item_layout,
                drop,
            };

it should be data: bevy_ptr::dangling_with_align(align) like we do for ZSTs (see also #6618 which fixed a similar bug for zsts)

BoxyUwU · 2023-01-09T09:55:44Z

This PR doesn't add a safety invariant to the new function's of Ptr/PtrMut/OwningPtr requiring the pointer to have correct alignment for the pointee. Although it was pointed out that it was useful to use these types for commands which are not aligned 🤔 I wonder if it would be worth adding a defaulted param to these for whether they're aligned or not: struct Ptr<'a, A = Aligned> {

(alternatively we could treat them normal raw pointers and not require alignment, instead opting to make read and friends unsafe with an invariant that they are correctly aligned... but I do not like this much as "borrow but without the pointee type known" is a nice mental model that would be a shame to lose)

james7132 · 2023-01-09T14:55:49Z

I guess my question is how do you enforce alignment while type erased?

We could provide safe conversions from normal borrows, mutable or not. However, we don't even have that in the ECS where there may not be a type at all to work with.

Only other option is to explicitly provide the layout or alignment to the constructor and at that point it feels sort of redundant/clunky. Like asking a student to grade their own homework.

We already support unaligned reads on the types so IMO treating these as "lifetimed type erased pointers" over "type erased borrows" makes a lot more sense.

BoxyUwU · 2023-01-09T16:01:33Z

You enforce it with a safety invariant // SAFETY: The pointer must be sufficiently aligned for the erased pointee type. You would still assert it in the read/deref methods like you're doing in this PR.

edit: If we aren't enforcing it on construction then we need to have safety invariants on read/deref etc that we are sufficiently aligned even though in practically every use case this is something we should know on construction of the Ptr so it just seems better to me to do it on construction

james7132 · 2023-01-09T16:17:56Z

I think #7113 adds exactly that. Perhaps we should merge that first?

BoxyUwU · 2023-01-09T16:24:55Z

I was not aware we already had PR open adding those invariants thats good to know before its merged...

alice-i-cecile · 2023-01-10T18:11:57Z

I want #7151 merged before this.

@JoJoJet

…7151) # Objective The types in the `bevy_ptr` accidentally did not document anything relating to alignment. This is unsound as many methods rely on the pointer being correctly aligned. ## Solution This PR introduces new safety invariants on the `$ptr::new`, `$ptr::byte_offset` and `$ptr::byte_add` methods requiring them to keep the pointer aligned. This is consistent with the documentation of these pointer types which document them as being "type erased borrows". As it was pointed out (by @JoJoJet in #7117) that working with unaligned pointers can be useful (for example our commands abstraction which does not try to align anything properly, see #7039) this PR also introduces a default type parameter to all the pointer types that specifies whether it has alignment requirements or not. I could not find any code in `bevy_ecs` that would need unaligned pointers right now so this is going unused. --- ## Changelog - Correctly document alignment requirements on `bevy_ptr` types. - Support variants of `bevy_ptr` types that do not require being correctly aligned for the pointee type. ## Migration Guide - Safety invariants on `bevy_ptr` types' `new` `byte_add` and `byte_offset` methods have been changed. All callers should re-audit for soundness.

Co-authored-by: JoJoJet <[email protected]>

james7132 · 2023-01-11T20:06:11Z

With #7151 merged, @BoxyUwU @JoJoJet @alice-i-cecile, this PR should be ready.

alice-i-cecile · 2023-01-11T23:12:03Z

bors r+

…7117) # Objective Improve safety testing when using `bevy_ptr` types. This is a follow-up to #7113. ## Solution Add a debug-only assertion that pointers are aligned when casting to a concrete type. This should very quickly catch any unsoundness from unaligned pointers, even without miri. However, this can have a large negative perf impact on debug builds. --- ## Changelog Added: `Ptr::deref` will now panic in debug builds if the pointer is not aligned. Added: `PtrMut::deref_mut` will now panic in debug builds if the pointer is not aligned. Added: `OwningPtr::read` will now panic in debug builds if the pointer is not aligned. Added: `OwningPtr::drop_as` will now panic in debug builds if the pointer is not aligned.

bors · 2023-01-11T23:31:14Z

Pull request successfully merged into main.

Build succeeded:

@JoJoJet

…evyengine#7151) # Objective The types in the `bevy_ptr` accidentally did not document anything relating to alignment. This is unsound as many methods rely on the pointer being correctly aligned. ## Solution This PR introduces new safety invariants on the `$ptr::new`, `$ptr::byte_offset` and `$ptr::byte_add` methods requiring them to keep the pointer aligned. This is consistent with the documentation of these pointer types which document them as being "type erased borrows". As it was pointed out (by @JoJoJet in bevyengine#7117) that working with unaligned pointers can be useful (for example our commands abstraction which does not try to align anything properly, see bevyengine#7039) this PR also introduces a default type parameter to all the pointer types that specifies whether it has alignment requirements or not. I could not find any code in `bevy_ecs` that would need unaligned pointers right now so this is going unused. --- ## Changelog - Correctly document alignment requirements on `bevy_ptr` types. - Support variants of `bevy_ptr` types that do not require being correctly aligned for the pointee type. ## Migration Guide - Safety invariants on `bevy_ptr` types' `new` `byte_add` and `byte_offset` methods have been changed. All callers should re-audit for soundness.

…evyengine#7117) # Objective Improve safety testing when using `bevy_ptr` types. This is a follow-up to bevyengine#7113. ## Solution Add a debug-only assertion that pointers are aligned when casting to a concrete type. This should very quickly catch any unsoundness from unaligned pointers, even without miri. However, this can have a large negative perf impact on debug builds. --- ## Changelog Added: `Ptr::deref` will now panic in debug builds if the pointer is not aligned. Added: `PtrMut::deref_mut` will now panic in debug builds if the pointer is not aligned. Added: `OwningPtr::read` will now panic in debug builds if the pointer is not aligned. Added: `OwningPtr::drop_as` will now panic in debug builds if the pointer is not aligned.

@JoJoJet

…evyengine#7151) # Objective The types in the `bevy_ptr` accidentally did not document anything relating to alignment. This is unsound as many methods rely on the pointer being correctly aligned. ## Solution This PR introduces new safety invariants on the `$ptr::new`, `$ptr::byte_offset` and `$ptr::byte_add` methods requiring them to keep the pointer aligned. This is consistent with the documentation of these pointer types which document them as being "type erased borrows". As it was pointed out (by @JoJoJet in bevyengine#7117) that working with unaligned pointers can be useful (for example our commands abstraction which does not try to align anything properly, see bevyengine#7039) this PR also introduces a default type parameter to all the pointer types that specifies whether it has alignment requirements or not. I could not find any code in `bevy_ecs` that would need unaligned pointers right now so this is going unused. --- ## Changelog - Correctly document alignment requirements on `bevy_ptr` types. - Support variants of `bevy_ptr` types that do not require being correctly aligned for the pointee type. ## Migration Guide - Safety invariants on `bevy_ptr` types' `new` `byte_add` and `byte_offset` methods have been changed. All callers should re-audit for soundness.

…evyengine#7117) # Objective Improve safety testing when using `bevy_ptr` types. This is a follow-up to bevyengine#7113. ## Solution Add a debug-only assertion that pointers are aligned when casting to a concrete type. This should very quickly catch any unsoundness from unaligned pointers, even without miri. However, this can have a large negative perf impact on debug builds. --- ## Changelog Added: `Ptr::deref` will now panic in debug builds if the pointer is not aligned. Added: `PtrMut::deref_mut` will now panic in debug builds if the pointer is not aligned. Added: `OwningPtr::read` will now panic in debug builds if the pointer is not aligned. Added: `OwningPtr::drop_as` will now panic in debug builds if the pointer is not aligned.

Ensure Ptr/PtrMut/OwningPtr are aligned when casting in debug modes

513cf03

james7132 requested review from BoxyUwU, TheRawMeatball and JoJoJet January 7, 2023 04:09

james7132 added the P-Unsound A bug that results in undefined compiler behavior label Jan 7, 2023

james7132 added 5 commits January 6, 2023 20:21

Use the proper way to ensure the pointer is aligned

59c7835

Disable the checks for miri builds

4309977

Fix syntax error

340e2a6

Fix formatting

ce6a32c

Fix CI

e186cfb

JoJoJet suggested changes Jan 7, 2023

View reviewed changes

crates/bevy_ptr/src/lib.rs Outdated Show resolved Hide resolved

crates/bevy_ptr/src/lib.rs Outdated Show resolved Hide resolved

crates/bevy_ptr/src/lib.rs Outdated Show resolved Hide resolved

james7132 and others added 3 commits January 7, 2023 09:27

Apply suggestions from code review

da74b3e

Co-authored-by: JoJoJet <[email protected]>

Add assertion to dangling_with_align

29cb4ad

Add type name to the assertion

f4a36f8

JoJoJet mentioned this pull request Jan 7, 2023

[Merged by Bors] - Improve safety for CommandQueue internals #7039

Closed

JoJoJet approved these changes Jan 8, 2023

View reviewed changes

crates/bevy_ptr/src/lib.rs Outdated Show resolved Hide resolved

alice-i-cecile added A-ECS Entities, components, systems, and events C-Code-Quality A section of code that is hard to understand or change D-Complex Quite challenging from either a design or technical perspective. Ask for help! labels Jan 8, 2023

james7132 added 3 commits January 8, 2023 18:43

EnsureAligned -> DebugEnsureAligned

a8e0d63

Remove panic documentation

5072f0d

Fix BlobVec's initialization for non-ZST types

8c34e8e

JoJoJet mentioned this pull request Jan 9, 2023

[Merged by Bors] - Add a method for converting MutUntyped -> Mut<T> #7113

Closed

JoJoJet self-requested a review January 9, 2023 21:00

BoxyUwU mentioned this pull request Jan 10, 2023

[Merged by Bors] - Document alignment requirements of Ptr, PtrMut and OwningPtr #7151

Closed

james7132 and others added 2 commits January 11, 2023 15:04

Merge branch 'main' into debug-ptr-align-assert

7701478

Make the assert only debug.

9ff927b

Co-authored-by: JoJoJet <[email protected]>

JoJoJet approved these changes Jan 11, 2023

View reviewed changes

alice-i-cecile approved these changes Jan 11, 2023

View reviewed changes

alice-i-cecile added the S-Ready-For-Final-Review This PR has been approved by the community. It's ready for a maintainer to consider merging it label Jan 11, 2023

BoxyUwU approved these changes Jan 11, 2023

View reviewed changes

bors bot changed the title ~~Ensure Ptr/PtrMut/OwningPtr are aligned when casting in debug builds~~ [Merged by Bors] - Ensure Ptr/PtrMut/OwningPtr are aligned when casting in debug builds Jan 11, 2023

bors bot closed this Jan 11, 2023

james7132 deleted the debug-ptr-align-assert branch January 20, 2023 01:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Merged by Bors] - Ensure Ptr/PtrMut/OwningPtr are aligned when casting in debug builds #7117

[Merged by Bors] - Ensure Ptr/PtrMut/OwningPtr are aligned when casting in debug builds #7117

james7132 commented Jan 7, 2023 •

edited

Loading

JoJoJet left a comment

BoxyUwU commented Jan 8, 2023 •

edited

Loading

BoxyUwU commented Jan 9, 2023 •

edited

Loading

james7132 commented Jan 9, 2023

BoxyUwU commented Jan 9, 2023 •

edited

Loading

james7132 commented Jan 9, 2023

BoxyUwU commented Jan 9, 2023

alice-i-cecile commented Jan 10, 2023

james7132 commented Jan 11, 2023

alice-i-cecile commented Jan 11, 2023

bors bot commented Jan 11, 2023

[Merged by Bors] - Ensure Ptr/PtrMut/OwningPtr are aligned when casting in debug builds #7117

[Merged by Bors] - Ensure Ptr/PtrMut/OwningPtr are aligned when casting in debug builds #7117

Conversation

james7132 commented Jan 7, 2023 • edited Loading

Objective

Solution

Changelog

JoJoJet left a comment

Choose a reason for hiding this comment

BoxyUwU commented Jan 8, 2023 • edited Loading

BoxyUwU commented Jan 9, 2023 • edited Loading

james7132 commented Jan 9, 2023

BoxyUwU commented Jan 9, 2023 • edited Loading

james7132 commented Jan 9, 2023

BoxyUwU commented Jan 9, 2023

alice-i-cecile commented Jan 10, 2023

james7132 commented Jan 11, 2023

alice-i-cecile commented Jan 11, 2023

bors bot commented Jan 11, 2023

james7132 commented Jan 7, 2023 •

edited

Loading

BoxyUwU commented Jan 8, 2023 •

edited

Loading

BoxyUwU commented Jan 9, 2023 •

edited

Loading

BoxyUwU commented Jan 9, 2023 •

edited

Loading