Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Merged by Bors] - fix some memory leaks detected by miri #4959

Closed
wants to merge 3 commits into from

Conversation

BoxyUwU
Copy link
Member

@BoxyUwU BoxyUwU commented Jun 7, 2022

The first leak:

    #[test]
    fn blob_vec_drop_empty_capacity() {
        let item_layout = Layout::new::<Foo>();
        let drop = drop_ptr::<Foo>;
        let _ = unsafe { BlobVec::new(item_layout, Some(drop), 0) };
    }

this is because we allocate the swap scratch in blobvec regardless of what the capacity is, but we only deallocate if capacity is > 0

The second leak:

    #[test]
    fn panic_while_overwriting_component() {
        let helper = DropTestHelper::new();

        let res = panic::catch_unwind(|| {
            let mut world = World::new();
            world
                .spawn()
                .insert(helper.make_component(true, 0))
                .insert(helper.make_component(false, 1));

            println!("Done inserting! Dropping world...");
        });

        let drop_log = helper.finish(res);

        assert_eq!(
            &*drop_log,
            [
                DropLogItem::Create(0),
                DropLogItem::Create(1),
                DropLogItem::Drop(0),
            ]
        );
    }

this is caused by us not running the drop impl on the to-be-inserted component if the drop impl of the overwritten component panics


managed to figure out where the leaks were by using this 10/10 command

cargo --quiet test --lib -- --list | sed 's/: test$//' | MIRIFLAGS="-Zmiri-disable-isolation" xargs -n1 cargo miri test --lib -- --exact

which runs every test one by one rather than all at once which let miri actually tell me which test had the leak 🙃

@BoxyUwU BoxyUwU added the A-ECS Entities, components, systems, and events label Jun 7, 2022
@TheRawMeatball
Copy link
Member

Can we get that command added to somewhere in the docs?

@cart cart added this to the Bevy 0.8 milestone Jun 7, 2022
Copy link
Member

@james7132 james7132 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM on the fix.

@alice-i-cecile alice-i-cecile added the P-Unsound A bug that results in undefined compiler behavior label Jun 27, 2022
Copy link
Member

@alice-i-cecile alice-i-cecile left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good eye. Nice to see miri catching these problems for us.

bors r+

bors bot pushed a commit that referenced this pull request Jun 27, 2022
remove `-Zmiri-ignore-leaks` from CI args and fix leaks that miri detects.

The first leak:
```rust
    #[test]
    fn blob_vec_drop_empty_capacity() {
        let item_layout = Layout::new::<Foo>();
        let drop = drop_ptr::<Foo>;
        let _ = unsafe { BlobVec::new(item_layout, Some(drop), 0) };
    }
```
this is because we allocate the swap scratch in blobvec regardless of what the capacity is, but we only deallocate if capacity is > 0

The second leak:
```rust
    #[test]
    fn panic_while_overwriting_component() {
        let helper = DropTestHelper::new();

        let res = panic::catch_unwind(|| {
            let mut world = World::new();
            world
                .spawn()
                .insert(helper.make_component(true, 0))
                .insert(helper.make_component(false, 1));

            println!("Done inserting! Dropping world...");
        });

        let drop_log = helper.finish(res);

        assert_eq!(
            &*drop_log,
            [
                DropLogItem::Create(0),
                DropLogItem::Create(1),
                DropLogItem::Drop(0),
            ]
        );
    }
```
this is caused by us not running the drop impl on the to-be-inserted component if the drop impl of the overwritten component panics

---

managed to figure out where the leaks were by using this 10/10 command
```
cargo --quiet test --lib -- --list | sed 's/: test$//' | MIRIFLAGS="-Zmiri-disable-isolation" xargs -n1 cargo miri test --lib -- --exact
```
which runs every test one by one rather than all at once which let miri actually tell me which test had the leak :upside_down_face:
@alice-i-cecile alice-i-cecile added the S-Ready-For-Final-Review This PR has been approved by the community. It's ready for a maintainer to consider merging it label Jun 27, 2022
@bors
Copy link
Contributor

bors bot commented Jun 27, 2022

Build failed:

@alice-i-cecile
Copy link
Member

test query::tests::self_conflicting_worldquery - should panic ... ok
warning: thread support is experimental and incomplete: weak memory effects are not emulated.

Looks like there's a warning that should be ignored?

@james7132
Copy link
Member

bors retry

bors bot pushed a commit that referenced this pull request Jun 30, 2022
remove `-Zmiri-ignore-leaks` from CI args and fix leaks that miri detects.

The first leak:
```rust
    #[test]
    fn blob_vec_drop_empty_capacity() {
        let item_layout = Layout::new::<Foo>();
        let drop = drop_ptr::<Foo>;
        let _ = unsafe { BlobVec::new(item_layout, Some(drop), 0) };
    }
```
this is because we allocate the swap scratch in blobvec regardless of what the capacity is, but we only deallocate if capacity is > 0

The second leak:
```rust
    #[test]
    fn panic_while_overwriting_component() {
        let helper = DropTestHelper::new();

        let res = panic::catch_unwind(|| {
            let mut world = World::new();
            world
                .spawn()
                .insert(helper.make_component(true, 0))
                .insert(helper.make_component(false, 1));

            println!("Done inserting! Dropping world...");
        });

        let drop_log = helper.finish(res);

        assert_eq!(
            &*drop_log,
            [
                DropLogItem::Create(0),
                DropLogItem::Create(1),
                DropLogItem::Drop(0),
            ]
        );
    }
```
this is caused by us not running the drop impl on the to-be-inserted component if the drop impl of the overwritten component panics

---

managed to figure out where the leaks were by using this 10/10 command
```
cargo --quiet test --lib -- --list | sed 's/: test$//' | MIRIFLAGS="-Zmiri-disable-isolation" xargs -n1 cargo miri test --lib -- --exact
```
which runs every test one by one rather than all at once which let miri actually tell me which test had the leak :upside_down_face:
@bors
Copy link
Contributor

bors bot commented Jun 30, 2022

Build failed:

@alice-i-cecile alice-i-cecile removed the S-Ready-For-Final-Review This PR has been approved by the community. It's ready for a maintainer to consider merging it label Jul 1, 2022
@BoxyUwU BoxyUwU force-pushed the dont_ignore_leaks branch from 22dcb40 to 960a7a8 Compare July 1, 2022 16:01
@BoxyUwU BoxyUwU removed the P-Unsound A bug that results in undefined compiler behavior label Jul 1, 2022
@BoxyUwU BoxyUwU changed the title remove -Zmiri-ignore-leaks from miri CI job fix some memory leaks detected by miri Jul 1, 2022
@alice-i-cecile
Copy link
Member

bors r+

bors bot pushed a commit that referenced this pull request Jul 1, 2022
The first leak:
```rust
    #[test]
    fn blob_vec_drop_empty_capacity() {
        let item_layout = Layout::new::<Foo>();
        let drop = drop_ptr::<Foo>;
        let _ = unsafe { BlobVec::new(item_layout, Some(drop), 0) };
    }
```
this is because we allocate the swap scratch in blobvec regardless of what the capacity is, but we only deallocate if capacity is > 0

The second leak:
```rust
    #[test]
    fn panic_while_overwriting_component() {
        let helper = DropTestHelper::new();

        let res = panic::catch_unwind(|| {
            let mut world = World::new();
            world
                .spawn()
                .insert(helper.make_component(true, 0))
                .insert(helper.make_component(false, 1));

            println!("Done inserting! Dropping world...");
        });

        let drop_log = helper.finish(res);

        assert_eq!(
            &*drop_log,
            [
                DropLogItem::Create(0),
                DropLogItem::Create(1),
                DropLogItem::Drop(0),
            ]
        );
    }
```
this is caused by us not running the drop impl on the to-be-inserted component if the drop impl of the overwritten component panics

---

managed to figure out where the leaks were by using this 10/10 command
```
cargo --quiet test --lib -- --list | sed 's/: test$//' | MIRIFLAGS="-Zmiri-disable-isolation" xargs -n1 cargo miri test --lib -- --exact
```
which runs every test one by one rather than all at once which let miri actually tell me which test had the leak :upside_down_face:
@bors bors bot changed the title fix some memory leaks detected by miri [Merged by Bors] - fix some memory leaks detected by miri Jul 1, 2022
@bors bors bot closed this Jul 1, 2022
bors bot pushed a commit that referenced this pull request Jul 6, 2022
# Objective

When `miri` runs in our build system to detect unsoundness, its output can be very unhelpful, as the tests are all run in parallel.

## Solution

Add a comment documenting the extremely obvious 10/10 command used by @BoxyUwU in #4959.

I've stuck this in the CI file, as it seems like the most obvious place to check when frustrated. I didn't put it  in CONTRIBUTING.md because this is an eldritch abomination and will never be useful to new contributors.
inodentry pushed a commit to IyesGames/bevy that referenced this pull request Aug 8, 2022
The first leak:
```rust
    #[test]
    fn blob_vec_drop_empty_capacity() {
        let item_layout = Layout::new::<Foo>();
        let drop = drop_ptr::<Foo>;
        let _ = unsafe { BlobVec::new(item_layout, Some(drop), 0) };
    }
```
this is because we allocate the swap scratch in blobvec regardless of what the capacity is, but we only deallocate if capacity is > 0

The second leak:
```rust
    #[test]
    fn panic_while_overwriting_component() {
        let helper = DropTestHelper::new();

        let res = panic::catch_unwind(|| {
            let mut world = World::new();
            world
                .spawn()
                .insert(helper.make_component(true, 0))
                .insert(helper.make_component(false, 1));

            println!("Done inserting! Dropping world...");
        });

        let drop_log = helper.finish(res);

        assert_eq!(
            &*drop_log,
            [
                DropLogItem::Create(0),
                DropLogItem::Create(1),
                DropLogItem::Drop(0),
            ]
        );
    }
```
this is caused by us not running the drop impl on the to-be-inserted component if the drop impl of the overwritten component panics

---

managed to figure out where the leaks were by using this 10/10 command
```
cargo --quiet test --lib -- --list | sed 's/: test$//' | MIRIFLAGS="-Zmiri-disable-isolation" xargs -n1 cargo miri test --lib -- --exact
```
which runs every test one by one rather than all at once which let miri actually tell me which test had the leak :upside_down_face:
inodentry pushed a commit to IyesGames/bevy that referenced this pull request Aug 8, 2022
# Objective

When `miri` runs in our build system to detect unsoundness, its output can be very unhelpful, as the tests are all run in parallel.

## Solution

Add a comment documenting the extremely obvious 10/10 command used by @BoxyUwU in bevyengine#4959.

I've stuck this in the CI file, as it seems like the most obvious place to check when frustrated. I didn't put it  in CONTRIBUTING.md because this is an eldritch abomination and will never be useful to new contributors.
james7132 pushed a commit to james7132/bevy that referenced this pull request Oct 28, 2022
The first leak:
```rust
    #[test]
    fn blob_vec_drop_empty_capacity() {
        let item_layout = Layout::new::<Foo>();
        let drop = drop_ptr::<Foo>;
        let _ = unsafe { BlobVec::new(item_layout, Some(drop), 0) };
    }
```
this is because we allocate the swap scratch in blobvec regardless of what the capacity is, but we only deallocate if capacity is > 0

The second leak:
```rust
    #[test]
    fn panic_while_overwriting_component() {
        let helper = DropTestHelper::new();

        let res = panic::catch_unwind(|| {
            let mut world = World::new();
            world
                .spawn()
                .insert(helper.make_component(true, 0))
                .insert(helper.make_component(false, 1));

            println!("Done inserting! Dropping world...");
        });

        let drop_log = helper.finish(res);

        assert_eq!(
            &*drop_log,
            [
                DropLogItem::Create(0),
                DropLogItem::Create(1),
                DropLogItem::Drop(0),
            ]
        );
    }
```
this is caused by us not running the drop impl on the to-be-inserted component if the drop impl of the overwritten component panics

---

managed to figure out where the leaks were by using this 10/10 command
```
cargo --quiet test --lib -- --list | sed 's/: test$//' | MIRIFLAGS="-Zmiri-disable-isolation" xargs -n1 cargo miri test --lib -- --exact
```
which runs every test one by one rather than all at once which let miri actually tell me which test had the leak :upside_down_face:
james7132 pushed a commit to james7132/bevy that referenced this pull request Oct 28, 2022
# Objective

When `miri` runs in our build system to detect unsoundness, its output can be very unhelpful, as the tests are all run in parallel.

## Solution

Add a comment documenting the extremely obvious 10/10 command used by @BoxyUwU in bevyengine#4959.

I've stuck this in the CI file, as it seems like the most obvious place to check when frustrated. I didn't put it  in CONTRIBUTING.md because this is an eldritch abomination and will never be useful to new contributors.
ItsDoot pushed a commit to ItsDoot/bevy that referenced this pull request Feb 1, 2023
The first leak:
```rust
    #[test]
    fn blob_vec_drop_empty_capacity() {
        let item_layout = Layout::new::<Foo>();
        let drop = drop_ptr::<Foo>;
        let _ = unsafe { BlobVec::new(item_layout, Some(drop), 0) };
    }
```
this is because we allocate the swap scratch in blobvec regardless of what the capacity is, but we only deallocate if capacity is > 0

The second leak:
```rust
    #[test]
    fn panic_while_overwriting_component() {
        let helper = DropTestHelper::new();

        let res = panic::catch_unwind(|| {
            let mut world = World::new();
            world
                .spawn()
                .insert(helper.make_component(true, 0))
                .insert(helper.make_component(false, 1));

            println!("Done inserting! Dropping world...");
        });

        let drop_log = helper.finish(res);

        assert_eq!(
            &*drop_log,
            [
                DropLogItem::Create(0),
                DropLogItem::Create(1),
                DropLogItem::Drop(0),
            ]
        );
    }
```
this is caused by us not running the drop impl on the to-be-inserted component if the drop impl of the overwritten component panics

---

managed to figure out where the leaks were by using this 10/10 command
```
cargo --quiet test --lib -- --list | sed 's/: test$//' | MIRIFLAGS="-Zmiri-disable-isolation" xargs -n1 cargo miri test --lib -- --exact
```
which runs every test one by one rather than all at once which let miri actually tell me which test had the leak :upside_down_face:
ItsDoot pushed a commit to ItsDoot/bevy that referenced this pull request Feb 1, 2023
# Objective

When `miri` runs in our build system to detect unsoundness, its output can be very unhelpful, as the tests are all run in parallel.

## Solution

Add a comment documenting the extremely obvious 10/10 command used by @BoxyUwU in bevyengine#4959.

I've stuck this in the CI file, as it seems like the most obvious place to check when frustrated. I didn't put it  in CONTRIBUTING.md because this is an eldritch abomination and will never be useful to new contributors.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-ECS Entities, components, systems, and events
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants