Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Aliasing rules are broken for closures #17403

Closed
kvark opened this issue Sep 20, 2014 · 11 comments
Closed

Aliasing rules are broken for closures #17403

kvark opened this issue Sep 20, 2014 · 11 comments
Labels
A-type-system Area: Type system
Milestone

Comments

@kvark
Copy link
Contributor

kvark commented Sep 20, 2014

This compiles and runs:

let mut v = vec![0i];
let f = |i: uint| v.get_mut(i);
let x = f(0);
let y = f(0);
(*x,*y)

x and y are not supposed to live at the same time. Results in an UB (in a more complex case)
Rust:

rustc 0.12.0-nightly (af3889f 2014-09-18 21:20:38 +0000)

@kvark
Copy link
Contributor Author

kvark commented Sep 20, 2014

Initially discovered in rust-voxel. CC @aepsil0n

@thestinger thestinger added I-wrong A-type-system Area: Type system labels Sep 20, 2014
@alexcrichton
Copy link
Member

Nominating.

@thestinger
Copy link
Contributor

Aren't the old closures going to be removed as part of implementing unboxed closures? Unless the problem exists for regular types / trait objects, I think it will solve itself.

milibopp added a commit to milibopp/rust-voxel that referenced this issue Sep 20, 2014
Related to rust-lang/rust#17403. Should not have
compiled in the first place.
@nikomatsakis
Copy link
Contributor

cc me

@nikomatsakis
Copy link
Contributor

I suspect this is related to how we handle lifetimes in closures, which is one of the areas where the code doesn't properly treat closures in the desugared way we'd like to. This is good because I've been looking for breakage that results from this, so I'm glad to have some concrete examples.

In particular I suspect the problem is that we currently consider the closure region bound to be its enclosing expression, which is patently wrong (#3696). There is a better algorithm for handling this described in a comment but I haven't gotten around to implementing it. I was waiting to look into this until the dust form @pcwalton's SEME region work settled, which seems to have happened. So now is the time, I guess. :)

@pnkfelix
Copy link
Member

Assigning P-backcompat-lang, 1.0.

@zwarich
Copy link

zwarich commented Oct 5, 2014

Here is an example of the same issue happening with unboxed closures:

#![feature(unboxed_closures, overloaded_calls)]

fn main() {
    let mut x = 0u;
    let mut f = |&mut:| &mut x;
    let _y = f();
    let _z = f();
}

@zwarich
Copy link

zwarich commented Oct 5, 2014

The equivalent desugared program

#![feature(overloaded_calls)]

struct F<'a> {
    x: &'a mut uint
}

impl<'a> Fn<(), &'a mut uint> for F<'a> {
    #[rust_call_abi_hack]
    fn call(&self, _: ()) -> &'a mut uint {
        self.x
    }
}

fn main() {
    let mut x = 0u;
    let f = F { x: &mut x };
    let _y = f();
    let _z = f();
}

gives an error

test.rs:10:9: 10:15 error: cannot infer an appropriate lifetime for automatic coercion due to conflicting requirements
test.rs:10         self.x
                   ^~~~~~
test.rs:9:5: 11:6 note: consider using an explicit lifetime parameter as shown: fn call(&'a self, _: ()) -> &'a mut uint
test.rs:9     fn call(&self, _: ()) -> &'a mut uint {
test.rs:10         self.x
test.rs:11     }

This suggests a fix for the issue by appropriately constraining the lifetime parameters of borrows in unboxed closures so that the constraints have no solution in this case.

@bkoropoff
Copy link
Contributor

My grasp of the region code is tenuous, but I think the interesting bit that causes the desugared version to be rejected is in link_reborrowed_region, where the mutability of F::x causes it to recurse and link regions higher up in the deref path to the region of the reborrow. In particular, this causes the region of self.x to become bounded by that of self, which creates a set of constraints with no solution. If F::x is changed to an immutable ref and the rest of the code updated accordingly, the extra link is not created and the code compiles, but of course this is perfectly sound.

It should be possible to make it handle the upvar in a similar way. I've been experimenting with a quick hack, and it seems to fix the simple unsound case above and pass unit tests, but it doesn't deal with the fact that the mutability of an upvar can be changed by later borrows or nested closures that have yet to be processed. It also produces really poor diagnostics.

Can anyone with deeper knowledge of regionck confirm if I'm on the right track here?

@nikomatsakis
Copy link
Contributor

@bkoropoff I will try to take a deeper look tomorrow (I'm traveling and haven't had a chance to dig into this). However, your analysis seems possibly reasonable.

bors added a commit that referenced this issue Oct 9, 2014
This fixes a soundness problem where `Fn` unboxed closures can mutate free variables in the environment.
The following presently builds:

```rust
#![feature(unboxed_closures, overloaded_calls)]

fn main() {
    let mut x = 0u;
    let _f = |&:| x = 42;
}
```

However, this is equivalent to writing the following, which borrowck rightly rejects:

```rust
struct F<'a> {
    x: &'a mut uint
}

impl<'a> Fn<(),()> for F<'a> {
    #[rust_call_abi_hack]
    fn call(&self, _: ()) {
        *self.x = 42; // error: cannot assign to data in a `&` reference
    }
}

fn main() {
    let mut x = 0u;
    let _f = F { x: &mut x };
}
```

This problem is unique to unboxed closures; boxed closures cannot be invoked through an immutable reference and are not subject to it.

This change marks upvars of `Fn` unboxed closures as freely aliasable in mem_categorization, which causes borrowck to reject attempts to mutate or mutably borrow them.

@zwarich pointed out that even with this change, there are remaining soundness issues related to regionck (issue #17403).  This region issue affects boxed closures as well.

Closes issue #17780
bkoropoff added a commit to bkoropoff/rust that referenced this issue Oct 16, 2014
- Unify the representations of `cat_upvar` and `cat_copied_upvar`
- In `link_reborrowed_region`, account for the ability of upvars to
  change their mutability due to later processing.  A map of recursive
  region links we may want to establish in the future is maintained,
  with the links being established when the kind of the borrow is
  adjusted.
- When categorizing upvars, add an explicit deref that represents the
  closure environment pointer for closures that do not take the
  environment by value.  The region for the implicit pointer is an
  anonymous free region type introduced for this purpose.  This
  creates the necessary constraint to prevent unsound reborrows from
  the environment.
- Add a note to categorizations to make it easier to tell when extra
  dereferences have been inserted by an upvar without having to
  perform deep pattern matching.
- Adjust borrowck to deal with the changes.  Where `cat_upvar` and
  `cat_copied_upvar` were previously treated differently, they are
  now both treated roughly like local variables within the closure
  body, as the explicit derefs now ensure proper behavior.  However,
  error diagnostics had to be changed to explicitly look through the
  extra dereferences to avoid producing confusing messages about
  references not present in the source code.

Closes issue rust-lang#17403.  Remaining work:

- The error diagnostics that result from failed region inference are
  pretty inscrutible and should be improved.

Code like the following is now rejected:

    let mut x = 0u;
    let f = || &mut x;
    let y = f();
    let z = f(); // multiple mutable references to the same location

This also breaks code that uses a similar construction even if it does
not go on to violate aliasability semantics.  Such code will need to
be reworked in some way, such as by using a capture-by-value closure
type.

[breaking-change]
bkoropoff added a commit to bkoropoff/rust that referenced this issue Oct 16, 2014
bors added a commit that referenced this issue Oct 16, 2014
…sakis

This PR is based on #17784, which fixes closure soundness problems in borrowck.  Only the last two commits are unique to this PR.

My understanding of regionck is still evolving, so I'm not sure if this is the right approach.  Feedback is appreciated.

- In `link_reborrowed_region`, we account for the ability of upvars to
  change their mutability due to later processing.  A map of recursive
  region links we may want to establish in the future is maintained,
  with the links being established when the mutability of the borrow
  is adjusted.
- When asked to establish a region link for an upvar, we link it to
  the region of the closure body.  This creates the necessary
  constraint to stop unsound reborrows from the closure environment.

This partially (maybe completely) solves issue #17403.  Remaining work:

- This is only known to help with by-ref upvars.  I have not looked at
  by-value upvars yet to see if they can cause problems.
- The error diagnostics that result from failed region inference are
  pretty inscrutible.
@aturon aturon mentioned this issue Oct 16, 2014
47 tasks
bkoropoff added a commit to bkoropoff/rust that referenced this issue Oct 17, 2014
- Unify the representations of `cat_upvar` and `cat_copied_upvar`
- In `link_reborrowed_region`, account for the ability of upvars to
  change their mutability due to later processing.  A map of recursive
  region links we may want to establish in the future is maintained,
  with the links being established when the kind of the borrow is
  adjusted.
- When categorizing upvars, add an explicit deref that represents the
  closure environment pointer for closures that do not take the
  environment by value.  The region for the implicit pointer is an
  anonymous free region type introduced for this purpose.  This
  creates the necessary constraint to prevent unsound reborrows from
  the environment.
- Add a note to categorizations to make it easier to tell when extra
  dereferences have been inserted by an upvar without having to
  perform deep pattern matching.
- Adjust borrowck to deal with the changes.  Where `cat_upvar` and
  `cat_copied_upvar` were previously treated differently, they are
  now both treated roughly like local variables within the closure
  body, as the explicit derefs now ensure proper behavior.  However,
  error diagnostics had to be changed to explicitly look through the
  extra dereferences to avoid producing confusing messages about
  references not present in the source code.

Closes issue rust-lang#17403.  Remaining work:

- The error diagnostics that result from failed region inference are
  pretty inscrutible and should be improved.

Code like the following is now rejected:

    let mut x = 0u;
    let f = || &mut x;
    let y = f();
    let z = f(); // multiple mutable references to the same location

This also breaks code that uses a similar construction even if it does
not go on to violate aliasability semantics.  Such code will need to
be reworked in some way, such as by using a capture-by-value closure
type.

[breaking-change]
bkoropoff added a commit to bkoropoff/rust that referenced this issue Oct 17, 2014
bors added a commit that referenced this issue Oct 17, 2014
…sakis

This PR is based on #17784, which fixes closure soundness problems in borrowck.  Only the last two commits are unique to this PR.

My understanding of regionck is still evolving, so I'm not sure if this is the right approach.  Feedback is appreciated.

- In `link_reborrowed_region`, we account for the ability of upvars to
  change their mutability due to later processing.  A map of recursive
  region links we may want to establish in the future is maintained,
  with the links being established when the mutability of the borrow
  is adjusted.
- When asked to establish a region link for an upvar, we link it to
  the region of the closure body.  This creates the necessary
  constraint to stop unsound reborrows from the closure environment.

This partially (maybe completely) solves issue #17403.  Remaining work:

- This is only known to help with by-ref upvars.  I have not looked at
  by-value upvars yet to see if they can cause problems.
- The error diagnostics that result from failed region inference are
  pretty inscrutible.
@bkoropoff
Copy link
Contributor

This bug can now be closed

kennytm added a commit to kennytm/qrcode-rust that referenced this issue Oct 23, 2014
Restructure Canvas::draw_data(), since change in rust-lang/rust#17403 no
longer allows returning a mutable ref directly.
lnicola pushed a commit to lnicola/rust that referenced this issue Jul 11, 2024
Use proper `ImplTraits` in `insert_inference_vars_for_impl_trait`

Fixes rust-lang#17199 and fixes rust-lang#17403

In the previous implementation, I passed `rpits` as a function parameter and used `idx` of `ImplTraitId` for indexing `ImplTrait`.

https://github.com/rust-lang/rust-analyzer/blob/4e836c622a7bdab41be8e82733dd9fe40af128b2/crates/hir-ty/src/infer.rs#L881-L887

But that `idx` is rather a "local" one, so in the cases like mentioned issues, the async function that can be expanded roughly as

```rust
type TypeAlias = impl Something;
fn expanded_async() -> impl Future<Output = TypeAlias> { ... }
```

there are two bundles of `ImplTraits`; one for the `impl Future` and the other one for `TypeAlias`.
So using `idx` with `rpits` returns `ImplTrait` for `impl Future` even if we are asking for `TypeAlias` and this caused a stack overflow.

This PR is a fix for that implementation miss 😅
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-type-system Area: Type system
Projects
None yet
Development

No branches or pull requests

8 participants