Sound ways to work with self-referential futures

[GoldsteinE]

Hi! I have relatively complex code that passed Miri previously but doesn’t pass it now.

The failing job is:
https://github.com/GoldsteinE/name-it/actions/runs/3594297524/jobs/6052384905

TL;DR: I’m doing something like this:

let x: Fut = async_function();
let y: [MaybeUninit<u8>; size_of::<Fut>()] = transmute(x);
let p = pin!(y); // Causes a retag!
let z: Pin<&mut Fut> = transmute(p);
z.poll();

As far as I understood, this is a general issue with soundness of futures (rust-lang/rust#63818). I’d be grateful for any pointers to things I could do to resolve this issue in my case on my side.

[GoldsteinE]

I’ve managed to create minimal-ish reproducible example:
https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=81a8e59973f1870b4d90e0c93ac24121

[RalfJung]

Which was the last version of Miri where this still worked?

[GoldsteinE]

That’s an interesting question, I’ll check it.

[GoldsteinE]

Ok, so, turns out this particular snippet:

1. Failed before my pipelines started failing
2. Fails without transmute shenanigans or any unsafe code at all: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=a408d5a73049f532280398b2a8559b01

As this reproduces without unsafe code, this is a bug either in futures, std or Miri.

[GoldsteinE]

My pipeline (test futures-channel-channel from the name-it repo) passes on miri 0.1.0 (c1a859b 2022-11-10) and fails on miri 0.1.0 (42325c5 2022-11-11)

[RalfJung]

Okay, it's probably field-retagging-by-default then that broke this.

Looking at this code, my first impression is that this is unsound since Arr is Unpin. Currently the alias handling of mutable references is tied to the Unpin trait. This is a temporary hack and not a stable guarantee (see this discussion for work towards a proper solution), but in this case probably adding Unpin would both fix that soundness bug and make Miri stop complaining?

EDIT: Okay, that's not it. Looks like you found an even simpler example anyway.

[GoldsteinE]

It seems like there’re two issues: one that affects futures and another that only affects transmute hack.

name-it code handles Unpin correctly, so that's not it

[GoldsteinE]

Or maybe that's a single issue that name-it just masked for older Miri versions. Either way, those issues seem to be very related.

[RalfJung]

Closing in favor of #381, which describes the core problem more precisely. Thanks for the report, which lead to be discovering this underlying problem!

Or maybe that's a single issue that name-it just masked for older Miri versions. Either way, those issues seem to be very related.

Yeah that is possible, but pretty hard to say until we figure out #381.