Stop using Reveal::All before borrowck

[lcnr]

Using Reveal::All before borrowck is incorrect, both because that means that we peek into opaque types at a point which is still observable by the user, and more importantly, because peaking into opaque types causes query cycles if it happens this early.

I completely remove param_env.with_reveal_all_normalized. This function changes the context of a param env from "I am used for userfacing stuff" to "nm, actually I am not", which should always be incorrect or at least dubious enough to require an explicit impl with its own comment.

This is a breaking change in a few subtle ways, so it definitely needs a crater run. Still have to add some more tests where this is impactful.

r? @rust-lang/types @rust-lang/wg-const-eval

[rustbot]

Some changes occurred to MIR optimizations

cc @rust-lang/wg-mir-opt

Some changes occurred to the CTFE / Miri engine

cc @rust-lang/miri

[lcnr]

woops, debug code 😁

[lcnr]

@bors try

[bors]

⌛ Trying commit 734d9d3 with merge 9b4fd9cf46c3c08bce1c5ed60b929af1c4aaed01...

[lcnr]

ctfe returing TooGeneric for opaque types in otherwise concrete constants causes ICE rn, have to fix this. changing to draft

[RalfJung]

What's a "constant with a hidden value"?

[lcnr]

that error message is bad but i wasn't able to think of a better one.

This can be hit in two cases:

• when encountering an opaque type during ctfe
• when encountering a default associated item (with feature(specialization))

we should definitely change that message before merging this PR but I am not yet sure about the best way to do so

[lcnr]

@bors try @rust-timer queue

[rust-timer]

Awaiting bors try build completion.

@rustbot label: +S-waiting-on-perf

[bors]

⌛ Trying commit 41066e2 with merge 0004f98fdd309b1940390200ae28913f7fbd736f...

[bors]

☀️ Try build successful - checks-actions
Build commit: 0004f98fdd309b1940390200ae28913f7fbd736f (0004f98fdd309b1940390200ae28913f7fbd736f)

[rust-timer]

Queued 0004f98fdd309b1940390200ae28913f7fbd736f with parent 0568b0a, future comparison URL.

[rustbot]

Some changes occurred to the CTFE / Miri engine

cc @rust-lang/miri

[lcnr]

@craterbot check

[craterbot]

👌 Experiment pr-101478 created and queued.
🤖 Automatically detected try build 0004f98fdd309b1940390200ae28913f7fbd736f
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[lcnr]

@rfcbot fcp cancel

[rfcbot]

@lcnr proposal cancelled.

[lcnr]

I lowered some of the changes to a future compat lint which should hopefully allow us to merge this PR quickly without figuring out our exact goals for opaque types.

---

This pull request changes the compiler to stop using Reveal::All at any point until borrowck is completed. Using Reveal::All means that it is possible to inspect

• the underlying type of opaque types
• the value of a "defaulted" associated item (requires feature(specialization))

In cases where this is known to break existing code, we retry the failing code with Reveal::All, emitting a future compatibility lint in case that succeeds. The cases where we emit a future compatibility lint are const evaluation for type system constants and transmute checking. We currently do not retry for patterns, this is a breaking change, even if not discovered using crater. See in-ctfe/match-arm-exhaustive.rs for an example. In case this causes an actual regression, we could extend the future compatibility lint to this area as well.

This PR is necessary as using Reveal::All before borrowck is incorrect and blocks work on implied bounds. Even if we want to fully embrace opaque types in these places - which is not trivial, see #101478 (comment), the approach then should not use Reveal::All. So even then, this PR will be "correct". I do not want to wait with my implied bounds work until this is figured out.

A crater run found 2 impacted crates:

mijit

Transmute in a test between an opaque type and the underlying concrete type. Opened PR fixing this in apt1002/mijit#51.

name-it

This crate cannot really be fixed to avoid the lint, as it fundamentally requires this pattern:

use std::mem;
fn returns_opaque() -> impl Sized {
    0u8
}

struct NamedOpaqueType {
    data: [mem::MaybeUninit<u8>; size_of_fut(returns_opaque)]
}

const fn size_of_fut<FUT>(x: fn() -> FUT) -> usize {
   mem::size_of::<FUT>()
}

@rfcbot fcp merge

[rfcbot]

Team member @lcnr has proposed to merge this. The next step is review by the rest of the tagged team members:

• @compiler-errors
• @cramertj
• @jackh726
• @joshtriplett
• @lcnr
• @nikomatsakis
• @oli-obk
• @pnkfelix
• @scottmcm
• @spastorino

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[rust-log-analyzer]

The job x86_64-gnu-llvm-13 failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

doc tests for: /checkout/src/doc/rustc/src/tests/index.md
doc tests for: /checkout/src/doc/rustc/src/what-is-rustc.md
 finished in 2.034 seconds
Generating lint docs (x86_64-unknown-linux-gnu)
error: failed to test example in lint docs for `hidden_type_of_opaque_types_in_type_system` in /checkout/compiler/rustc_lint_defs/src/builtin.rs:3985: lint docs should start with the text "The `hidden_type_of_opaque_types_in_type_system` lint" to introduce the lint
This error was generated by the lint-docs tool.
This tool extracts documentation for lints from the source code and places
This tool extracts documentation for lints from the source code and places
them in the rustc book. See the declare_lint! documentation
https://doc.rust-lang.org/nightly/nightly-rustc/rustc_lint_defs/macro.declare_lint.html


To re-run these tests, run: ./x.py test --keep-stage=0 src/tools/lint-docs
The --keep-stage flag should be used if you have already built the compiler

Build completed unsuccessfully in 0:33:01

[bors]

☔ The latest upstream changes (presumably #98588) made this pull request unmergeable. Please resolve the merge conflicts.

[RalfJung]

This crate cannot really be fixed to avoid the lint, as it fundamentally requires this pattern:

Wasn't the plan to allow querying the size of an opaque type during CTFE? Or does that not work / lead to the same query cycles?

Similar to the second example of my previous comment: playground. With different variations you may end up getting ICE at some other places here. Generally you have non-wf type at places which can't deal with them.

I am trying to understand why this is non-wf. We have a usize: CDefault assumption so in that context, everything should be WF, right? We just must not try to normalize that instance, just like we would if this was a generic T: CDefault.

[lcnr]

Wasn't the plan to allow querying the size of an opaque type during CTFE? Or does that not work / lead to the same query cycles?

Yes, but I am not comfortable with that change because it requires LayoutOf to work with non-wf types to be correct. My hope was that going with a future compat warning and explicitly stating "please show us your usecase, some of the code that's currently will be allowed without lints in the future" allows me to merge this PR without having to think about that because its blocking other work which is far more important to me.

I am trying to understand why this is non-wf. We have a usize: CDefault assumption so in that context, everything should be WF, right? We just must not try to normalize that instance, just like we would if this was a generic T: CDefault.

(talking about this example) Yeah, that should actually be fine, bad example 😅 this currently causes an ICE because Reveal::All erases global bounds which isn't really correct considering how it is used 😁

Only revealing the opaque type in the value - and not the param_env - is sound as long as there are no opaque types in the param_env, which is the case there. So as long as we use a conversion which doesn't drop trivial bounds, this should actually be alright 🤔

Doing that with opaque types in where-bounds does result in non-wf types like this previous example.

[RalfJung]

Doing that with opaque types in where-bounds does result in non-wf types like this previous example.

You'll have to spell this out in more detail for me, I don't see it.^^ (Also is there a better place to discuss this?)

In a context where Foo implements Trait, what is wrong with <Foo as Trait>::ASSOC? Sure that context might be inconsistent, but no more inconsistent than a context that has usize: CDefault (as in the first example) Box<_>: Copy or other where bounds that I can add to a function.

So as long as we use a conversion which doesn't drop trivial bounds, this should actually be alright thinking

I don't know what makes a bound trivial, but if that includes trivially false bounds like usize: CDefault then yes dropping them sounds like a big disaster to me -- it can turn a well-formed context into an ill-formed one. Dropping a truly trivial bound like usize: Copy should be fine though. (Speaking just from general type system intuition here without any knowledge about how this works in Rust.)

FWIW this does sound like the kind of conceptual question that I mentioned in our discussion a month or so ago, the kind of question that would benefit from some theory formalization and drafting.

[lcnr]

I don't know what makes a bound trivial, but if that includes trivially false bounds like usize: CDefault then yes dropping them sounds like a big disaster to me

jup, trivially false bounds are the ones which are dangerous to drop. We currently drop all global ones I think, i.e. bounds which don't refer to any generic parameters 🤔

[lcnr]

Also is there a better place to discuss this?

moved this to zulip: https://rust-lang.zulipchat.com/#narrow/stream/144729-t-types/topic/.60Reveal.3A.3AAll.60.20borrowck.20yeet

[lcnr]

talked about this with @nikomatsakis and @oli-obk in a recent meeting and we're still unhappy with the current solution. Going to experiment with a different approach where we lazily reveal opaque types on use and will open a new PR for that.