More implicit bounds (?Sized, ?DynSized, ?Move)

[kennytm]

The concept of implicit bound was introduced together with dynamic sized type. While having a : Sized bound is more specialized than no bound at all, we expect that : Sized is much more common, and thus should be present by default. We introduced the : ?Sized syntax in #490 to opt-out of this default bound. The syntax : ?Sized is called "relaxed bound".

Later, new RFCs and PRs tried to introduce more traits like Sized which the common case is "this should be implemented", e.g.

• Immovable types  #1858 and Immovable types prototype where the Move trait is builtin rust#44917 introduced the ?Move bound, since most types do not contain internal references and thus can be freely moved around.
• Opaque Data structs for FFI #1993 and Add DynSized trait (rebase of #44469) rust#46108 introduced the ?DynSized bound, since most types have known size and alignment at runtime.
• https://gankro.github.io/blah/linear-rust (semiseriously) introduced the ?Leave bound, since most types can be drop'ed.

These traits themselves are often necessary at language level beyond trait bounds, e.g. Move is needed for immovable generators (for lack of a better solution), DynSized is needed to reject struct tails without known alignment, and Leave is needed to support linear type.

However, the expansion of implicit bounds experienced push back from lang team due to ergonomic cost, that

• ?Sized itself being a "negative feature" confuses users, adding ?Move and ?DynSized will only make the situation worse,

• introducing new relaxed bound means downstream packages will need to reevaluate every API to see if adding : ?Trait makes sense, and this needs to be done for every new relaxed bound.

• the necessity of Move and DynSized is orthogonal to whether they need to be default.

• the backward-compatibility may be a lie 🍰 — Relaxing the bounds in associated type, in particular FnOnce::Output, means the user of the trait will get less promises, which is a breaking change:

  let should_be_movable: Closure::Output = closure();
  let but_it_errored = should_be_movable;

  Essentially, the bounds on an associated type cannot be added (which breaks implementers) or removed (which breaks users).

So the questions are,

• Should we embrace ?Trait and allow language to grow more similar traits, despite the stated problems above?
• If we decide to stop at ?Sized, what else should we do regarding Move and DynSized?
• Would there be a balanced solution in the middle of the two extremes?

---

I am filing this issue here to Move the discussion from those two different PRs to a potentially more proper place, as it seems having a non-Sized relaxed bound itself should require more discussion around the language design.

[est31]

?Sized itself being a "negative feature" confuses users, adding ?Move and ?DynSized will only make the situation worse,

I think it will make it better, because sized is right now one single exception. If there are more of it, it becomes more of a rule.

[mikeyhew]

@kennytm I think some of your links go to the wrong comment... the last three all go to the same comment by @arielb1

[Manishearth]

Relevant: https://gankro.github.io/blah/linear-rust/

(I really would like more implicit bounds, however I am concerned about making things too complicated)

[Manishearth]

Worth noting that while Sized currently is not implied in many cases (e.g. on a trait) we probanly should make things like Move get implied everywhere, and require explicit where Self: ?Move to opt out.

[eddyb]

@Manishearth I'd expect it to be written more often as trait Foo: ?Move.

[kennytm]

@mikeyhew thanks, I’ll update it after the holiday.

Edit: Fixed links.

[crlf0710]

Is there a "induction based" solution? I mean, what if

trait Foo {
    fn foo1(&self);
    fn foo(self);
}

automatically implies

trait Foo : ?Sized + ?DynSized + ?Move + ... {
    fn foo1(&self);
    fn foo(self) where Self: Move;
}

The (naive) idea is that wherever 'move operation' is implied in the method signature, an implicit 'where clause' requiring the type is Move is automatically added on a method basis. The existence of Move bound should automatically assume Sized and DynSized, i think...

On the concrete type side, make these auto trait. So they're automatically inferred just like Send and Sync...

I wonder if this can work?

[withoutboats]

In addition to the concerns @kennytm identified in the original post, it appears that adding additional ?Traits is not backwards compatible - in the case of ?Move this is a very serious problem for the proposal because of how it interacts with the return values of closures.

In terms of usability, the deep problem with these traits is not that they're weird or confusing, but that by their nature they make some question into everyone's problem. Making more of them does not reduce the problem, it multiplies it.

Today, it is everyone's problem to know if they really need to know the size of this type at compile time. Tomorrow, we would have everyone also worry about whether they need to know the size at runtime, whether they need to be able to move it after they reference it, and whatever else anyone might propose to use these kinds of traits for.

I'm pretty certain we can't add any additional traits like this, and both DynSized and Move need to discover an alternative solution.

Fortunately, I think this has been a situation where we have a sledge hammer, and people have picked it up and swung it. That is, both DynSized and Move feel like very brute force attempts to solve the problem they are aiming at, and with more finesse we could develop more balanced and targeted solutions to these problems.

[arielb1]

Not all traits are born equal

DynSized feels less terrible than Move, because of trait Sized: DynSized - that's it, it only makes sense to consider Sized vs. DynSized when you're adding a ?Sized bound anyway.

You need a T: ?Sized + DynSized type exactly when you are handling T behind a safe smart pointer. Code that doesn't handle safe smart pointers generically doesn't need to know about the distinction, which is a small amount of library code.

In fact, the main place that non-data-structure code cares about DynSized vs. Sized is in impls - e.g. impl<T: ?Sized + Foo> Foo for Box<T>. In that case, the impl itself doesn't require anything but ?DynSized, and with implied bounds, the user could always type T: ?DynSized and add other bounds when the compiler prompts them (or, with implied bounds, never).

Move is basically the opposite - as the C++ STL shows, many data structures and algorithms can be made uglier while working with !Move types, which means that !Move might cause a split of the data-structure ecosystem.

On the other hand, the payoff of non-movable types is fairly large - easy safe self-borrowing generators! Because generators sit at the corner of complicated, high-performance, hostile-data-handling code, making them usable safely is a big safety and quality-of-life improvement, possibly even worth an ugly "data structure ecosystem split".

NEEDSRESEARCH

BTW, I didn't research this, but how bad is the issue with Move closure return types? Today stable code has to use them like T, F: FnOnce() -> T, where T has an implicit Move bound.

[arielb1]

Moving is a big decision

Actually, taking a second look, it might seem that we make trait Move: DynSized, then DynSized might "cancel out" the collateral badness of Move.

That's it, you might think you could have the following mental model:

1. If you are working with an "ordinary" T, then you probably want T to be both Move and Sized, so you can have an unannotated type parameter - in the worst case, the user can always get that by sacrificing some performance and adding a Box.
2. If you are "delegating" and don't care what T is, you can require T: ?DynSized and let implied bounds (or bound copy-pasting) do the rest.
3. If you are writing a generic container, then you should know what you are doing and put the right bounds.

The reason this is not as perfect as it looks like is that there are actually not that much non-delegating uses of Sized types: if you pass a type by-value to a function, or put in an enum or a middle field of a struct, then that type must always be Sized! Because most uses of generic types end up doing that, almost all non-"relay" type parameters are required to be Sized.

On the other hand, exposing a reference to a type and then moving it is a much rarer thing - there are many places where T: ?Move makes sense, or where T: Move makes sense, or where both make some amount of sense and the programmer has to choose what they want to implement and/or guarantee.

The "moving is a big decision" problem is a far bigger thing than worries about backwards-compatibility. It makes the mental model for anyone who is writing generic code more complex - do I want to commit to allowing T to be movable? Might I ever want to move it around? These are not obvious questions, and their implications might be even less obvious.

On the other hand, that makes !Move again far more strategic than closing a corner case where size_of_val panics/return None/whatever does. Being able to work with immovable data is a very powerful and often requested feature for high-performance code. However, if we can find some other 95% solution to that that does not force every programmer to make a decision, that would be much better.

[mikeyhew]

So, I just posted something over in internals https://internals.rust-lang.org/t/pre-erfc-lets-fix-dsts/6663. There's a lot of stuff in that post, but I had an idea as I was writing that that could help with the issue of ? traits. I'll try to distill it down to the relevant bits.

Part of that proposal involves adding a Referent trait, which every type implements. For reference, here is a list of traits and their supertraits:

Referent
DynAligned: Referent
DynSized: DynAligned
Aligned: DynAligned
Sized: Aligned + DynSized

I've taken out SizeFromMeta and AlignFromMeta to make things a bit simpler.

So the idea is, Sized is still a default trait bound, but as soon as one of the above traits is listed as a trait bound, the default bound is removed. So instead of T: ?Sized, you write T: DynSized, and instead of T: ?DynSized you write T: Referent.

Here is an example from the post I linked above:

// `T: Referent` makes it clear that `T` is any type that can be pointed to
fn assemble<T: Referent>(data: *const (), meta: T::Meta) -> *const T;

I think this makes things easier to reason about. I know @withoutboats was saying that that's not the issue here, but I have a hunch that the ?Trait syntax really does make things confusing, and life would be a lot better without them. When you write ?DynSized, it means "we won't necessarily be able to get the size or alignment of a value at runtime", whereas if you write Referent it means "something that can be pointed to, and nothing more". The argument for DynSized in favour of ?Sized is even stronger, I think. When you see T: DynSized, you think, "Oh, it can be dynamically-sized".

It's altogether a reduction in cognitive load, because you stop having to worry about which trait bounds are there that you can't see. (And then if you didn't write any of those traits, Sized is still there by default, so you don't have to think about it unless you're doing something fancy where it's not Sized)

I'm not quite sure how this works with Move. I guess it could be included in the list of traits that removes the default bound... that way if you write T: Move, it implies T: DynSized and removes the T: Sized bound.

[withoutboats]

@mikeyhew I think (for the specific case of DynSized) you may have something here, because we have this hierarchical relationship with Sized. It's definitely worth considering whether being able to phrase these loosening bounds in positive terms makes them easier to understand (I'm inclined to think it does).

I'm a little confused about how T: ?DynSized could be T: Referent, but maybe I need to read your thread more closely.

However, I'm doubtful that this generalizes to any ?Trait, as opposed to working for DynSized because it has this hierarchical relationship with Sized already.

[eddyb]

T: ?DynSized being usually written as T: Referent implies !Referent types might exist?

[mikeyhew]

@eddyb

T: ?DynSized being usually written as T: Referent implies !Referent types might exist?

No. I mean, theoretically !Referent types could exist – that could be the case for types that can only be used at the type level, like byteorder::BigEndian which I mentioned in a footnote in the linked thread. But my assumption was that !Referent types do not exist.

The point of writing T: Referent is that it opts out the default Sized or DynSized bounds, and it's supposed to be better than writing T: ?DynSized because it says what you can do with T, rather than what you can't do.

@withoutboats

I'm a little confused about how T: ?DynSized could be T: Referent, but maybe I need to read your thread more closely.

?DynSized implies Referent because, after the default DynSized bound is removed, Referent is all that is left (in this scenario, DynAligned is not a default bound). And Referent implies ?DynSized because it causes the default Sized or DynSized bound to be removed.

Does that clear things up a bit, or am I just making things worse?

[Ixrec]

I definitely understand the appeal of writing a bound like T: LiterallyAnyType rather than a bound like T: DoesntHaveToBeDynSized since the latter only implies the former if you're aware that DynSized is...the most implicit bound? The bound that applies to a larger subset of Rust types than any other bound? The bottom/top of the autoimpl'd trait hierarchy? None of those seem technically correct, but that's probably part of why a "positive-looking bound" like T: LiterallyAnyType seems so much more straightforward.

I think it's confusing only because Referent does not sound like it means LiterallyAnyType. It sounds like it exists solely for the custom DST use case and should never be mentioned in code that's not specifically messing with custom DSTs. If the goal is instead for all Rust types to implement those possibly-fat pointer create/split methods (is there any reason we wouldn't want this?), we should probably change the strawman name for it to Anything or AnyType or Unrestricted or something like that.

[eddyb]

The only way I see to have all types implement such a trait is use Meta = ! to indicate you can't have a pointer / reference pointing to that type.

[comex]

So we have a hierarchy: ?DynSized < DynSized < Move < Sized.

You need a T: ?Sized + DynSized type exactly when you are handling T behind a safe smart pointer. Code that doesn't handle safe smart pointers generically doesn't need to know about the distinction, which is a small amount of library code.

Can we avoid the DynSized bound on smart pointers?

For example, Box uses size_of_val (and min_align_of_val) in its destructor, to know the right memory layout to pass to the deallocation function. size_of_val will now require DynSized, so the obvious solution would be giving Box's type parameter a ?Sized + DynSized bound. But instead, we could say that Box<NotDynSized> is a valid type, just one which you can never legitimately obtain an instance of. After all, non-DynSized types are meant to be used for FFI, for cases where you receive a pointer from external code which you can pass back to it, but know nothing about its referent. It doesn't make sense to claim you have a boxed allocation for such a type - how would you have allocated it without knowing the size?

This implies that the following should be illegal for unsafe code:

1. Implementing Unsize<U> for T if U: !DynSized and there is some way to obtain a Box<T>. In other words, coercions can't just forget about size.
2. With custom allocators, synthesizing Box<T, MyAllocator> where T: !DynSized. That is, in a world where stuffing your own pointer into a Box<T, MyAllocator> is considered legitimate as long as you know that MyAllocator can handle freeing that pointer. I don't know whether we want to allow that, but even if we do - the allocator trait's 'free' methods all take a size/alignment, so by definition, if a type has no size, you can't reasonably say that MyAllocator can handle freeing it. So this isn't really an additional restriction.

I'm not worried about 2, but 1 seems like a somewhat arbitrary restriction.

However, the benefit would be simplifying the hierarchy: we go from four possibilities to three, at least as far as the vast majority of generic code needs to care.

Perhaps we should bring that down to two, and say: even if it becomes possible to move DSTs, most generic code shouldn't bother supporting that because it's not very useful. So the recommendation would be:

• If you're handling T by value, then you want Sized (same as today).
• For everything else - including&T, *mut T, Box<T>, etc., as well as phantom type parameters where you never handle an instance - you probably want ?DynSized (changed from ?Sized today).

edit: ?Sized becomes essentially deprecated: almost all existing uses should migrate to ?DynSized, and some epoch could change the default ?Sized on trait self types to ?DynSized. Ironically, it would become an appropriate bound when you are handling a type by value - basically the opposite of today - if you want to allow DSTs for maximum flexibility. Since ?Sized is an awkward way to express this, you should be encouraged to write ?DynSized + Move (or something like that).

[mikeyhew]

@comex

So we have a hierarchy: ?DynSized < DynSized < Move < Sized

Having Move be a supertrait for Sized is something I've thought about, but there are potential types that are Sized + ?Move: for example, immovable generators or self-borrowing structs. Move definitely requires DynSized – you can't move something unless its size (and alignment, presumably) is known.

But Sized + ?Move is probably rare enough that we can just have people write exactly that for it.

Sized means Sized + Move
Move means Move (no Sized, and implies DynSized)
Sized + ?Move means Sized (without Move)
DynSized means DynSized (no Sized or Move)

The only difference between this and what you wrote is that Sized + ?Move is an option.

[mikeyhew]

@lxrec

I think it's confusing only because Referent does not sound like it means LiterallyAnyType. It sounds like it exists solely for the custom DST use case and should never be mentioned in code that's not specifically messing with custom DSTs. If the goal is instead for all Rust types to implement those possibly-fat pointer create/split methods (is there any reason we wouldn't want this?), we should probably change the strawman name for it to Anything or AnyType or Unrestricted or something like that.

I hear you. If you're coming at it with the goal of removing all possible trait bounds, or to specify that T can be any type without any restriction, then having to write T: Referent does sound like a weird detail.

But in what context would you actually want to do that? Like, when would you have a generic type argument, associated type, or trait Self type that can literally be any type, without restriction? Usually you want to do something with values of that type. So if you only need to have it behind a pointer or reference, then you write T: Referent. If for some reason you need to be able to get its size and alignment, you write T: DynSized. And so on.

Perhaps there are cases where you're writing generic code and you literally have no requirements about the type, you don't even need it to be behind a pointer, but I'd like to see concrete examples. Two that I know of are LittleEndian and BigEndian from the byteorder crate. They are currently implemented as empty enums, which are Sized, so you don't need to write T: Referent. But they could, theoretically, be implemented as !Referent types, if we added them to the languages – and if that is desired, I would also like to add Type to your list of potential (pseudo) trait names. The behaviour of writing T: Type would be it just removes any and all default trait bounds. By writing T: Referent now, !Referent types could be backward-compatibly added to the language without the need for ?Referent syntax.

[Ericson2314]

@withoutboats you say in #2255 (comment) that the general concern less weirdness than making things "everyone's problem". But isn't it the other way around? Once can always ignore ?-trait bounds along with the weird use-cases that motivate them, but with positive trait, now everyone needs to care just to get back to the default case. See rust-lang/rust#43467 (comment) where @nikomatsakis is more interested in the positive DynSized proposal, which has just that problem.

[mikeyhew]

I had an idea that might be able to help with the ergonomics of ?-traits, and with commonly-repeated trait bounds in general: a language feature to opt into or opt out of a default bound at the crate or module level. For example, if you are implementing a smart pointer type, and you find that for every type parameter, you have to write ?Sized, then you could opt out of the default Sized bound for type parameters in that module.

Making default bounds a first-class language feature that libraries can use, instead of a special case for only a handful of builtin traits, would hopefully make them less confusing. It would also pave the way for changing which traits are default bounds in a new epoch edition.

[Ixrec]

For anyone who stumbles across this in the future:

The ?Moved trait bound didn't happen, because a new implicit bound was always the "brute force", "sledgehammer" option for those problems, and we've since found a better, more targeted solution: the Pin<T> library type. See https://doc.rust-lang.org/std/pin/ and https://boats.gitlab.io/blog/post/2018-03-20-async-vi/ for details on that.

The situation with ?DynSized is different, but mostly because Sized implies DynSized, which gives that proposal a loophole for some of the biggest problems discussed here. Thus, it's better discussed in the specific RFC thread proposing it: #2594.

AFAIK, there are no other suggestions for new implicit bounds with strong motivation/demand.

[Ericson2314]

Yeah i still don't fully grok all the compatibility concerns. With associated types, for example would say rather than Trait::Assoc: DynSized is not part of the trait itself, but a well-formedness condition on anything referring to the trait, and one would write Trait::Assoc: ?DynSized at the usage site to opt-out, just as one does with fresh type variables.

[RalfJung]

I am trying to find out what the general concerns are around ? bounds. I am coming here from #3729 which suggests that we should have traits where adding a bound relaxes the requirements, which I find quite concerning and non-intuitive.

The issue here is rather long, is there a good summary anywhere? Replying to the concerns in the issue description:

?Sized itself being a "negative feature" confuses users, adding ?Move and ?DynSized will only make the situation worse,

Sure, but spelling this as T: ValueSized and making this implicitly mean "we now know less about T than if we didn't write anything" is IMO even more confusing.

introducing new relaxed bound means downstream packages will need to reevaluate every API, to see if adding : ?Trait makes sense, and this needs to be done for every new relaxed bound.

This is true no matter the syntax we use for this, so avoiding ? bounds does not help.

@davidtwco you mention several times in your RFC that the lang team doesn't like ? bounds; do you know of a good write-up explaining why? Niko said he'd most that here but it seems that never happened.

[davidtwco]

@davidtwco you mention several times in your RFC that the lang team doesn't like ? bounds; do you know of a good write-up explaining why? Niko said he'd most that here but it seems that never happened.

I don't know of a good write-up, I got that impression from reading all of prior art that I linked in #3729 - I got the impression that ?Sized is generally seen as confusing and hard to understand for new users, and as such adding more ?Traits isn't desirable.

I personally don't find the proposed alternative from #3729 more confusing, it's not necessarily the most intuitive but no more or less than ?Sized (I'm more familiar with ?Sized by now, but that's about the only difference).

In the recent language team design meeting on #3729 (while it was still a draft and ValueSized was called DynSized), there was this exchange on the subject:

Josh: General 👍 to the design here, but I think we should take a moment to consider the alternative of using ?DynSized, and the tradeoff between that and the proposed approach.

Among other things, using ?DynSized seems like it would not interact as cleanly with const, creating a more complex model for people. So on balance the RFC's approach seems like the right one. But I didn't see any discussion of that in the RFC.

davidtwco: I avoided introducing any ?Trait syntax because it seemed like the language team were firmly against introducing more of that in all the prior art.

Josh: You're not wrong, and we probably still are. Just want to figure out if we're losing anything by dismissing that approach, or if (as appears to be the case) the design in this RFC is just better in every way and we wouldn't want to do ?DynSized even if lang was completely fine with it.

NM: I am not sure what ?DynSized would even mean but it just seems megaconfusing to me. You are opting out from the default bound but it's not a binary thing, so what level are you opting out to?

NM: Note that there is a separate question of "when do you add entirely new kinds of default bounds"-- some features I want will require that (e.g., must move types) -- but that's a different set of complexities from this proposal, which is more about the "binary to multi-level".

With the proposals from #3729 in mind, I got the impression that ?Trait syntax was seen as even less desirable as with const traits and with hierarchies, it's even more confusing.

[Lokathor]

?Sized definitely is confusing, but no more than the minimum amount of confusion. The concept is normally ignored through a lot of a programmer's early Rust, so of course having to think about it suddenly is going to catch people by surprise. I'd make a comparison to lifetimes: introducing a lifetime into a function or a structure takes people by surprise until they get used to it, because you can write a lot of Rust with totally elided lifetimes. That's just where the ergonomic optimum is.

I'm with Ralf: spelling negative bounds as positive bounds just to avoid typing a ? doesn't actually help anyone, it's just worse.

[Ixrec]

The best write-up I can remember off the top of my head is that withoutboats' recent blog post on Pins has a "?Move" section about why a ?Move bound was ultimately rejected as a solution to the self-referential futures problem, in particular why it would've been a backwards-incompatible change for reasons involving associated types I still don't fully understand.

[RalfJung]

That's an orthogonal problem, I think. That problem is not solved by picking a different syntax, it is a semantic issue. The question of how to write down a type that is "not statically const Sized but still weaker-Sized" is a purely syntactic one.

[traviscross]

Niko wrote up his thoughts on this here.

The lang team hasn't made a decision on this matter, and personally, I'd like to see more discussion and analysis.