Extending core::ops::Index to return non-references

[shepmaster]

Motivation

There are times where "indexing" is desired but returning a literal reference is suboptimal or impossible.

@cuviper said (lightly edited by me):

For example, ndarray would want to return ArrayView<'_, ..>. For mutable indexing, it would want ArrayViewMut<'_, ..>. GAT indexing would need a distinct type Output<'_> [for each mutability]. Currently, without GAT, they use slice and slice_mut methods.

We recognize that this requires GATs and something else to be able to truly supplant core::ops::Index and the [] syntax.

Details

A concrete example of the problem:

#![feature(generic_associated_types)]

fn main() {
    let c = Container(42);
    // Nightly
    let w = c.index(());
    // Desired, strawman syntax
    let w = &c[()];
    w.hello();
}

struct Wrapper<'a>(&'a u8);

impl Wrapper<'_> {
    fn hello(&self) {
        eprintln!("Hello, {}", self.0);
    }
}

struct Container(u8);

impl Index<()> for Container {
    type Output<'a> = Wrapper<'a>;

    fn index(&self, index: ()) -> Self::Output<'_> {
        Wrapper(&self.0)
    }
}

pub trait Index<Idx>
where
    Idx: ?Sized,
{
    type Output<'a>: ?Sized
    where
        Self: 'a;

    fn index(&self, index: Idx) -> Self::Output<'_>;
}

References in the wild

• How to implement std::ops::Index when the output value is not part of the type
• Implementing Index trait to return a value that is not a reference
• How to wrap a borrowed value in a newtype that is also a borrowed value?

Related traits

These traits might also conceptually benefit from similar ability:

• Deref
• Borrow

[danielhenrymantilla]

"Moved" from #3:

---

Based on the "GAT motivation" discussion over Zulip, and more precisely, around this question (archive link for those not on Zulip); people have started pondering about the idea that traits representing some of our sugary operations, such as […]-indexing, or deref-coercions, could maybe be GAT-ified in some retro-compatible fashion, so that they not be required to necessarily yield & or &mut references respectively.

Motivation

Plenty of examples in that Zulip thread; but people can also provide some to this very GH "issue".

But to summarize the main ones:

• custom (re-)borrowing (e.g., Pin<&mut _>) could be featured with a GAT-ified Deref{,Mut}), and more general borrowing (e.g., &mut pinned_box being able to coerce to pinned_box.as_mut());

• custom indexing/slicing.

  • without involving unsafe or other RefCast-like approaches;
  • when the reference type does not necessarily have the layout of a Rust reference (e.g., more embedded data (could be solved by ptr_metadata), or FFI requirements).
  • when other GAT APIs become widespread, and thus types such as <Ty as Trait>::Gat<'lt> become more common.

Questions:

• What would exactly be the semantics of a GAT-ified operator such as Index{,Mut} (I suspect the same concerns apply to Deref{,Mut})?

  indexable[idx]

  is actually sugar for a *-dereference (to a place expression) of the &Index::Output or &mut Index::Output (depending on the kind of usage of the place), so the starting point of these traits is &indexable[idx] or &mut indexable[idx].

  Wouldn't GAT-ifying this result in either:

  • reversing the relationship;
  • and/or having &indexable[idx] not yield a &T (and same for &mut).
    This last idea seems like the most likely solution? Maybe?

• How could the change be made in a retro-compatible fashion? I suspect we'll need new traits, to be the really sugared ones, and have the current ones be subsets which blanketly implement the new ones.

[SkiFire13]

and/or having &indexable[idx] not yield a &T (and same for &mut).

To be backward compatible indexable[idx] should still yield T though.

Also, currently indexable[idx].foo() desugars to either indexable.index(idx).foo() or indexable.index_mut(idx).foo() depending whether foo takes &self or &mut self. How would this work if the GATified Index and IndexMut allowed different associated types?

[danielhenrymantilla]

Yeah, that's a problem: even though one could add a "let's try both paths" heuristic there, it means that multiply-chained indexing operations ([][][][].method()) would end up exploring an exponentially big set of candidates 😬

[cuviper]

Ugh, that ambiguity is unfortunate. I fear that will affect all possible uses of GAT indexing, not just my example, because the type mutability is hidden. That is, if we had fn index(&self, ..) -> Self::Output<'_>, there's no mechanism to write the "same" return type in a mutable form for index_mut. (compared to the current &Output and &mut Output)

[TennyZhuang]

Another similar example may be BitVec.

[nikomatsakis]

Thanks for opening this. @jackh726 and I were discussing similar questions around lending iterator and thinking that, before stabilizing, it would be good for us to spell out our plans here (whatever they are).