Place left arrow syntax (place <- expr)

[pnkfelix]

Summary

Rather than trying to find a clever syntax for placement-new that leverages
the in keyword, instead use the syntax PLACE_EXPR <- VALUE_EXPR.

This takes advantage of the fact that <- was reserved as a token via
historical accident (that for once worked out in our favor).

rendered draft.

[nagisa]

“Lets add @ and ~ back too!” was my first reaction. I don’t like the in syntax and I don’t like <- either, albeit a little less so. box x in PLACE sounded OK to me, but it didn’t stick, sadly.

Expanding: let x = HEAP <- 10 doesn’t look good at all. Perhaps even worse than let x = in HEAP 10 ± the braces.

[aturon]

I didn't follow the original syntax debate closely, but I agree with @nagisa that let x = HEAP <- 10; doesn't read terribly well. In particular, the RFC correctly points out that <- is strongly associated with assignment, but there's a follow-up point there: the use of <- in an assignment context treats the thing to the left of <- as being the thing assigned to.

[pnkfelix]

(BTW I don't know if this is clear from this RFC, but the intention is that we will keep the box EXPR form, and fix our inference and/or coercions so that box EXPR will infer from the context whether it needs to place the result into e.g. a Box or an Rc, etc... see eddyb's hard efforts here e.g. in rust-lang/rust/pull/27292)

In other words, my intention is that people would not write let x = HEAP <- 10;. They would write let x = box 10; (and type inference would figure out that a Box<_> is needed).

The place where PLACE <- VALUE is needed is when the PLACE is a more interesting expression than HEAP. E.g. let handle = arena <- expr;

[pnkfelix]

(but also I don't actually see what the problem is with let x = HEAP <- 10;. You're putting a 10 into the boxed::HEAP, and returning that boxed value... seems straight-forward to me...)

[pnkfelix]

Another alternative that perhaps I should have listed in the alternatives section

let handle = in arena <- expr;

(man and I just thought this was going to be a slam dunk so there would be no need. ;)

[seanmonstar]

<- and in both read weird to me. I'd prefer if we could just not use new syntax, such that Box::new(10) and Rc::new(5) are what we're write, and those functions would be optimized some way to allow allocating in different places (with an attribute?).

[eddyb]

@seanmonstar But as @pnkfelix is trying to clarify, you would use box 10 and box 5 for those cases, not in or <-.
Maybe type ascription should be merged at the same time, to allow shorthands like (box Foo(a, b, c)): Rc<Trait> (are the parens needed? I can't remember the precedence rules).

[petrochenkov]

Yay!

@nagisa @aturon
It's not Box, but other containers, who will benefit most from this change.

For example, emplacement into Vec:

let v = Vec::new();
// The old syntax
in v.back() { 10 };
// The new syntax. 
v.back() <- 10; // This clearly looks better to me.

or emplacement into HashSet:

let s = HashSet::new();
// The old syntax
in s.new_place() { 10 }; 
in s {10}; // If the set itself can be a "place"
// The new syntax
s.new_place() <- 10;
s <- 10; // Yep, much better

Ideally, a couple of basic containers (Vec, HashMap, HashSet) should get actually working emplacement interfaces before finalizing the emplacement syntax. Does the current state of implementation (i.e. rust-lang/rust#27215) allow implementing such interfaces? (I haven't digged into it yet.)

Edit: HashSet is not the best example because it still needs to construct the RHS before insertion for hash calculation and comparison.

[nagisa]

I’ve been convinced. I can’t think up anything better; <- sounds like the best way to go here. And general uglyness of the Box/Rc<_> case actually makes you want to use box more – for the better.

👍

[aturon]

Thanks @petrochenkov; usage in those examples hews much closer to my intuitions about <- as assignment syntax. I withdraw my objection.

[seanmonstar]

@eddyb yea, and I don't necessarily find that better.

let x: Rc<i32> = box 5;
// or
let x = Rc::new(5);

[Gankra]

@seanmonstar In my experience the box would almost always be inferred -- but that also has issues, perhaps (akin to just invoking collect or default).

[liigo]

box EXPR into PLACE is always the best!

(I don't think PLACE evaluating before EXPR is a big violator to any explicit rules. Or I don't mind that.)

[nagisa]

@liigo it was mentioned in previous discussion EXPR might often get quite big, e.g.

let x = box {
    a;
    b;
    c
} into HEAP;

which is quite a lot of scanning required to even notice this is not a regular box, but box-in.

That is, it is preferable to have EXPR after PLACE, IMO.

[oli-obk]

while let x = arena <- value; looks odd (not more than let x = in arena { value };), for existing containers as shown by @petrochenkov this looks very intuitive (Note: I wasn't around back when Rust had the <- operator).

[eddyb]

I had another look at @petrochenkov's examples, and I believe the following could be made to work:

let map = HashMap::new();
map.entry(k) <- v

Maybe map[k] = v will be sugar for it, maybe not.

[nagisa]

@eddyb if we’re speaking concrete examples, Vec<_> is even more concise; the only reasonable place to emplace elements is its back, therefore Vec could implement the necessary traits itself:

let v = Vec::new();
v <- element;

[pnkfelix]

@petrochenkov asked:

Ideally, a couple of basic containers (Vec, HashMap, HashSet) should get actually working emplacement interfaces before finalizing the emplacement syntax. Does the current state of implementation (i.e. rust-lang/rust#27215) allow implementing such interfaces? (I haven't digged into it yet.)

Yes, now that rust-lang/rust#27215 has landed, the standard library should be able to experiment with adding implementations of the placement protocol to allow use of the in PLACE { EXPR } syntax.

[apasel422]

@petrochenkov For what it's worth, I've gotten placement working with my tree map library here: https://github.com/apasel422/bst/tree/place.

[petrochenkov]

Maybe in {}/<- should do autoref? Otherwise these nicely looking statements would consume the arena (twice!)

[eddyb]

You generally don't keep an owned arena around, i.e. arena would most likely have the type &Arena.

[petrochenkov]

You generally don't keep an owned arena around

The same can't be said about vectors. I.e. the v <- elem syntax proposed by @nagisa would have to roll back to v.back() <- elem (or &mut v <- elem). It's not especially terrible, though.

[nagisa]

Alternative I’ve been having in mind here is to change Placer::make_place to take &mut self instead of self.

[ftxqxd]

@nagisa That wouldn’t really work for all cases, because, e.g., [Typed]Arena::alloc takes &self, not &mut self, meaning that you’d be forced to use mutable references when you shouldn’t have to.

[pnkfelix]

It might end up being moot for Arenas. Namely, We might be able to do

impl<'a> Placer for &'a Arena { ... }

Which in that particular case side steps the choice of self vs &mut self.

I am personally becoming more convinced that it should be &mut self. I spent a while last night looking through my notes to see why I had picked self but did not find a motivating example.

[apasel422]

Would &mut self work with map.entry(5) <- "foo"?

[killercup]

I'm not sure how I feel about this syntax. At first it looks like another part of Haskell finally made it's way into rust, but when I see @eddyb's

let map = HashMap::new();
map.entry(k) <- v

I can't help but be reminded of Go's channel syntax (docs).

[glaebhoerl]

Just to be clear, is it accurate to say that the only reason we want these special placement APIs and operators, instead of moving in the direction of the otherwise simpler and more general &out/&uninit type (e.g. fn back<'a, T>(self: &'a mut Vec<T>) -> &'a out T { ... }; *vec.back() = foo();), is to be able to recover from panics in the to-be-placed expression (foo()) without aborting?

[pnkfelix]

@glaebhoerl the phrase "recover from panics" is a bit ambiguous.

We do want to be able to continue to properly unwind the stack, but I don't call that "recovery."

(My personal take on it is that the Placer protocol is a simpler change than trying to add &uninit T, which seems like it would require more linguistic infrastructure, somewhat analogous to type-state. Perhaps I am wrong here and there is no more infrastructure than is required to represent e.g. the temporary freezing of &mut when they are reborrowed... but that doen't match my gut feeling about it)

[aturon]

@glaebhoerl I agree with essentially your entire comment, except that I am slightly less sure about &out. In particular, it feels to me that placement should effectively be a version of &out that, instead of being tied to lifetimes, uses ownership and dynamic finalization, and thereby nicely accounts for unwinding etc. I haven't played enough with the placement protocol to say how close we are to that goal.

I wonder, for example, whether we can address the current overhead of buffer zeroing in Read::read implementations by introducing a method that takes a place instead.

[glaebhoerl]

In particular, it feels to me that placement should effectively be a version of &out that, instead of being tied to lifetimes, uses ownership and dynamic finalization, and thereby nicely accounts for unwinding etc.

Hmm... could you perhaps elaborate further on this idea? (Are there analogous owned/dynamic counterparts to the other lifetime-constrained reference types? Can you have first-class places while accomodating both unwinding and mem::forget?)

One of the reasons for my uncertainty is indeed that while &out feels more simple, general, orthogonal, etc., it is still the case that Rust has unwinding (even if I may wish it didn't) and that &out doesn't mesh well with it, and so, at a minimum, we may want something else alongside &out to better reconcile the two. But I can't tell what that something would be, or if it would look like the placement protocol (in part because I haven't yet managed to push myself through to really comprehend it), only that it makes me somewhat uneasy that we're trying to figure the latter out before the former.

I wonder, for example, whether we can address the current overhead of buffer zeroing in Read::read implementations by introducing a method that takes a place instead.

I had assumed this would be a natural use case for &out, but didn't dive into the details. Is there a reason why one or the other would be better suited here?

[arielb1]

@glaebhoerl

The placement protocol is indeed a version of &out that does not kill the process on unwinding. It also "supports" mem::forget in the same way Drain does.

[nagisa]

I noticed this RFC does not specify precedence nor associativity of the operator.

As far as associativity goes, it should be right-associative (i..e a <- b <- c parses as (a <- (b <- c))).

Precedence is less clear. It should bind stronger than assignment (=) for sure, but I can’t say whether it should bind more strongly than comparison/binary/math (i.e. let x = a & b <- c * d might make sense as both let x = ((a & b) <- (c * d)) and as let x = (a & ((b <- c) * d))).

[petrochenkov]

@nagisa
Since <- is very similar to =, I'd expect it to have the same associativity and the same precedence + 1.

[pnkfelix]

Okay this thread has gone off the rails. we need a FAQ wrt the placement-in system because people are chiming in without knowledge of what came before.

Just to advertise this at the (hopefully appropriate) audience; here is the current draft, which I plan to update as feedback comes in:

• https://internals.rust-lang.org/t/placement-nwbi-faq-new-box-in-left-arrow/2789
• (please deliver such feedback via the comment thread on that internals post)

[vadimcn]

Hmm, am I totally off-base with this comment: https://internals.rust-lang.org/t/placement-nwbi-faq-new-box-in-left-arrow/2789/19?
Because if not, there are no downsides to using closures for delaying creation of the placed value, which would simplify the placer protocol significantly. Nor would we need the <- operator...

[eddyb]

@vadimcn Hmm, this should be the first item of a FAQ: @pnkfelix tried closures. Literally the first thing he tried.
Doesn't work because of control flow, the try! example is just an example of misbehaving code.
You don't want to use placement syntax only when it helps, you want to use it everywhere, for consistency, and closures would make that impossible in certain cases.

[vadimcn]

@eddyb: yes, but what I am saying is that in order to use try!, the inner code must return a Result<>, which, as far as I can see, will force creation of a temporary on the stack. And if you can't use try!, is the whole thing worth it?

[fenduru]

@vadimcn try! diverges, and returns in the error case. So if you use it inside a closure it will return from that closure instead of the outer function

[eddyb]

@vadimcn Maybe a clearer example would be:

let mut v = vec![];
for x in a {
    v.push(box {
        if x == 0 { break; }
        x + 1
    });
}

Were box to be implemented using closures, break would simply not work.

The try! example is more subtle, as return works inside a closure, but you end up with a type conflict between the early return of Result<_, E> of try! and the value try! produces.
In the corner case of nested Result (e.g. || try!(Ok(Ok(0)))), compilation may succeed, but without an early return from the enclosing function in case of error.

[vadimcn]

Folks, I totally get what you are saying about the control flow.

But look at it from the other side: the raison d'etre of the placement operator is to eliminate temporaries in the process of moving the value into its place. And it does not work for this purpose in cases when the value creator returns anything but the value itself (such as Result<LargeValue,Error>).
AFAIK, enabling fallible value creation was the main justification for not simply using closures. Once this is out, the feature no longer carries it's weight, in my opinion.

As for the second @eddyb`s motivating example, I find it unconvincing: you can easily re-write it as:

let mut v = vec![];
for x in a {
    if x == 0 { break; }
    v.emplace(|| x+1);
}

There may still be useful control flows that work with <- and don't work with closures, but they need to be identified and documented before we go forward with this RFC.

[nikomatsakis]

@vadimcn I would certainly love to just rely on closures. It's also plausible that this is indeed the best path. And maybe if we played enough tricks we might even be able to make foo.push() "just work" if you supply a "closure returning T" (though there is the obvious ambiguity problems), which has always been a kind of holy grail for me. :)

All that said, I think there are two points being lost here:

1. Using closures as the desugaring for the <- operator is a non-starter, I think, because of control-flow problems. If just have no operator, and hence no desugaring, this goes away.
2. Just because fallible things occur as part of the creation expression doesn't mean that those fallible things are the main result. For example:

vec <- Entry {
    field1: foo?,
    field2: bar?,
    field3: [0; 1024]
};

Something like that might still make sense, it seems to me.

You could also imagine things like:

vec <- if some_condition {
    return;
} else if some_other_condition {
    break;
} else {
    generate_big_result(some_fallible_thing()?)
};

(which I guess is just sort of a variation on a theme).

[vadimcn]

Using closures as the desugaring for the <- operator is a non-starter, I think, because of control-flow problems. If just have no operator, and hence no desugaring, this goes away.

I agree, this is what we should do.

vec <- Entry {
    field1: foo?,
    field2: bar?,
    field3: [0; 1024]
};

Since computation of foo1? and foo2? involves creation of temporaries anyways, it can be moved outside. Slightly more verbose to write, but doesn't really add any runtime overhead.

let field1 = foo?;
let field2 = bar?;
vec.emplace(|| Entry { 
    field1: field1,
    field2: field2,
    field3: [0; 1024]
});

Same applies to the last example.

[withoutboats]

I think the major cost of syntactic support for emplacement, really the major cost for emplacement in general, is pedagogical. Everyone has to learn what emplacement is and when they need to use it. As a rule, additional syntax tends to increase the learning burden, but I think this is an exception. it seems easier to understand the notion of emplacement by dedicating syntax to it than it would be to understand why in some cases you should use a higher order function instead of the more at-hand function.

Put another way, I don't think it is easier to explain to new users why they should not use vec.push(Entry) and instead use vec.emplace(|| Entry) than it is to explain why they should use vec <- Entry. I think the syntax makes it easier to comprehend.

I think it also makes it easier to comprehend when scanning code, because higher order functions can have many different purposes.

[Centril]

If Rust decides do add do-notation for Monads (HKTs required), will this RFC have made the do-notation syntax that haskell uses impossible for Rust? Or is it still perhaps possible to give a different context based meaning to <- when started with a do "block"?

[nagisa]

@Centril Rust is not a Haskell and it doesn’t face the same constraints as Haskell does/did. We might be considering HKTs for Rust, but Monads and sugar around them has little purpose even with HKTs in play.

[ticki]

@nagisa, no that's simply not true. While they have less importance in Rust (due to e.g. sideffects), they're still very useful for e.g. error handling.

[Centril]

@nagisa Having a Monad HKT trait allows you to define common operations for a whole host of data structures once. There are already a bunch of Monads in Rust, they are just not grouped together under a trait... Vec, Option, Result, BinaryHeap.

Monads are not just a way to get "side-effects", but to redefine the meaning of ; (semicolon)

Other interesting monads: Parsers, Futures, Language-based-security (http://www.cse.chalmers.se/edu/course/TDA342_Advanced_Functional_Programming/lecture9.html).

Add to this: Monad transformers, where you have a stack of monads... http://www.cse.chalmers.se/edu/course/TDA342_Advanced_Functional_Programming/lecture6.html

Also: monad comprehensions are even more powerful than list-comprehensions.

[eddyb]

@Centril In a systems language, fancier functional tricks meet the real world and tend to fall apart.

As an example, Monad::bind needs to be generic over T, U and F: Fn(T) -> Self<U>.
Concerns over the choice of Fn, FnMut or FnOnce aside, this means that if you don't call F before returning, Self<*> cannot hold onto it without boxing it and turning calls to it into virtual calls.

If devirtualization doesn't occur, the slowdown compared to our current iterators or future hypothetical state-machine-based generators can be more than an order of magnitude.

Parsers and futures also tend to use implicit existentials and/or GADTs, which admittedly Rust needs, but still have the overhead of boxing and virtual dispatch.

We get it, monads are cool. But unless you can find a formalism for "do notation" which allows pervasive static dispatch and no boxing by default instead of as an optimization (and maybe actually integrate it with imperative control-flow primitives), I'm afraid there will always be more specialized solutions, not unlike the theoretical purity of linked lists contrasted with the real-world cache-efficiency of arrays.

[Centril]

@eddyb Good comment. Didn't realize that F becomes a virtual call. But just to be clear, you're not saying that Monad::bind itself becomes virtual?

Rust is a systems language, but not just. It can be used for other things =)
Anyways, if <- is reserved for placement new you can always reserve <= for do notation.
I guess we can continue discussion on HKT-related issues/RFCs.

[eddyb]

@Centril Indeed, you can have Option or Result's and_then methods without any losses if you keep taking FnOnce closures (so by-value captures can be used).
Any usecase where you call the closure immediately remains static dispatch, problems appear when you try to do something like Iterator::map without having all of those explicit generics.