Rust 2020 roadmap

[nikomatsakis]

Summary

Lays out the Rust roadmap for 2020 in pursuit of our mission
to empower everyone to build reliable and efficient software.
The roadmap takes the form of the following goals for the project:

• Prepare for a possible Rust 2021 Edition
• Followthrough on in-progress designs and efforts
• Improve project functioning and governance:
  • Improve visibility into the state of initiatives and design efforts
  • Increase mentoring, leadership, and organizational bandwidth
  • Make design discussions more productive and less exhausting

The roadmap this year is different in structure than prior years. In
particular, we have avoided setting precise goals, in favor of
describing more general mandates and themes.

We chose to take this approach for a few reasons:

• The roadmap RFC doesn't seem like an appropriate place to make
  decisions on specific solutions. Those should be discussed in their
  own, dedicated RFCs.
• We wanted to encourage teams and project members to think about how these
  mandates apply best to the particular questions that they are working with.

Rendered view

cc @rust-lang/core

--

A note on authorship: This RFC was jointly edited. Particular thanks to @Mark-Simulacrum, @nrc, and @aidanhs for their feedback and edits.

[Lokathor]

edit: nevermind me, covered in the above review

[Centril]

Based on my experience with managing the edition rollout last time, I think we should deal with editions differently this time. Instead of being a rallying point for various tooling, domain, and language improvements, which in my view produced both stress in developers and confusion with users, I would like it to be strictly about the opt-in breaking edition changes. I think that also lessens the need to commit to a deadline. Instead, we can accumulate a list of edition changes under a nightly-only edition-next and then once we feel that the set of breaking things is sufficiently saturated, we can make the final preparations for the edition and give the edition a name/year.

[Lokathor]

"rally point for a marketing push" mixed into other things was definitely a bad idea.

However, "3 years" is easy to understand for users, and it's already what most people think, so if there are any changes that can be arranged by the end of 2020 and pushed out the door by mid to late 2021 we should really try to stick to something in 2021, and anything after that just plain has to wait until 2024. Same as we push rustc every 6 weeks no matter how much or how little was done that 6 weeks.

[Manishearth]

We discussed this a bunch internally, it seemed like consensus was either to do it as:

• once every three years, but not push super super hard for breaking lang changes (if it slips, it slips, it'll happen next time)
• whenever we feel it is necessary, and push the button when everything is ready

(You seem to be suggesting the latter)

an observation was that with the latter it's harder to do things very predictably, as @Lokathor said a regular 3 year cadence is easy to understand. It's also nice to have a periodic longer-term thing where the rest of the world realizes the progress we've made. I think @nikomatsakis and @nrc had some really well-written thoughts on this

[Centril]

It's also nice to have a periodic longer-term thing where the rest of the world realizes the progress we've made. I think @nikomatsakis and @nrc had some really well-written thoughts on this

I don't mind such a longer-term thing, but I think it shouldn't be organized together with editions as it pushes us to stabilize features on a specific deadline, before things are free of known issues.

[Lokathor]

It might push us to focus more on a given issue, but if it slips it slips.

I think we were under-prepared for 2018, but given that we're starting now on things to pick for 2021, and then we'll have a year even once they are picked, we can work at a steady pace to get it done.

[Centril]

@Centril you've mentioned deadlines a few times - can you elaborate on where you find the perception of these come from? Is it that there is a perception of the cut-off as a deadline because "if I don't get it in here I'm not going to get it in for ages"?

I think that's partly the case. Mostly, however, I think it is the effect that deadlines have themselves. Once we decided that e.g. NLL should make it into Edition 2018 well in advance of the bugs and other issues actually having been fixed, I think that made us more inclined to stick to that deadline (due to social pressures, ...) whether or not those bugs were fixed (this in turn created problems like rust-lang/rust#59159). Similar issues happened with async/await where we also set deadlines well in advance of the issues being resolved. Another issue that we rushed was the module system, which many last minute decisions being made (although I'm overall quite happy with the end result).

My concern with pushing out the cut-off is it unintuitively increases the pressure to get it in next time, because there's now no regular schedule and "who knows when the next chance will come around".

Fair points! I do think this is a possibility. I've sorta mixed together two distinct issues here, one regarding cut-off, and one regarding setting deadlines for stabilizations. My primary concern here is the latter aspect...

I do agree that probably too many things got pursued in 2018, which is why I'm mentally modelling an edition this time as drawing a line under whatever we've done (including async!) and bundling 'obviously required' breaking changes with it. A 'rallying point' doesn't necessarily mean we have to get a bunch of things done in pursuit of something - it can also mean just taking stock of what we've achieved.

...which is why I find this to be a particularly helpful distinction and a key point! I love the idea of focusing editions more on documenting to users what we have already stabilized (e.g. slice patterns, rust-lang/rust#67712) rather than a push to stabilize more on a deadline (e.g. specialization, at least based on today's prospects, heh). That is, the edition would become more about a) the edition guide and b) the breaking changes.

Let me just phrase that another way. If we make editions run on a schedule, the emphasis for big features should be to get enough of the pre-work done so that their deprecations are part of the edition, but not necessarily any new additions. One nice thing about this particular edition cycle is that I don't think we have a lot of big features, so we're going to be able to just focus on smalle changes.

Yes, I agree entirely. I think this strikes a healthy work balance and solidifies editions as migration and documentation polish.

[Lokathor]

Nitpick: aren't slice patterns a general language change available in all editions with a new enough compiler?

It also happened in the 2018 edition that things that happened to come out around that time were marketed as being "new in 2018 edition!" even though they were just new in the latest compiler versions and also available in 2015. I think that we should very much keep edition marketing to only the things that you get by changing 2018 to 2021 in your cargo file. Not anything else at all.

[Centril]

Nitpick: aren't slice patterns a general language change available in all editions with a new enough compiler?

Yes, they are, but they could go under "taking stock" / enhancing the documentation of "what has been improved in the language since the last 3 years".

I think that we should very much keep edition marketing to only the things that you get by changing 2018 to 2021 in your cargo file. Not anything else at all.

That's... going to be a very short list that fits in 1/4 A4 page most likely. =D That writing is something you finish up in maybe 2 hours of work for a really polished version (beyond the work in the compiler itself).

[Lokathor]

Short and sweet is good. Great in fact. It means we didn't need many changes, which means things mostly went right the first time.

Immediately after 2018 came out it was hard to find material that maintained the distinction between 1.30 vs 1.31 and 2015 vs 2018. Many people got confused about if they had to change edition to get ability X or Y. I just don't want any of that repeated.

Just to pick an example, slice patterns is given in the edition guide as being part of 2018 edition with a minimum rust version of 1.26, which is clearly nonsense because 2018 edition landed in 1.31.

[Centril]

Immediately after 2018 came out it was hard to find material that maintained the distinction between 1.30 vs 1.31 and 2015 vs 2018. Many people got confused about if they had to change edition to get ability X or Y. I just don't want any of that repeated.

Yeah I very clearly remember discussing this confusion with you and others at the time on Discord. ;) I believe we can do better this time around with the experience of what not to do from last time. However it's also possible that we would add to the confusion.

[Centril]

This could use some discussion in the lang team re. scheduling of the decision to stabilize a new edition in the compiler.

[Centril]

I'm pretty happy with the new strategies of a) posting implementation history updates in tracking issues, e.g. as in rust-lang/rust#54883, b) using feature gate labels (e.g. F-const_generics).

[Mark-Simulacrum]

(Resolving -- it seems like this is not a concern? :)

I will say that I agree (and I think we all do) that we have been doing better in this area, but I think it is also true that we could do a lot better too.

[Centril]

I thought it was worth mentioning in the text as useful information on the progress made thus far. I'm actually not sure we could do so much better other than perhaps some automation.

[aidanhs]

One big audience we aren't serving with those strategies is the person who has never visited the Rust issue tracker. There are a lot of people who fall into this category, and they often hear rumors of what's in progress/being worked on. I think it'd be nice to serve these people in a way better than pointing them at the issue tracker.

[mark-i-m]

@Centril I strongly and respectfully disagree.

I'm sure the teams internally do a good job of keeping track of stuff, but unless you are on one of the teams, the process is very opaque. You have to actively lurk on zulip, discord, internals, and github, and read TWiR religiously to find out what is being worked on, what recent developments are, what team priorities are, etc. As a maintainer of rustc-guide and member of WG-learning, I consider myself relatively knowledgeable, and I still feel somewhat left in the dark.

In the ideal case, if one wanted highest visibility to newbies, there would be something like a giant kanban board somewhere that lists what everyone is doing/prioritizing across all teams, what's blocked, what's being worked on, what is de-prioritized, what is urgent, and it would be updated say weekly. Of course, I realize that's a lot of work, but perhaps that's the ideal to work towards.

To be clear, I'm not saying the teams should do this for themselves; whatever process you're already using seems to work fine. I'm saying there should be some medium intended for consumption by non-team-members that accomplishes this purpose. I think the "Inside Rust" blog is a good step in this direction.

[Centril]

You have to actively lurk on zulip, discord, internals, and github, and read TWiR religiously to find out what is being worked on, what recent developments are, what team priorities are, etc.

You can also subscribe to the tracking issues you are interested in and get updates there. (Unless the tracking issue is poorly maintained, which is the case for the old ones, and not everyone is following the new process of feature gate labels and whatnot, so I need to do some PSAs about that). However, the primary function of these labels and tracking issues is to keep track of the bugs, open issues, and history for review by the teams so that everything is covered. I think it does function well for finding

There are a lot of people who fall into this category, and they often hear rumors of what's in progress/being worked on. I think it'd be nice to serve these people in a way better than pointing them at the issue tracker.

In the ideal case, if one wanted highest visibility to newbies, there would be something like a giant kanban board somewhere that lists what everyone is doing/prioritizing across all teams, what's blocked, what's being worked on, what is de-prioritized, what is urgent, and it would be updated say weekly. Of course, I realize that's a lot of work, but perhaps that's the ideal to work towards.

Could you please articulate why it is important that we serve newbies with the highest visibility for unfinished features that are being worked on beyond what e.g. rust-lang/rust#54883 would offer? I would personally find it stressful to report exactly what I'm doing, what I'm prioritizing, etc. on a weekly basis, and it would make me feel more like an employee in a company, developing a commodity, rather than working on FOSS. I think saying that we should work towards this ideal needs to be, but hasn't been justified.

To be clear, I'm not saying the teams should do this for themselves; whatever process you're already using seems to work fine. I'm saying there should be some medium intended for consumption by non-team-members that accomplishes this purpose. I think the "Inside Rust" blog is a good step in this direction.

As long as we can continue tracking issues like in rust-lang/rust#54883 (as opposed to moving out our tracking to some other place where I cannot use feature gate labels, links and backlinks to issues and PRs) and that this extra "newbie friendly" work doesn't have to be done by language team members then I have no objections.

[mark-i-m]

I would argue that the following are highly desirable (and maybe to some extent, crucial):

• It should be easy for contributors (especially new contributors) to see what is being worked on, what priorities are there, etc
• It should be easy for users to see where the language is headed, what features are likely or unlikely to land, etc

By "easy" what I mean is that a person who makes a reasonable effort should be able to gain visibility. IMHO this is not the case, as I mentioned before.

Note that these goals are broader than a single issue; they are project-level, so tracking issues are insufficient to solve this problem. I think this is why there are often questions like "any progress on this?" on tracking issues.

To be clear, I'm not arguing that we should actually make the giant kanban board. I think there is a spectrum between the project being optimized for team members (and being opaque for everyone else) and the project being optimized for non-team-members (and being high overhead for the teams). I am arguing that perhaps we need adjust where on the spectrum we fall to be a bit more friendly to "outsiders".

Perhaps one idea would be to have a "rust status" website somewhere, where all of the teams/WGs can asynchronously update what they've been up to recently, kind of like the WG checkins that happen at the T-compiler meetings on Thursday. I don't feel that would impose undue overhead, though perhaps others are more qualified to comment.

[Centril]

I would argue that the following are highly desirable (and maybe to some extent, crucial):

Can you articulate why it is crucial?...

• It should be easy for contributors (especially new contributors) to see what is being worked on, what priorities are there, etc

...my sense is that the reasoning wrt. crucial here is that we need old contributors to stay and new ones to come in for us to get things done (e.g. all the technical debt that we need to work off). That does seem important. The needs of contributors, as compared to users, do however seem different. In particular, I would expect that "what is being worked on" and "what priorities there are" would be clarified, above the feature-gate level, primarily what by project / working groups there are.

• It should be easy for users to see where the language is headed, what features are likely or unlikely to land, etc

These two aspects are quite distinct from each other. Specifically, "where the language is headed" is something which requires a very broad overview of a large amount of features (well, except for features which are very large, in which case see discussion below re. tracking issues for feature gates).

Meanwhile, the likelihood of a feature landing or not primarily depends on the number of open bugs, open design questions, and the strength of the test suite. This is not always clear from a tracking issue, especially old ones without F-my_gate labels and without a noted implementation history, but that is a matter of technical debt that I'm trying to pay off. Let's take a concrete example of a new-style tracking issue.. Do you find that the information in rust-lang/rust#54883 (which I've linked quite a few times, heh...) makes the current status and stabilization-right-now unclear? If so, how?

Note that these goals are broader than a single issue; they are project-level, so tracking issues are insufficient to solve this problem. I think this is why there are often questions like "any progress on this?" on tracking issues.

Typically, I find that the language team works most of the time on a granularity level of individual feature gates, PRs, and issues. Sometimes we also have larger efforts, and then those should be tracked using meta-issues (e.g. rust-lang/rust#63066 and rust-lang/rust#57563). So I think a lot of the high level overview information just does not exist because we do not work that way.

I do indeed see people leaving "any progress on this?" or "what's he status?" on a tracking issue from time to time.

• Sometimes that's because the tracking issue is genuinely messy (Tracking issue for specialization (RFC 1210) rust#31844). This, the specialization tracking issue is an example of a poorly moderated old-style issue, which largely does not actually track anything. With new tracking issues I setup things differently: I note explicitly (and enforce) that tracking issues are for tracking implementation progress and status updates. Design questions, bugs, and other larger pieces of discussions are moved to issues or other places. This ensures that the relevant updates are not lost in a see of irrelevant information to sift through and ignore for a user (and for contributors & team members).
• Sometimes it is because the person cannot be bothered to read the OP which details the status that is clearly provided.
• Other times, the person wants really means that there should be more progress on an issue whereas there isn't ("any updates?" really means "please make progress", but most of the time that's not an actionable comment and thus unhelpful).

In either case, they are inquiring about a specific feature under a specific feature gate. So I don't understand how e.g. a full accounting of the implementation history, with updates whenever a PR lands (as in e.g., rust-lang/rust#54883) does not satiate what the person wants to know.

To be clear, I'm not arguing that we should actually make the giant kanban board. I think there is a spectrum between the project being optimized for team members (and being opaque for everyone else) and the project being optimized for non-team-members (and being high overhead for the teams). I am arguing that perhaps we need adjust where on the spectrum we fall to be a bit more friendly to "outsiders".

The language team has already substantially increased its transparency, with language team minutes being published in the repo, and all meetings being public and open for everyone to attend, as well as posted to YouTube. As aforementioned, I've also tried to improve the organization of tracking issues to link PRs, bug reports, and tracking issues together. Just this baseline work itself is time consuming as right now, I have to watch PRs and issues to make sure the labels are added (though I haven't communicated the new system as well as I could have).

Perhaps one idea would be to have a "rust status" website somewhere, where all of the teams/WGs can asynchronously update what they've been up to recently, kind of like the WG checkins that happen at the T-compiler meetings on Thursday. I don't feel that would impose undue overhead, though perhaps others are more qualified to comment.

If there are more people who can put in the hours do this managerial work, then sure, that can work. However, I am concerned that this will in reality result in an increase of work for the team members themselves, work which is also unevenly balanced between the members by the way (e.g. I think much of this falls to Niko and myself). If that is the consequence then I think that is in fact an undue burden. I personally cannot add more hours to the essentially unpaid (more-than-)full time work I'm doing today.

[Lokathor]

I would expect that the additional organization tracking effort would come from triage. They already have their ear to the ground on all things Rust to begin with.

Naturally, if everything is already working well then triage won't even have to ask extra questions from team/group members because it'll all be well documented in the ways that you say. There would only need to be "hey tell me what's going on lately" questions if the suggested methods don't pan out (which will naturally happen from time to time when people get extra busy).

Of course, I'm also told that they feel a little overwhelmed sometimes, so as always "we need to increase our bandwidth" pops up again. At least with triage it's a lot easier to on-board new members.

[nikomatsakis]

I agree with @mark-i-m but for I would add a few different, if complementary, reasons. My sense is that it would be great if we can do a better job of exposing what is going on not only for newcomers but for the team members. I think it's hard to keep up and having more communication about "these are the things we are tracking" would help us in doing so. It would also help us ensure that we're paying attention to the things we mean to and that we're achieving good progress towards our objectives (it's very easy to get lost in reactive work, I think).

In the lang team meetings, for example, I think that the project group focus has helped me to have a sense of what's going on, although I think the current setup isn't ideal (and in particular it often doesn't include things that should still, imo, be tracked, like tracking issues and features that we are moving to stability).

Similarly, one of the motivations behind the compiler-team working groups was the idea "we should absolutely be writing down what is actively happening" and making it obvious where to go to ask questions about it.

I don't really think we've hit the nail on the head here in terms of setup, and as a result, things aren't getting used quite as much as I would like. But I think the goal is useful and important and worthy of further consideration.

On that final point, I definitely agree with @Centril that the "busywork factor" should not be overlooked. I am obviously a big fan of IntoRust and so forth, but I've been negligent at posting some announcement style posts lately, because there's just a lot to keep up with. I suspect there's room for us to innovate a bit on how to handle these things in a lightweight fashion.

[jplatte]

A note on authorship: This RFC was jointly edited. Particular thanks to @Mark-Simulacrum, @nrc, and @aidanhs for their feedback and edits.

By the way: You can encode this in a git commit by adding

Co-authored-by: Other person <[email protected]>
Co-authored-by: Yet another person <[email protected]>

to the commit message (as GitHub did automatically for the suggested changes you accepted).

[matthiasbeyer]

Co-authored-by: Other person <[email protected]>
Co-authored-by: Yet another person <[email protected]>

Minor nitpick: https://git.wiki.kernel.org/index.php/CommitMessageConventions suggests CamelCase for this git trailer: Co-Authored-By: Other Person <[email protected]>.

[Mark-Simulacrum]

I have pushed up a few new commits which try to incorporate the outstanding feedback, though for all elements I pretty much feel that for the most part it is mostly broadening the RFC into discussion of the general idea rather than scope-setting, i.e., not normative content.

[nikomatsakis]

@rfcbot fcp merge

Thanks to everyone for the feedback and participation. This thread seems to have approached a stable state, so I'm moving that we merge this RFC.

[rfcbot]

Team member @nikomatsakis has proposed to merge this. The next step is review by the rest of the tagged team members:

• @Manishearth
• @Mark-Simulacrum
• @aidanhs
• @alexcrichton
• @ashleygwilliams
• @carols10cents
• @nikomatsakis
• @nrc
• @pietroalbini
• @steveklabnik

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[rfcbot]

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

The RFC will be merged soon.

[Manishearth]

Thanks everyone!

[Diggsey]

The roadmap says that the Rust 2019 survey results will be released in January. Unless someone in the core team has a time machine that seems unlikely 😛

[Manishearth]

wanna make a PR fixing that?

[XAMPPRocky]

@Diggsey We have an outline of the post about the survey results, and are actively working on it. It hasn't been forgotten.

[mikeyhew]

Can someone update the Rendered link to point to https://github.com/rust-lang/rfcs/blob/master/text/2857-roadmap-2020.md?

[XAMPPRocky]

@mikeyhew Done.