Move the extended lifetime resolution into typeck context #95563

dingxiangfei2009 · 2022-04-01T13:20:23Z

Related to #15023

This PR is based on the idea of #15023 by @nikomatsakis.

This PR specifically proposes to

Delay the resolution of scopes of rvalues to a later stage, so that enough type information is available to refine those scopes based on relationships of lifetimes.
Highlight relevant parts that would help future reviews on the next installments of works to fully implement a solution to RFC 66.

rust-highfive · 2022-04-01T13:20:26Z

r? @estebank

(rust-highfive has picked a reviewer for you, use r? to override)

nikomatsakis · 2022-04-05T07:46:59Z

r? @nikomatsakis

bors · 2022-04-13T14:19:37Z

☔ The latest upstream changes (presumably #95656) made this pull request unmergeable. Please resolve the merge conflicts.

bors · 2022-04-16T07:30:21Z

☔ The latest upstream changes (presumably #94468) made this pull request unmergeable. Please resolve the merge conflicts.

nikomatsakis

Left some nits, but the bigger question is: can we move the "rvalue candidate" stuff into the same place that the scopes are computed? (typeck)

I think we will want that eventually, because the candidates will be influenced by the results of type check.

nikomatsakis · 2022-04-19T14:52:06Z

compiler/rustc_middle/src/middle/region.rs

+    /// `rvalue_candidates` includes entries that are selected for extended
+    /// lifetime according to rvalue scoping rules.
+    /// Those candidates will be considered for extension of a lifetime,
+    /// whose cleanup scope will be larger than the default.
+    /// For rvalues not present in this table,
+    /// the appropriate cleanup scope is the innermost enclosing statement,
+    /// conditional expression, or repeating block (see `terminating_scopes`).


Suggested change

/// `rvalue_candidates` includes entries that are selected for extended

/// lifetime according to rvalue scoping rules.

/// Those candidates will be considered for extension of a lifetime,

/// whose cleanup scope will be larger than the default.

/// For rvalues not present in this table,

/// the appropriate cleanup scope is the innermost enclosing statement,

/// conditional expression, or repeating block (see `terminating_scopes`).

/// Identifies expressions which, if captured into a temporary, ought to

/// have a temporary whose lifetime extends to the end of the enclosing *block*,

/// and not the enclosing *statement*. (See `RvalueCandidate` for more information.)

compiler/rustc_middle/src/middle/region.rs

nikomatsakis · 2022-04-19T14:55:17Z

compiler/rustc_typeck/src/check/generator_interior.rs

@@ -189,10 +190,15 @@ pub fn resolve_interior<'a, 'tcx>(
    kind: hir::GeneratorKind,
 ) {
    let body = fcx.tcx.hir().body(body_id);
+    let typeck_results = fcx.inh.typeck_results.borrow();
+    let Some(rvalue_scopes) = &typeck_results.rvalue_scopes else {
+        bug!("rvalue scope information is not avilable yet")


Suggested change

bug!("rvalue scope information is not avilable yet")

bug!("rvalue scope information is not available yet")

maybe make this a span_bug? always better.

compiler/rustc_typeck/src/check/rvalue_scopes.rs

nikomatsakis · 2022-04-19T15:00:16Z

compiler/rustc_mir_build/src/thir/cx/mod.rs

@@ -64,12 +65,16 @@ struct Cx<'tcx> {
 impl<'tcx> Cx<'tcx> {
    fn new(tcx: TyCtxt<'tcx>, def: ty::WithOptConstParam<LocalDefId>) -> Cx<'tcx> {
        let typeck_results = tcx.typeck_opt_const_arg(def);
+        let Some(rvalue_scopes) = &typeck_results.rvalue_scopes else {
+            bug!("rvalue scope information is not available yet")


Suggested change

bug!("rvalue scope information is not available yet")

bug!("rvalue scope information is not available yet")

span bug!

nikomatsakis · 2022-04-19T15:00:53Z

compiler/rustc_middle/src/ty/rvalue_scopes.rs

+use crate::middle::region::{Scope, ScopeData, ScopeTree};
+use rustc_hir as hir;
+
+use super::context::RvalueScopes;


Why not move the definition into this file and pub use it from the super module, if necessary?

nikomatsakis · 2022-04-19T15:01:10Z

compiler/rustc_middle/src/ty/context.rs

@@ -367,6 +368,11 @@ pub struct GeneratorInteriorTypeCause<'tcx> {
    pub expr: Option<hir::HirId>,
 }

+#[derive(TyEncodable, TyDecodable, Clone, Debug, Eq, PartialEq, HashStable)]


This deserves a comment for sure =)

nikomatsakis · 2022-04-19T15:01:38Z

compiler/rustc_middle/src/ty/context.rs

@@ -367,6 +368,11 @@ pub struct GeneratorInteriorTypeCause<'tcx> {
    pub expr: Option<hir::HirId>,
 }

+#[derive(TyEncodable, TyDecodable, Clone, Debug, Eq, PartialEq, HashStable)]
+pub struct RvalueScopes {
+    pub inner: FxHashMap<hir::ItemLocalId, Option<Scope>>,


Suggested change

pub inner: FxHashMap<hir::ItemLocalId, Option<Scope>>,

map: FxHashMap<hir::ItemLocalId, Option<Scope>>,

Nit: I'd move the things that manipulate the map to methods on RvalueScopes (and move this definition over to where its impl is)

Yes, I think making it pub was actually not necessary.

bors · 2022-05-09T22:31:18Z

☔ The latest upstream changes (presumably #96473) made this pull request unmergeable. Please resolve the merge conflicts.

bors · 2022-05-18T23:19:11Z

☔ The latest upstream changes (presumably #97019) made this pull request unmergeable. Please resolve the merge conflicts.

bors · 2022-05-21T09:27:07Z

☔ The latest upstream changes (presumably #97239) made this pull request unmergeable. Please resolve the merge conflicts.

nikomatsakis

Yes, this looks great!

compiler/rustc_middle/src/middle/region.rs

nikomatsakis · 2022-05-21T14:57:29Z

@bors delegate+

@dingxiangfei2009 This is r=me but I can't r+ because it needs a rebase. I'm delegating to you so that you can rebase and then do bors r+

bors · 2022-05-21T14:57:31Z

✌️ @dingxiangfei2009 can now approve this pull request

bors · 2022-05-21T23:07:47Z

💔 Test failed - checks-actions

remove region_scope_tree from RegionCtxt Apply suggestions from code review Co-authored-by: Niko Matsakis <[email protected]>

dingxiangfei2009 · 2022-05-22T08:49:24Z

@bors r-
@bors r=nikomatsakis

Let me try this again. 😅

bors · 2022-05-22T08:49:26Z

📌 Commit 6044fbe has been approved by nikomatsakis

bors · 2022-05-22T09:00:33Z

⌛ Testing commit 6044fbe with merge 6534637...

bors · 2022-05-22T11:26:54Z

☀️ Test successful - checks-actions
Approved by: nikomatsakis
Pushing 6534637 to master...

rust-timer · 2022-05-22T12:46:34Z

Finished benchmarking commit (6534637): comparison url.

Instruction count

Primary benchmarks: mixed results
Secondary benchmarks: mixed results

	Regressions 😿 (primary)	Regressions 😿 (secondary)	Improvements 🎉 (primary)	Improvements 🎉 (secondary)	All 😿 🎉 (primary)
count¹	97	42	42	34	139
mean²	1.8%	3.0%	-0.6%	-0.5%	1.1%
max	5.8%	8.9%	-1.9%	-1.4%	5.8%

Max RSS (memory usage)

Results

Primary benchmarks: mixed results
Secondary benchmarks: mixed results

	Regressions 😿 (primary)	Regressions 😿 (secondary)	Improvements 🎉 (primary)	Improvements 🎉 (secondary)	All 😿 🎉 (primary)
count¹	62	31	17	9	79
mean²	3.5%	3.7%	-2.1%	-1.5%	2.3%
max	11.8%	8.3%	-6.6%	-2.2%	11.8%

Cycles

Results

Primary benchmarks: 😿 relevant regressions found
Secondary benchmarks: 😿 relevant regressions found

	Regressions 😿 (primary)	Regressions 😿 (secondary)	Improvements 🎉 (primary)	Improvements 🎉 (secondary)	All 😿 🎉 (primary)
count¹	31	25	3	0	34
mean²	2.5%	2.8%	-1.1%	N/A	2.2%
max	4.7%	5.4%	-1.1%	N/A	4.7%

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

Next Steps: If you can justify the regressions found in this perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please open an issue or create a new PR that fixes the regressions, add a comment linking to the newly created issue or PR, and then add the perf-regression-triaged label to this PR.

@rustbot label: +perf-regression

number of relevant changes ↩ ↩² ↩³
the arithmetic mean of the percent change ↩ ↩² ↩³

Mark-Simulacrum · 2022-05-24T14:22:09Z

@nikomatsakis @dingxiangfei2009 It looks like this PR was a regression across a fairly large number of incremental benchmarks -- I don't see a previous perf run done on this PR or discussion of performance as part of this change. Was this regression expected at an architectural level?

It looks like non-incremental benchmarks actually improved, so perhaps we're not caching something we were before?

dingxiangfei2009 · 2022-05-24T14:38:40Z

compiler/rustc_middle/src/query/mod.rs

@@ -1048,12 +1048,6 @@ rustc_queries! {
        desc { "reachability" }
    }

-    /// Per-body `region::ScopeTree`. The `DefId` should be the owner `DefId` for the body;
-    /// in the case of closures, this will be redirected to the enclosing function.
-    query region_scope_tree(def_id: DefId) -> &'tcx region::ScopeTree {


@Mark-Simulacrum I think by removing region_scope_tree from the list of queries, I suspect that the caching is lost since this computation could get repeated.

dingxiangfei2009 · 2022-05-24T14:40:57Z

I have a question now. Is there a way to cache things by keys of type DefId without queries? Or keeping it as a query is the way to go?

I will make a PR to restore that query and see if the perf numbers are restored.

pnkfelix · 2022-05-24T16:26:24Z

In addition to the instruction count regression, it seems like it also caused a regression to the compiler's memory usage (max-rss):

https://perf.rust-lang.org/compare.html?start=acfd327fd4e3a302ebb0a077f422a527a7935333&end=653463731a7f01f519cf85f444869def27f00395&stat=max-rss

Will restoring the query also address the max-rss regression?

…-tree-query, r=dingxiangfei2009 Try to cache region_scope_tree as a query This PR will attempt to restore `region_scope_tree` as a query so that caching works again. It seems that `region_scope_tree` could be re-computed for nested items after all, which could explain the performance regression introduced by rust-lang#95563. cc `@Mark-Simulacrum` `@pnkfelix` I will try to trigger a perf run here.

rust-highfive assigned estebank Apr 1, 2022

rustbot added the T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. label Apr 1, 2022

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Apr 1, 2022

This comment has been minimized.

Sign in to view

dingxiangfei2009 force-pushed the dxf-rfc66-refactor branch from 8b98476 to 602c945 Compare April 1, 2022 13:38

rust-highfive assigned nikomatsakis and unassigned estebank Apr 5, 2022

dingxiangfei2009 force-pushed the dxf-rfc66-refactor branch from 602c945 to 30c8717 Compare April 15, 2022 16:25

dingxiangfei2009 force-pushed the dxf-rfc66-refactor branch from 30c8717 to bee6bd1 Compare April 18, 2022 15:49

nikomatsakis requested changes Apr 19, 2022

View reviewed changes

dingxiangfei2009 force-pushed the dxf-rfc66-refactor branch from bee6bd1 to 6547d94 Compare April 19, 2022 15:53

dingxiangfei2009 force-pushed the dxf-rfc66-refactor branch 3 times, most recently from bea3352 to 3957b0a Compare May 8, 2022 14:15

dingxiangfei2009 requested a review from nikomatsakis May 8, 2022 14:16

dingxiangfei2009 force-pushed the dxf-rfc66-refactor branch 2 times, most recently from 49783a5 to 3091472 Compare May 12, 2022 09:52

dingxiangfei2009 force-pushed the dxf-rfc66-refactor branch from 3091472 to 1460a75 Compare May 19, 2022 15:54

nikomatsakis approved these changes May 21, 2022

View reviewed changes

compiler/rustc_middle/src/middle/region.rs Outdated Show resolved Hide resolved

dingxiangfei2009 force-pushed the dxf-rfc66-refactor branch 2 times, most recently from 3ef602d to 0cb3dfe Compare May 21, 2022 15:12

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels May 21, 2022

This comment has been minimized.

Sign in to view

factor out the rvalue lifetime rule

6044fbe

remove region_scope_tree from RegionCtxt Apply suggestions from code review Co-authored-by: Niko Matsakis <[email protected]>

dingxiangfei2009 force-pushed the dxf-rfc66-refactor branch from 0cb3dfe to 6044fbe Compare May 22, 2022 08:47

bors removed the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label May 22, 2022

bors added the merged-by-bors This PR was explicitly merged by bors. label May 22, 2022

bors merged commit 6534637 into rust-lang:master May 22, 2022

rustbot added this to the 1.63.0 milestone May 22, 2022

rustbot added the perf-regression Performance regression. label May 22, 2022

dingxiangfei2009 deleted the dxf-rfc66-refactor branch May 23, 2022 09:49

dingxiangfei2009 commented May 24, 2022

View reviewed changes

dingxiangfei2009 mentioned this pull request May 25, 2022

Try to cache region_scope_tree as a query #97383

Merged

nikomatsakis mentioned this pull request May 26, 2022

Investigate eliminate region_scope_tree #97426

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move the extended lifetime resolution into typeck context #95563

Move the extended lifetime resolution into typeck context #95563

dingxiangfei2009 commented Apr 1, 2022 •

edited

Loading

rust-highfive commented Apr 1, 2022

This comment has been minimized.

nikomatsakis commented Apr 5, 2022

bors commented Apr 13, 2022

bors commented Apr 16, 2022

nikomatsakis left a comment

nikomatsakis Apr 19, 2022

nikomatsakis Apr 19, 2022

nikomatsakis Apr 19, 2022

nikomatsakis Apr 19, 2022

nikomatsakis Apr 19, 2022

nikomatsakis Apr 19, 2022

dingxiangfei2009 May 8, 2022

bors commented May 9, 2022

bors commented May 18, 2022

bors commented May 21, 2022

nikomatsakis left a comment

nikomatsakis commented May 21, 2022

bors commented May 21, 2022

bors commented May 21, 2022

This comment has been minimized.

dingxiangfei2009 commented May 22, 2022

bors commented May 22, 2022

bors commented May 22, 2022

bors commented May 22, 2022

rust-timer commented May 22, 2022

Mark-Simulacrum commented May 24, 2022

dingxiangfei2009 May 24, 2022

dingxiangfei2009 commented May 24, 2022 •

edited

Loading

pnkfelix commented May 24, 2022 •

edited

Loading

	bug!("rvalue scope information is not avilable yet")
	bug!("rvalue scope information is not available yet")

	pub inner: FxHashMap<hir::ItemLocalId, Option<Scope>>,
	map: FxHashMap<hir::ItemLocalId, Option<Scope>>,

Move the extended lifetime resolution into typeck context #95563

Move the extended lifetime resolution into typeck context #95563

Conversation

dingxiangfei2009 commented Apr 1, 2022 • edited Loading

rust-highfive commented Apr 1, 2022

This comment has been minimized.

nikomatsakis commented Apr 5, 2022

bors commented Apr 13, 2022

bors commented Apr 16, 2022

nikomatsakis left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bors commented May 9, 2022

bors commented May 18, 2022

bors commented May 21, 2022

nikomatsakis left a comment

Choose a reason for hiding this comment

nikomatsakis commented May 21, 2022

bors commented May 21, 2022

bors commented May 21, 2022

This comment has been minimized.

dingxiangfei2009 commented May 22, 2022

bors commented May 22, 2022

bors commented May 22, 2022

bors commented May 22, 2022

rust-timer commented May 22, 2022

Footnotes

Mark-Simulacrum commented May 24, 2022

Choose a reason for hiding this comment

dingxiangfei2009 commented May 24, 2022 • edited Loading

pnkfelix commented May 24, 2022 • edited Loading

dingxiangfei2009 commented Apr 1, 2022 •

edited

Loading

dingxiangfei2009 commented May 24, 2022 •

edited

Loading

pnkfelix commented May 24, 2022 •

edited

Loading