Add a query cache for dense local DefIds #69303

Zoxc · 2020-02-19T22:48:04Z

Waiting on #68988 to land for a perf run.

r? @eddyb

Zoxc · 2020-02-20T11:37:42Z

@bors try @rust-timer queue

rust-timer · 2020-02-20T11:37:43Z

Awaiting bors try build completion

bors · 2020-02-20T11:37:53Z

⌛ Trying commit 948880c95b0e5c8501c490bf18b5f06ea557acc7 with merge 26c75c5e3b892d085b4815677fd07f9f84583c4c...

bors · 2020-02-20T14:17:21Z

☀️ Try build successful - checks-azure
Build commit: 26c75c5e3b892d085b4815677fd07f9f84583c4c (26c75c5e3b892d085b4815677fd07f9f84583c4c)

rust-timer · 2020-02-20T14:17:23Z

Queued 26c75c5e3b892d085b4815677fd07f9f84583c4c with parent de362d8, future comparison URL.

rust-timer · 2020-02-20T18:47:45Z

Finished benchmarking try commit 26c75c5e3b892d085b4815677fd07f9f84583c4c, comparison URL.

Centril · 2020-03-11T08:42:03Z

Looks like perf was regressed?

Zoxc · 2020-03-16T09:26:30Z

@bors try @rust-timer queue

rust-timer · 2020-03-16T09:26:31Z

Awaiting bors try build completion

bors · 2020-03-16T09:26:42Z

⌛ Trying commit 9bac16a with merge bbed581aa2e500028b8ff65f2f35aa8a64c00948...

rust-highfive · 2020-03-16T10:00:35Z

The job x86_64-gnu-llvm-7 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

2020-03-16T09:23:02.1000826Z ========================== Starting Command Output ===========================
2020-03-16T09:23:02.1005073Z [command]/bin/bash --noprofile --norc /home/vsts/work/_temp/a97f3e21-087c-44f0-b400-9445f7412c75.sh
2020-03-16T09:23:02.1005698Z 
2020-03-16T09:23:02.1010405Z ##[section]Finishing: Disable git automatic line ending conversion
2020-03-16T09:23:02.1030923Z ##[section]Starting: Checkout rust-lang/rust@refs/pull/69303/merge to s
2020-03-16T09:23:02.1034491Z Task         : Get sources
2020-03-16T09:23:02.1034816Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2020-03-16T09:23:02.1035132Z Version      : 1.0.0
2020-03-16T09:23:02.1035346Z Author       : Microsoft
---
2020-03-16T09:23:03.3028942Z ##[command]git remote add origin https://github.com/rust-lang/rust
2020-03-16T09:23:03.3043255Z ##[command]git config gc.auto 0
2020-03-16T09:23:03.3073297Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2020-03-16T09:23:03.3081111Z ##[command]git config --get-all http.proxy
2020-03-16T09:23:03.3092606Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/69303/merge:refs/remotes/pull/69303/merge
---
2020-03-16T09:59:53.9683560Z    Compiling rustc_parse v0.0.0 (/checkout/src/librustc_parse)
2020-03-16T09:59:55.4816292Z error: unnecessary parentheses around type
2020-03-16T09:59:55.4817705Z     --> src/librustc/query/mod.rs:78:20
2020-03-16T09:59:55.4818581Z      |
2020-03-16T09:59:55.4819520Z 38   | / rustc_queries! {
2020-03-16T09:59:55.4820644Z 39   | |     Other {
2020-03-16T09:59:55.4821927Z 40   | |         query trigger_delay_span_bug(key: DefId) -> () {
2020-03-16T09:59:55.4823357Z 41   | |             desc { "trigger a delay span bug" }
2020-03-16T09:59:55.4825131Z ...    |
2020-03-16T09:59:55.4826150Z 78   | |             storage(caches::LocalDenseDefIdCacheSelector<&'tcx HirOwner<'tcx>>)
2020-03-16T09:59:55.4828769Z ...    |
2020-03-16T09:59:55.4829543Z 1257 | |     }
2020-03-16T09:59:55.4830375Z 1258 | | }
2020-03-16T09:59:55.4830375Z 1258 | | }
2020-03-16T09:59:55.4836994Z      | |_- in this expansion of `rustc_query_append!`
2020-03-16T09:59:55.4838741Z     ::: src/librustc/ty/query/mod.rs:110:1
2020-03-16T09:59:55.4839403Z      |
2020-03-16T09:59:55.4839403Z      |
2020-03-16T09:59:55.4840152Z 110  |   rustc_query_append! { [define_queries!][<'tcx>] }
2020-03-16T09:59:55.4845195Z      |
2020-03-16T09:59:55.4845936Z      = note: `-D unused-parens` implied by `-D warnings`
2020-03-16T09:59:55.4853038Z 
2020-03-16T09:59:55.4881376Z error: unnecessary parentheses around type
2020-03-16T09:59:55.4881376Z error: unnecessary parentheses around type
2020-03-16T09:59:55.4882130Z     --> src/librustc/query/mod.rs:87:20
2020-03-16T09:59:55.4882743Z      |
2020-03-16T09:59:55.4883466Z 38   | / rustc_queries! {
2020-03-16T09:59:55.4884339Z 39   | |     Other {
2020-03-16T09:59:55.4885446Z 40   | |         query trigger_delay_span_bug(key: DefId) -> () {
2020-03-16T09:59:55.4886574Z 41   | |             desc { "trigger a delay span bug" }
2020-03-16T09:59:55.4887548Z ...    |
2020-03-16T09:59:55.4888661Z 87   | |             storage(caches::LocalDenseDefIdCacheSelector<&'tcx HirOwnerItems<'tcx>>)
2020-03-16T09:59:55.4891299Z ...    |
2020-03-16T09:59:55.4892052Z 1257 | |     }
2020-03-16T09:59:55.4893063Z 1258 | | }
2020-03-16T09:59:55.4893063Z 1258 | | }
2020-03-16T09:59:55.4894007Z      | |_- in this expansion of `rustc_query_append!`
2020-03-16T09:59:55.4895373Z     ::: src/librustc/ty/query/mod.rs:110:1
2020-03-16T09:59:55.4895962Z      |
2020-03-16T09:59:55.4895962Z      |
2020-03-16T09:59:55.4896733Z 110  |   rustc_query_append! { [define_queries!][<'tcx>] }
2020-03-16T09:59:55.4902303Z 
2020-03-16T10:00:11.0334921Z error: aborting due to 2 previous errors
2020-03-16T10:00:11.0335975Z 
2020-03-16T10:00:11.0336717Z error: could not compile `rustc`.
---
2020-03-16T10:00:29.0422398Z   local time: Mon Mar 16 10:00:29 UTC 2020
2020-03-16T10:00:29.5975731Z   network time: Mon, 16 Mar 2020 10:00:29 GMT
2020-03-16T10:00:29.5976335Z == end clock drift check ==
2020-03-16T10:00:30.1296762Z 
2020-03-16T10:00:30.1382695Z ##[error]Bash exited with code '1'.
2020-03-16T10:00:30.1399191Z ##[section]Finishing: Run build
2020-03-16T10:00:30.1519428Z ##[section]Starting: Checkout rust-lang/rust@refs/pull/69303/merge to s
2020-03-16T10:00:30.1525348Z Task         : Get sources
2020-03-16T10:00:30.1525762Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2020-03-16T10:00:30.1526128Z Version      : 1.0.0
2020-03-16T10:00:30.1526465Z Author       : Microsoft
2020-03-16T10:00:30.1526465Z Author       : Microsoft
2020-03-16T10:00:30.1526882Z Help         : [More Information](https://go.microsoft.com/fwlink/?LinkId=798199)
2020-03-16T10:00:30.1527369Z ==============================================================================
2020-03-16T10:00:30.5460283Z Cleaning any cached credential from repository: rust-lang/rust (GitHub)
2020-03-16T10:00:30.5537887Z ##[section]Finishing: Checkout rust-lang/rust@refs/pull/69303/merge to s
2020-03-16T10:00:30.5643817Z Cleaning up task key
2020-03-16T10:00:30.5645066Z Start cleaning up orphan processes.
2020-03-16T10:00:30.6947161Z Terminate orphan process: pid (3819) (python)
2020-03-16T10:00:30.7003760Z ##[section]Finishing: Finalize Job

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @rust-lang/infra. (Feature Requests)

rust-highfive · 2020-03-16T10:47:13Z

The job dist-x86_64-linux-alt of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

2020-03-16T10:46:19.3153616Z    Compiling rustc_parse v0.0.0 (/checkout/src/librustc_parse)
2020-03-16T10:46:22.3793866Z error: unnecessary parentheses around type
2020-03-16T10:46:22.3795511Z     --> src/librustc/query/mod.rs:78:20
2020-03-16T10:46:22.3796198Z      |
2020-03-16T10:46:22.3796946Z 38   | / rustc_queries! {
2020-03-16T10:46:22.3797855Z 39   | |     Other {
2020-03-16T10:46:22.3799148Z 40   | |         query trigger_delay_span_bug(key: DefId) -> () {
2020-03-16T10:46:22.3800576Z 41   | |             desc { "trigger a delay span bug" }
2020-03-16T10:46:22.3801424Z ...    |
2020-03-16T10:46:22.3802461Z 78   | |             storage(caches::LocalDenseDefIdCacheSelector<&'tcx HirOwner<'tcx>>)
2020-03-16T10:46:22.3805077Z ...    |
2020-03-16T10:46:22.3805838Z 1257 | |     }
2020-03-16T10:46:22.3806692Z 1258 | | }
2020-03-16T10:46:22.3806692Z 1258 | | }
2020-03-16T10:46:22.3807639Z      | |_- in this expansion of `rustc_query_append!`
2020-03-16T10:46:22.3809013Z     ::: src/librustc/ty/query/mod.rs:110:1
2020-03-16T10:46:22.3809637Z      |
2020-03-16T10:46:22.3809637Z      |
2020-03-16T10:46:22.3810404Z 110  |   rustc_query_append! { [define_queries!][<'tcx>] }
2020-03-16T10:46:22.3812273Z      |
2020-03-16T10:46:22.3813031Z      = note: `-D unused-parens` implied by `-D warnings`
2020-03-16T10:46:22.3813412Z 
2020-03-16T10:46:22.3847681Z error: unnecessary parentheses around type
2020-03-16T10:46:22.3847681Z error: unnecessary parentheses around type
2020-03-16T10:46:22.3848561Z     --> src/librustc/query/mod.rs:87:20
2020-03-16T10:46:22.3849169Z      |
2020-03-16T10:46:22.3849932Z 38   | / rustc_queries! {
2020-03-16T10:46:22.3850831Z 39   | |     Other {
2020-03-16T10:46:22.3851914Z 40   | |         query trigger_delay_span_bug(key: DefId) -> () {
2020-03-16T10:46:22.3853094Z 41   | |             desc { "trigger a delay span bug" }
2020-03-16T10:46:22.3853892Z ...    |
2020-03-16T10:46:22.3854962Z 87   | |             storage(caches::LocalDenseDefIdCacheSelector<&'tcx HirOwnerItems<'tcx>>)
2020-03-16T10:46:22.3857875Z ...    |
2020-03-16T10:46:22.3858650Z 1257 | |     }
2020-03-16T10:46:22.3859497Z 1258 | | }
2020-03-16T10:46:22.3859497Z 1258 | | }
2020-03-16T10:46:22.3860521Z      | |_- in this expansion of `rustc_query_append!`
2020-03-16T10:46:22.3861979Z     ::: src/librustc/ty/query/mod.rs:110:1
2020-03-16T10:46:22.3862587Z      |
2020-03-16T10:46:22.3862587Z      |
2020-03-16T10:46:22.3863368Z 110  |   rustc_query_append! { [define_queries!][<'tcx>] }
2020-03-16T10:46:22.3864940Z 
2020-03-16T10:46:36.2374925Z error: aborting due to 2 previous errors
2020-03-16T10:46:36.2375218Z 
2020-03-16T10:46:42.4615748Z    Compiling rustc_ast_passes v0.0.0 (/checkout/src/librustc_ast_passes)
---
2020-03-16T10:47:07.3362261Z   local time: Mon Mar 16 10:47:07 UTC 2020
2020-03-16T10:47:07.9478792Z   network time: Mon, 16 Mar 2020 10:47:07 GMT
2020-03-16T10:47:07.9479250Z == end clock drift check ==
2020-03-16T10:47:08.5047517Z 
2020-03-16T10:47:08.5147377Z ##[error]Bash exited with code '1'.
2020-03-16T10:47:08.5238686Z ##[section]Starting: Checkout rust-lang/rust@try to s
2020-03-16T10:47:08.5244791Z ==============================================================================
2020-03-16T10:47:08.5245212Z Task         : Get sources
2020-03-16T10:47:08.5245649Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @rust-lang/infra. (Feature Requests)

Zoxc · 2020-03-16T11:10:53Z

@bors try

bors · 2020-03-16T11:11:08Z

⌛ Trying commit 40567e6 with merge 62c144a0ca9aff8af0b2c73b944dd21cca2f0007...

cjgillot · 2020-03-16T11:45:45Z

src/librustc/ty/query/caches.rs

 }

-#[derive(Default)]
-pub struct DefaultCache;
+pub struct DefaultCache<D>(PhantomData<D>);


What does this generic parameter D do?

Avoids compiler errors for the pub type LocalDenseDefIdCacheSelector<V> alias.

bors · 2020-03-16T13:51:41Z

☀️ Try build successful - checks-azure
Build commit: 62c144a0ca9aff8af0b2c73b944dd21cca2f0007 (62c144a0ca9aff8af0b2c73b944dd21cca2f0007)

rust-timer · 2020-03-16T13:51:43Z

Queued 62c144a0ca9aff8af0b2c73b944dd21cca2f0007 with parent 97eda01, future comparison URL.

Dylan-DPC-zz · 2020-04-09T18:41:33Z

@Zoxc can you resolve conflicts? we can get this reviewed

Zoxc · 2020-04-09T22:05:39Z

@eddyb Is this not what you had in mind?

eddyb · 2020-04-09T22:10:55Z

@Zoxc Oops, I failed to read which PR this was on, for some reason I thought this one landed.

eddyb · 2020-04-09T22:13:35Z

r=me after rebase

Zoxc · 2020-04-09T22:19:35Z

Perf is still an issue with this one though =P

eddyb · 2020-04-09T22:22:20Z

src/librustc/query/mod.rs

@@ -75,6 +75,7 @@ rustc_queries! {
        // This can be conveniently accessed by methods on `tcx.hir()`.
        // Avoid calling this query directly.
        query hir_owner(key: DefId) -> &'tcx HirOwner<'tcx> {


Change the query keys to LocalDefId, then you won't have any is_local checks, maybe that helps?

They are LocalDefId in master now. I don't expect removing a single branch to help much though.

nikomatsakis · 2020-04-10T18:13:57Z

Closing this pull request as Zoxc is stepping back from compiler development; see rust-lang/team#316.

rust-highfive assigned eddyb Feb 19, 2020

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Feb 19, 2020

Zoxc force-pushed the query-caches-apply branch from e4a34ca to 948880c Compare February 20, 2020 11:00

Zoxc mentioned this pull request Mar 11, 2020

Use queries for the HIR map #68944

Merged

Dylan-DPC-zz added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 11, 2020

Zoxc added 2 commits March 16, 2020 10:17

Add LocalDenseDefIdCache

2e1eb89

Add storage attribute to HIR queries

9bac16a

Zoxc force-pushed the query-caches-apply branch from 948880c to 9bac16a Compare March 16, 2020 09:22

Exclude parenthesis for storage query attribute

40567e6

cjgillot reviewed Mar 16, 2020

View reviewed changes

joelpalmer added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Mar 30, 2020

joelpalmer added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Apr 9, 2020

This comment has been minimized.

Sign in to view

eddyb approved these changes Apr 9, 2020

View reviewed changes

eddyb reviewed Apr 9, 2020

View reviewed changes

nikomatsakis closed this Apr 10, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a query cache for dense local DefIds #69303

Add a query cache for dense local DefIds #69303

Zoxc commented Feb 19, 2020

Zoxc commented Feb 20, 2020

rust-timer commented Feb 20, 2020

bors commented Feb 20, 2020

bors commented Feb 20, 2020

rust-timer commented Feb 20, 2020

rust-timer commented Feb 20, 2020

Centril commented Mar 11, 2020

Zoxc commented Mar 16, 2020

rust-timer commented Mar 16, 2020

bors commented Mar 16, 2020

rust-highfive commented Mar 16, 2020

rust-highfive commented Mar 16, 2020

Zoxc commented Mar 16, 2020

bors commented Mar 16, 2020

cjgillot Mar 16, 2020

Zoxc Mar 16, 2020

bors commented Mar 16, 2020

rust-timer commented Mar 16, 2020

Dylan-DPC-zz commented Apr 9, 2020

This comment has been minimized.

Zoxc commented Apr 9, 2020

eddyb commented Apr 9, 2020

eddyb commented Apr 9, 2020

Zoxc commented Apr 9, 2020

eddyb Apr 9, 2020

Zoxc Apr 9, 2020

nikomatsakis commented Apr 10, 2020

Add a query cache for dense local DefIds #69303

Add a query cache for dense local DefIds #69303

Conversation

Zoxc commented Feb 19, 2020

Zoxc commented Feb 20, 2020

rust-timer commented Feb 20, 2020

bors commented Feb 20, 2020

bors commented Feb 20, 2020

rust-timer commented Feb 20, 2020

rust-timer commented Feb 20, 2020

Centril commented Mar 11, 2020

Zoxc commented Mar 16, 2020

rust-timer commented Mar 16, 2020

bors commented Mar 16, 2020

rust-highfive commented Mar 16, 2020

rust-highfive commented Mar 16, 2020

Zoxc commented Mar 16, 2020

bors commented Mar 16, 2020

cjgillot Mar 16, 2020

Choose a reason for hiding this comment

Zoxc Mar 16, 2020

Choose a reason for hiding this comment

bors commented Mar 16, 2020

rust-timer commented Mar 16, 2020

Dylan-DPC-zz commented Apr 9, 2020

This comment has been minimized.

Zoxc commented Apr 9, 2020

eddyb commented Apr 9, 2020

eddyb commented Apr 9, 2020

Zoxc commented Apr 9, 2020

eddyb Apr 9, 2020

Choose a reason for hiding this comment

Zoxc Apr 9, 2020

Choose a reason for hiding this comment

nikomatsakis commented Apr 10, 2020