rustc_metadata: Encode `doc(hidden)` flag to metadata #107136

petrochenkov · 2023-01-20T22:49:29Z

To retrieve these flags rustdoc currently has to mass decode full attributes for items in the whole crate tree, so it's better to pre-compute it in advance.

This is especially important for short-term performance of #107054 because resolver cannot use memoization of query results yet.

rustbot · 2023-01-20T22:49:35Z

r? @WaffleLapkin

(rustbot has picked a reviewer for you, use r? to override)

rustbot · 2023-01-20T22:49:39Z

Some changes occurred in src/librustdoc/clean/types.rs

cc @camelid

petrochenkov · 2023-01-20T22:49:53Z

@bors try @rust-timer queue

bors · 2023-01-20T22:50:02Z

⌛ Trying commit e30a7b14ed6e08633da6a17327dfc7304bb55881 with merge fef423fa15efb051084177cd1bc4a7994647d0e9...

bors · 2023-01-21T01:34:41Z

☀️ Try build successful - checks-actions
Build commit: fef423fa15efb051084177cd1bc4a7994647d0e9 (fef423fa15efb051084177cd1bc4a7994647d0e9)

rust-timer · 2023-01-21T03:25:45Z

Finished benchmarking commit (fef423fa15efb051084177cd1bc4a7994647d0e9): comparison URL.

Overall result: ❌✅ regressions and improvements - ACTION NEEDED

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	0.6%	[0.2%, 1.0%]	87
Regressions ❌ (secondary)	0.7%	[0.2%, 2.2%]	33
Improvements ✅ (primary)	-1.9%	[-5.8%, -0.4%]	18
Improvements ✅ (secondary)	-3.5%	[-5.9%, -0.9%]	27
All ❌✅ (primary)	0.1%	[-5.8%, 1.0%]	105

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	0.7%	[0.5%, 0.9%]	7
Regressions ❌ (secondary)	2.0%	[2.0%, 2.0%]	1
Improvements ✅ (primary)	-2.9%	[-5.3%, -1.0%]	15
Improvements ✅ (secondary)	-3.8%	[-6.1%, -2.0%]	24
All ❌✅ (primary)	-1.8%	[-5.3%, 0.9%]	22

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	3.4%	[2.9%, 3.9%]	2
Improvements ✅ (primary)	-3.1%	[-6.0%, -1.9%]	8
Improvements ✅ (secondary)	-4.1%	[-5.9%, -1.0%]	21
All ❌✅ (primary)	-3.1%	[-6.0%, -1.9%]	8

petrochenkov · 2023-01-21T08:48:38Z

The result is generally expected - large improvements for rustdoc, and smaller regressions for rustc metadata writing.

The question is why the metadata writing regressions are so large.
The time delta comes mostly from incr_comp_encode_dep_graph, not even is_doc_hidden.

petrochenkov · 2023-01-21T10:46:29Z

Let's try to remove the query call.
@bors try @rust-timer queue

bors · 2023-01-21T10:46:38Z

⌛ Trying commit c0f95bed535a0d8561144a6615c6517dab63e428 with merge 8679a365bb3d15cdc5d1577fd4082c435b3b2513...

bors · 2023-01-21T12:54:11Z

☀️ Try build successful - checks-actions
Build commit: 8679a365bb3d15cdc5d1577fd4082c435b3b2513 (8679a365bb3d15cdc5d1577fd4082c435b3b2513)

bors · 2023-01-21T12:54:11Z

☀️ Try build successful - checks-actions
Build commit: 8679a365bb3d15cdc5d1577fd4082c435b3b2513 (8679a365bb3d15cdc5d1577fd4082c435b3b2513)

rust-timer · 2023-01-21T15:24:27Z

Finished benchmarking commit (8679a365bb3d15cdc5d1577fd4082c435b3b2513): comparison URL.

Overall result: ✅ improvements - no action needed

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-1.7%	[-5.7%, -0.3%]	21
Improvements ✅ (secondary)	-3.4%	[-5.8%, -0.9%]	28
All ❌✅ (primary)	-1.7%	[-5.7%, -0.3%]	21

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	3.1%	[3.1%, 3.1%]	1
Improvements ✅ (primary)	-2.7%	[-4.9%, -0.8%]	15
Improvements ✅ (secondary)	-3.4%	[-6.0%, -1.9%]	23
All ❌✅ (primary)	-2.7%	[-4.9%, -0.8%]	15

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-3.0%	[-6.2%, -0.8%]	11
Improvements ✅ (secondary)	-4.6%	[-6.8%, -1.1%]	22
All ❌✅ (primary)	-3.0%	[-6.2%, -0.8%]	11

cjgillot · 2023-01-21T15:57:14Z

@bors r+

bors · 2023-01-21T15:57:16Z

📌 Commit c0f95bed535a0d8561144a6615c6517dab63e428 has been approved by cjgillot

It is now in the queue for this repository.

To retrieve these flags rustdoc currently has to mass decode full attributes for items in the whole crate tree, so it's better to pre-compute it in advance. This is especially for short-term performance of rust-lang#107054 because resolver cannot use memoization of query results yet.

petrochenkov · 2023-01-21T19:07:23Z

Renamed DocFlags to AttrFlags and rebased.
@bors r=cjgillot

bors · 2023-01-21T19:07:25Z

📌 Commit 415c141 has been approved by cjgillot

It is now in the queue for this repository.

bors · 2023-01-23T10:13:00Z

⌛ Testing commit 415c141 with merge 5bef91c...

bors · 2023-01-23T13:07:04Z

☀️ Test successful - checks-actions
Approved by: cjgillot
Pushing 5bef91c to master...

rust-timer · 2023-01-23T14:25:41Z

Finished benchmarking commit (5bef91c): comparison URL.

Overall result: ✅ improvements - no action needed

@rustbot label: -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.2%	[0.2%, 0.2%]	1
Improvements ✅ (primary)	-1.9%	[-5.7%, -0.5%]	18
Improvements ✅ (secondary)	-3.3%	[-5.8%, -0.8%]	28
All ❌✅ (primary)	-1.9%	[-5.7%, -0.5%]	18

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-3.0%	[-5.0%, -1.3%]	16
Improvements ✅ (secondary)	-4.2%	[-6.4%, -2.2%]	24
All ❌✅ (primary)	-3.0%	[-5.0%, -1.3%]	16

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	1.1%	[1.1%, 1.2%]	3
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-3.4%	[-6.9%, -1.6%]	10
Improvements ✅ (secondary)	-5.0%	[-7.2%, -1.9%]	20
All ❌✅ (primary)	-2.3%	[-6.9%, 1.2%]	13

This is a convenience feature for cases in which "no value in the table" and "default value in the table" are equivalent. Tables using `Table<DefIndex, ()>` are migrated in this PR, some other cases can be migrated later. This helps `DocFlags` in rust-lang#107136 in particular.

rustc_metadata: Support non-`Option` nullable values in metadata tables This is a convenience feature for cases in which "no value in the table" and "default value in the table" are equivalent. Tables using `Table<DefIndex, ()>` are migrated in this PR, some other cases can be migrated later. This helps `DocFlags` in rust-lang#107136 in particular.

rustc_metadata: Fix `encode_attrs` This function didn't do what the authors intended it to do. - Due to `move` in the closure `is_public` wasn't captured by mutalbe reference and wasn't used as a cache. - Due to iterator cloning all the `should_encode_attr` logic run for the second time to calculate `may_have_doc_links` This PR fixes these issues, and calculates all the needed attribute flags in one go. (Noticed while implementing rust-lang#107136.)

rustbot assigned WaffleLapkin Jan 20, 2023

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. labels Jan 20, 2023

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jan 20, 2023

This comment has been minimized.

Sign in to view

rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Jan 21, 2023

petrochenkov added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 21, 2023

petrochenkov force-pushed the dochidden branch from e30a7b1 to c0f95be Compare January 21, 2023 10:45

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jan 21, 2023

This comment has been minimized.

Sign in to view

rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jan 21, 2023

petrochenkov mentioned this pull request Jan 21, 2023

rustdoc: Collect "rustdoc-reachable" items during early doc link resolution #107054

Merged

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 21, 2023

This comment was marked as resolved.

Sign in to view

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jan 21, 2023

petrochenkov mentioned this pull request Jan 21, 2023

rustc_metadata: Support non-Option nullable values in metadata tables #107166

Merged

petrochenkov force-pushed the dochidden branch from c0f95be to 415c141 Compare January 21, 2023 19:05

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jan 21, 2023

petrochenkov mentioned this pull request Jan 21, 2023

rustc_metadata: Fix encode_attrs #107171

Merged

bors added the merged-by-bors This PR was explicitly merged by bors. label Jan 23, 2023

bors merged commit 5bef91c into rust-lang:master Jan 23, 2023

rustbot added this to the 1.69.0 milestone Jan 23, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

rustc_metadata: Encode `doc(hidden)` flag to metadata #107136

rustc_metadata: Encode `doc(hidden)` flag to metadata #107136

petrochenkov commented Jan 20, 2023

rustbot commented Jan 20, 2023

rustbot commented Jan 20, 2023

petrochenkov commented Jan 20, 2023

This comment has been minimized.

bors commented Jan 20, 2023

bors commented Jan 21, 2023

This comment has been minimized.

rust-timer commented Jan 21, 2023

petrochenkov commented Jan 21, 2023

petrochenkov commented Jan 21, 2023

This comment has been minimized.

bors commented Jan 21, 2023

bors commented Jan 21, 2023

bors commented Jan 21, 2023

This comment has been minimized.

rust-timer commented Jan 21, 2023

cjgillot commented Jan 21, 2023

bors commented Jan 21, 2023

This comment was marked as resolved.

petrochenkov commented Jan 21, 2023

bors commented Jan 21, 2023

bors commented Jan 23, 2023

bors commented Jan 23, 2023

rust-timer commented Jan 23, 2023

rustc_metadata: Encode doc(hidden) flag to metadata #107136

rustc_metadata: Encode doc(hidden) flag to metadata #107136

Conversation

petrochenkov commented Jan 20, 2023

rustbot commented Jan 20, 2023

rustbot commented Jan 20, 2023

petrochenkov commented Jan 20, 2023

This comment has been minimized.

bors commented Jan 20, 2023

bors commented Jan 21, 2023

This comment has been minimized.

rust-timer commented Jan 21, 2023

Overall result: ❌✅ regressions and improvements - ACTION NEEDED

petrochenkov commented Jan 21, 2023

petrochenkov commented Jan 21, 2023

This comment has been minimized.

bors commented Jan 21, 2023

bors commented Jan 21, 2023

bors commented Jan 21, 2023

This comment has been minimized.

rust-timer commented Jan 21, 2023

Overall result: ✅ improvements - no action needed

cjgillot commented Jan 21, 2023

bors commented Jan 21, 2023

This comment was marked as resolved.

petrochenkov commented Jan 21, 2023

bors commented Jan 21, 2023

bors commented Jan 23, 2023

bors commented Jan 23, 2023

rust-timer commented Jan 23, 2023

Overall result: ✅ improvements - no action needed

rustc_metadata: Encode `doc(hidden)` flag to metadata #107136

rustc_metadata: Encode `doc(hidden)` flag to metadata #107136