Optimize dep node backtrace and ignore fatal errors #108524

Zoxc · 2023-02-27T05:58:44Z

This attempts to optimize #91742 while also passing through fatal errors.

Zoxc · 2023-02-27T06:00:44Z

This also changes the output to use stderr and is closer to the query stack output. Example:

there was a panic while trying to force a dep node
try_mark_green dep node stack:
#0 hir_attrs (c2dc90d69dbb7f80-28f8bb30b7a6b1f0)
#1 entry_fn (0-0)
end of try_mark_green dep node stack

compiler-errors · 2023-03-01T03:17:43Z

@bors try @rust-timer queue

bors · 2023-03-01T03:17:52Z

⌛ Trying commit 5970f1aa22f9344e87cd8c190ffccdccf62b9385 with merge eadb1f6088ecc53b080a9c90867a22052ee7a433...

bors · 2023-03-01T05:48:01Z

☀️ Try build successful - checks-actions
Build commit: eadb1f6088ecc53b080a9c90867a22052ee7a433 (eadb1f6088ecc53b080a9c90867a22052ee7a433)

bors · 2023-03-01T05:48:01Z

☀️ Try build successful - checks-actions
Build commit: eadb1f6088ecc53b080a9c90867a22052ee7a433 (eadb1f6088ecc53b080a9c90867a22052ee7a433)

rust-timer · 2023-03-01T07:05:12Z

Finished benchmarking commit (eadb1f6088ecc53b080a9c90867a22052ee7a433): comparison URL.

Overall result: ✅ improvements - no action needed

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.4%	[-0.5%, -0.3%]	6
Improvements ✅ (secondary)	-0.2%	[-0.2%, -0.2%]	3
All ❌✅ (primary)	-0.4%	[-0.5%, -0.3%]	6

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.7%	[2.0%, 3.0%]	3
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-3.4%	[-3.4%, -3.4%]	1
All ❌✅ (primary)	-	-	0

Cycles

This benchmark run did not return any relevant results for this metric.

cjgillot · 2023-03-07T18:34:38Z

compiler/rustc_query_system/src/dep_graph/graph.rs

+    frame: Option<&MarkFrame<'_>>,
+) {
+    eprintln!("there was a panic while trying to force a dep node");
+    eprintln!("try_mark_green dep node stack:");


This needs to use handler.force_print_diagnostic to play nice with the rest of the infra, like print_query_stack does.

report_ice -> try_print_query_stack -> print_query_stack just prints to stderr like this. It does not rely on the session emitter at all.

compiler/rustc_query_system/src/dep_graph/graph.rs

compiler/rustc_query_system/src/dep_graph/mod.rs

bors · 2023-03-12T01:20:02Z

☔ The latest upstream changes (presumably #108417) made this pull request unmergeable. Please resolve the merge conflicts.

cjgillot · 2023-03-18T16:46:36Z

@bors r+

bors · 2023-03-18T16:46:37Z

📌 Commit f48ff4a has been approved by cjgillot

It is now in the queue for this repository.

bors · 2023-03-18T21:47:10Z

⌛ Testing commit f48ff4a with merge d178f7d928ee7e0218c148b7ec08c2ef25178755...

bors · 2023-03-18T21:53:41Z

💔 Test failed - checks-actions

ehuss · 2023-03-18T22:03:11Z

@bors retry
wasm SIGSEGV

rust-log-analyzer · 2023-03-18T23:59:30Z

The job wasm32 failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

Building stage1 library artifacts (x86_64-unknown-linux-gnu) 
error: failed to run `rustc` to learn about target-specific information

Caused by:
  process didn't exit successfully: `/checkout/obj/build/bootstrap/debug/rustc - --crate-name ___ --print=file-names -Csymbol-mangling-version=legacy -Zunstable-options -Zunstable-options '--check-cfg=values(bootstrap)' '--check-cfg=values(stdarch_intel_sde)' '--check-cfg=values(no_fp_fmt_parse)' '--check-cfg=values(no_global_oom_handling)' '--check-cfg=values(no_rc)' '--check-cfg=values(no_sync)' '--check-cfg=values(freebsd12)' '--check-cfg=values(backtrace_in_libstd)' '--check-cfg=values(target_env,"libnx")' '--check-cfg=values(target_arch,"asmjs","spirv","nvptx","xtensa")' -Zmacro-backtrace -Clink-args=-Wl,-z,origin '-Clink-args=-Wl,-rpath,$ORIGIN/../lib' -Csplit-debuginfo=off -Cprefer-dynamic -Cembed-bitcode=yes '-Zcrate-attr=doc(html_root_url="https://doc.rust-lang.org/nightly/")' --target x86_64-unknown-linux-gnu --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro --print=sysroot --print=split-debuginfo --print=crate-name --print=cfg` (exit status: 254)
  --- stderr
  /checkout/obj/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-fdca23ff32bf7124.so(+0x97f0b3)[0x7faee84e60b3]
  /lib/x86_64-linux-gnu/libc.so.6(+0x43090)[0x7faee77f2090]
  /checkout/obj/build/x86_64-unknown-linux-gnu/stage1/lib/libLLVM-16-rust-1.70.0-nightly.so(+0x7262e06)[0x7faee608fe06]
  /checkout/obj/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-fdca23ff32bf7124.so(+0xd10ad4)[0x7faee8877ad4]
  /checkout/obj/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-fdca23ff32bf7124.so(+0xbcb6cc)[0x7faee87326cc]
  /checkout/obj/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-fdca23ff32bf7124.so(+0xbca804)[0x7faee8731804]
  /checkout/obj/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-fdca23ff32bf7124.so(_RNvXs5_Cs7DTEPoQIU0M_18rustc_codegen_llvmNtB5_18LlvmCodegenBackendNtNtNtCsiydnvZ3aRh1_17rustc_codegen_ssa6traits7backend14CodegenBackend15target_features+0x1b)[0x7faee879183b]
  /checkout/obj/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-fdca23ff32bf7124.so(_RNvNtCsbMR7lNZiyEF_15rustc_interface4util17add_configuration+0x37)[0x7faee85d38f7]
  /checkout/obj/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-fdca23ff32bf7124.so(_RNvNtCsbMR7lNZiyEF_15rustc_interface4util14create_session+0x89d)[0x7faee85d439d]
  /checkout/obj/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-fdca23ff32bf7124.so(+0x989600)[0x7faee84f0600]
  /checkout/obj/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-fdca23ff32bf7124.so(+0x984380)[0x7faee84eb380]
  /checkout/obj/build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-fdca23ff32bf7124.so(+0x985ec3)[0x7faee84ecec3]
  /checkout/obj/build/x86_64-unknown-linux-gnu/stage1/lib/libstd-2268eb07c4da6a07.so(rust_metadata_std_e292dd3bffb96032+0xca4ae)[0x7faee7a6f4ae]
  /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609)[0x7faedec0d609]
  /lib/x86_64-linux-gnu/libc.so.6(clone+0x43)[0x7faee78ce133]
  [RUSTC-TIMING] ___ test:false 0.203
  rustc exited with signal: 11 (SIGSEGV) (core dumped)

bors · 2023-03-20T11:54:03Z

⌛ Testing commit f48ff4a with merge 13b7aa4...

bors · 2023-03-20T14:36:37Z

☀️ Test successful - checks-actions
Approved by: cjgillot
Pushing 13b7aa4 to master...

rust-timer · 2023-03-20T16:43:35Z

Finished benchmarking commit (13b7aa4): comparison URL.

Overall result: ❌✅ regressions and improvements - ACTION NEEDED

Next Steps: If you can justify the regressions found in this perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please open an issue or create a new PR that fixes the regressions, add a comment linking to the newly created issue or PR, and then add the perf-regression-triaged label to this PR.

@rustbot label: +perf-regression
cc @rust-lang/wg-compiler-performance

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	0.4%	[0.4%, 0.4%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.4%	[-0.6%, -0.2%]	9
Improvements ✅ (secondary)	-0.3%	[-0.3%, -0.3%]	1
All ❌✅ (primary)	-0.4%	[-0.6%, 0.4%]	10

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	2.3%	[0.5%, 3.1%]	4
Regressions ❌ (secondary)	2.8%	[1.2%, 4.3%]	46
Improvements ✅ (primary)	-0.8%	[-0.8%, -0.8%]	2
Improvements ✅ (secondary)	-2.3%	[-2.3%, -2.3%]	1
All ❌✅ (primary)	1.3%	[-0.8%, 3.1%]	6

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	0.7%	[0.7%, 0.7%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.4%	[-0.4%, -0.4%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.1%	[-0.4%, 0.7%]	2

nnethercote · 2023-03-21T05:10:51Z

Instruction counts are fine here. max-rss is a bit worrying, but it might be noise. @Zoxc, would this have affected memory consumption?

@rustbot label: +perf-regression-triaged

Zoxc · 2023-03-21T05:21:38Z

At worst it would increase stack usage a bit, but it shouldn't affect full at all so it's probably noise.

rustbot assigned cjgillot Feb 27, 2023

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Mar 1, 2023

This comment has been minimized.

Sign in to view

rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Mar 1, 2023

cjgillot reviewed Mar 7, 2023

View reviewed changes

cjgillot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 7, 2023

Zoxc force-pushed the node-backtrace branch from 5970f1a to 84ec26c Compare March 9, 2023 07:38

Zoxc added 3 commits March 12, 2023 09:55

Optimize dep node backtrace and ignore fatal errors

8dd0f20

Remove graph parameter from try_force_from_dep_node

867de8b

Use Debug for formatting the dep nodes

f48ff4a

Zoxc force-pushed the node-backtrace branch from 84ec26c to f48ff4a Compare March 12, 2023 09:03

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Mar 18, 2023

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Mar 18, 2023

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 18, 2023

bors added the merged-by-bors This PR was explicitly merged by bors. label Mar 20, 2023

bors merged commit 13b7aa4 into rust-lang:master Mar 20, 2023

rustbot added this to the 1.70.0 milestone Mar 20, 2023

Zoxc deleted the node-backtrace branch March 20, 2023 15:04

bors mentioned this pull request Mar 20, 2023

Only use the new DepNode hashmap for anonymous nodes. #109050

Open

rustbot added the perf-regression Performance regression. label Mar 20, 2023

rustbot added the perf-regression-triaged The performance regression has been triaged. label Mar 21, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Optimize dep node backtrace and ignore fatal errors #108524

Optimize dep node backtrace and ignore fatal errors #108524

Zoxc commented Feb 27, 2023

Zoxc commented Feb 27, 2023

compiler-errors commented Mar 1, 2023

This comment has been minimized.

bors commented Mar 1, 2023

bors commented Mar 1, 2023

bors commented Mar 1, 2023

This comment has been minimized.

rust-timer commented Mar 1, 2023

cjgillot Mar 7, 2023

Zoxc Mar 9, 2023

bors commented Mar 12, 2023

cjgillot commented Mar 18, 2023

bors commented Mar 18, 2023

bors commented Mar 18, 2023

bors commented Mar 18, 2023

ehuss commented Mar 18, 2023

rust-log-analyzer commented Mar 18, 2023

bors commented Mar 20, 2023

bors commented Mar 20, 2023

rust-timer commented Mar 20, 2023

nnethercote commented Mar 21, 2023

Zoxc commented Mar 21, 2023

Optimize dep node backtrace and ignore fatal errors #108524

Optimize dep node backtrace and ignore fatal errors #108524

Conversation

Zoxc commented Feb 27, 2023

Zoxc commented Feb 27, 2023

compiler-errors commented Mar 1, 2023

This comment has been minimized.

bors commented Mar 1, 2023

bors commented Mar 1, 2023

bors commented Mar 1, 2023

This comment has been minimized.

rust-timer commented Mar 1, 2023

Overall result: ✅ improvements - no action needed

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bors commented Mar 12, 2023

cjgillot commented Mar 18, 2023

bors commented Mar 18, 2023

bors commented Mar 18, 2023

bors commented Mar 18, 2023

ehuss commented Mar 18, 2023

rust-log-analyzer commented Mar 18, 2023

bors commented Mar 20, 2023

bors commented Mar 20, 2023

rust-timer commented Mar 20, 2023

Overall result: ❌✅ regressions and improvements - ACTION NEEDED

nnethercote commented Mar 21, 2023

Zoxc commented Mar 21, 2023