ICE: already have hash for FileMap(DefId {… #42101

SimonSapin · 2017-05-19T11:26:41Z

In Servo with rustc 1.19.0-nightly (0ed1ec9 2017-05-18), with incremental compilation enabled. (To reproduce, copy servobuild.example to .servobuild and change incremental = false to incremental = true.)

   Compiling gfx_traits v0.0.1 (file:///home/simon/servo1/components/gfx_traits)
error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: run with `RUST_BACKTRACE=1` for a backtrace

thread 'rustc' panicked at 'already have hash for FileMap(DefId { krate: CrateNum(14), node: DefIndex(0) => range/92375a1ff3a3ef6721ea553f45171916 }, "/home/simon/servo1/ports/servo/<proc-macro source code>")', /checkout/src/librustc_incremental/persist/hash.rs:254
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
stack backtrace:
   0: std::sys::imp::backtrace::tracing::imp::unwind_backtrace
             at /checkout/src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
   1: std::sys_common::backtrace::_print
             at /checkout/src/libstd/sys_common/backtrace.rs:71
   2: std::panicking::default_hook::{{closure}}
             at /checkout/src/libstd/sys_common/backtrace.rs:60
             at /checkout/src/libstd/panicking.rs:355
   3: std::panicking::default_hook
             at /checkout/src/libstd/panicking.rs:365
   4: std::panicking::rust_panic_with_hook
             at /checkout/src/libstd/panicking.rs:549
   5: std::panicking::begin_panic
             at /checkout/src/libstd/panicking.rs:511
   6: std::panicking::begin_panic_fmt
             at /checkout/src/libstd/panicking.rs:495
   7: rustc_incremental::persist::hash::HashContext::load_data
   8: rustc_incremental::persist::hash::HashContext::hash
   9: rustc_incremental::persist::save::save_dep_graph
  10: rustc_driver::driver::phase_4_translate_to_llvm
  11: rustc_driver::driver::compile_input::{{closure}}
  12: rustc_driver::driver::phase_3_run_analysis_passes::{{closure}}
  13: rustc_driver::driver::phase_3_run_analysis_passes
  14: rustc_driver::driver::compile_input
  15: rustc_driver::run_compiler

error: Could not compile `gfx_traits`.

The text was updated successfully, but these errors were encountered:

(Do not upgrade yet because of rust-lang/rust#42101)

CraZySacX · 2017-05-19T20:30:29Z

Same thing compiling clippy

   Compiling clippy v0.0.134 (https://github.com/Manishearth/rust-clippy.git#dd905150)                                                                                                                                                                                                      
error: internal compiler error: unexpected panic                                                                                                                                                                                                                                            
                                                                                                                                                                                                                                                                                            
note: the compiler unexpectedly panicked. this is a bug.                                                                                                                                                                                                                                    
                                                                                                                                                                                                                                                                                            
note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports                             

note: run with `RUST_BACKTRACE=1` for a backtrace                      

thread 'rustc' panicked at 'already have hash for FileMap(DefId { krate: CrateNum(62), node: DefIndex(0) => cargo_metadata/6da37b3662e680ee62d183ed0639e271 }, "/home/jozias/<proc-macro source code>")', /checkout/src/librustc_incremental/persist/hash.rs:254                            
stack backtrace:                                                       
   0:     0x7fe8f1fb8953 - std::sys::imp::backtrace::tracing::imp::unwind_backtrace::h5301f9f0d14e6b67                                        
                               at /checkout/src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:49                                                 
   1:     0x7fe8f1fb391a - std::sys_common::backtrace::_print::h65c3509ebfb6cbe2                                                              
                               at /checkout/src/libstd/sys_common/backtrace.rs:71                                                             
   2:     0x7fe8f1fc771a - std::panicking::default_hook::{{closure}}::hf5d8886edb67784b                                                       
                               at /checkout/src/libstd/sys_common/backtrace.rs:60                                                             
                               at /checkout/src/libstd/panicking.rs:355
   3:     0x7fe8f1fc72bb - std::panicking::default_hook::h755dd67224d34962                                                                    
                               at /checkout/src/libstd/panicking.rs:365
   4:     0x7fe8f1fc7b2b - std::panicking::rust_panic_with_hook::haea0fb9c3665ef73                                                            
                               at /checkout/src/libstd/panicking.rs:549
   5:     0x7fe8f1fc79b4 - std::panicking::begin_panic::hda25ba764f58caf0                                                                     
                               at /checkout/src/libstd/panicking.rs:511
   6:     0x7fe8f1fc7939 - std::panicking::begin_panic_fmt::h6b075a9283828bd5                                                                 
                               at /checkout/src/libstd/panicking.rs:495
   7:     0x7fe8ef51f9f4 - rustc_incremental::persist::hash::HashContext::load_data::hae1f3fc2ccf17866                                        
   8:     0x7fe8ef51dada - rustc_incremental::persist::hash::HashContext::hash::h1232829f2d1e3e5d                                             
   9:     0x7fe8ef531d49 - rustc_incremental::persist::save::save_dep_graph::h8e9814f873d08e7f                                                
  10:     0x7fe8f238b85a - rustc_driver::driver::phase_4_translate_to_llvm::hb14a1476b0c4dc8c                                                 
  11:     0x7fe8f235677b - rustc_driver::driver::compile_input::{{closure}}::h0687853991e0d748                                                
  12:     0x7fe8f2389afc - rustc_driver::driver::phase_3_run_analysis_passes::{{closure}}::h84f0c6a235da6c1e                                  
  13:     0x7fe8f236a9b0 - rustc_driver::driver::phase_3_run_analysis_passes::h6f3ddf052db34356                                               
  14:     0x7fe8f235431c - rustc_driver::driver::compile_input::he6e8d6b966d9c6e4                                                             
  15:     0x7fe8f239b330 - rustc_driver::run_compiler::h3831281dea1e4e2e                                                                      
  16:     0x7fe8f22a522b - std::sys_common::backtrace::__rust_begin_short_backtrace::ha0f945ccf2b93dc0                                        
  17:     0x7fe8f1fd0e7a - __rust_maybe_catch_panic                    
                               at /checkout/src/libpanic_unwind/lib.rs:98                                                                     
  18:     0x7fe8f22d8510 - <F as alloc::boxed::FnBox<A>>::call_box::hc6e0e3dbbddf221c                                                         
  19:     0x7fe8f1fc6525 - std::sys::imp::thread::Thread::new::thread_start::h74d0491aca13f8bd                                                
                               at /checkout/src/liballoc/boxed.rs:658  
                               at /checkout/src/libstd/sys_common/thread.rs:21                                                                
                               at /checkout/src/libstd/sys/unix/thread.rs:84                                                                  
  20:     0x7fe8ed38f296 - start_thread                                
  21:     0x7fe8f1c7f25e - __clone                                     
  22:                0x0 - <unknown>                                   

error: failed to compile `clippy v0.0.134 (https://github.com/Manishearth/rust-clippy.git#dd905150)`, intermediate artifacts can be found at `/tmp/cargo-install.f6nCYMDsVbC0`

Fix warning in a future compiler version. (Do not upgrade yet because of rust-lang/rust#42101)  --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/16950)

king6cong · 2017-05-21T07:59:23Z

same thing happened

<proc-macro source code>")', src/librustc_incremental/persist/hash.rs:254
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
stack backtrace:
   0: std::sys::imp::backtrace::tracing::imp::unwind_backtrace
   1: std::panicking::default_hook::{{closure}}
   2: std::panicking::default_hook
   3: std::panicking::rust_panic_with_hook
   4: std::panicking::begin_panic
   5: std::panicking::begin_panic_fmt
   6: rustc_incremental::persist::hash::HashContext::load_data
   7: rustc_incremental::persist::hash::HashContext::hash
   8: rustc_incremental::persist::save::save_dep_graph
   9: rustc_driver::driver::phase_4_translate_to_llvm
  10: rustc_driver::driver::compile_input::{{closure}}
  11: rustc_driver::driver::phase_3_run_analysis_passes::{{closure}}
  12: rustc_driver::driver::phase_3_run_analysis_passes
  13: rustc_driver::driver::compile_input
  14: rustc_driver::run_compiler

rustc 1.19.0-nightly (01951a6 2017-05-20)
cargo 0.20.0-nightly (397359840 2017-05-18)

…servo:warn); r=nox (Do not upgrade yet because of rust-lang/rust#42101) Source-Repo: https://github.com/servo/servo Source-Revision: dbd4adf3b266fb8d02cb717bc255c04f0fe41c05 --HG-- extra : subtree_source : https%3A//hg.mozilla.org/projects/converted-servo-linear extra : subtree_revision : b024b9d4b46a80c7c8d69c4473cae0617613d68b

michaelwoerister · 2017-05-22T07:30:39Z

Thanks for the report. I'm looking into it.

kennytm · 2017-05-22T09:26:07Z

@donhcd @king6cong @CraZySacX could you post the command line and source code which triggers the ICE?

…servo:warn); r=nox (Do not upgrade yet because of rust-lang/rust#42101) Source-Repo: https://github.com/servo/servo Source-Revision: dbd4adf3b266fb8d02cb717bc255c04f0fe41c05

michaelwoerister · 2017-05-23T10:00:05Z

So the problem here is that the code assumes that the file name uniquely identifies a FileMap within a crate. However, each instantiation of a proc-macro allocates a new FileMap it seems and all of them are named <proc-macro source code>.

There are several ways of solving this:

Don't track FileMaps after all. We would have to rely on the fact that everything having access to a Span value must have a transitive dependency on a piece of HIR, the hash of which contains the expanded version of the span (including a possibly remapped file name).
Disambiguate FileMaps with equal name via an additional counter, like we do for DefPath components. However, since all of them are in one global space, this seems like a bad idea: Introducing a new proc-macro invocation would invalidate all proc-macro FileMaps coming after it in the CodeMap. This option is pretty much out of the question.
Special-case synthetic FileMaps, that is, FileMaps that don't correspond to an actual source file, and add a hash of their contents to the DepNode identifier. This would have the effect that such FileMaps would appear to be deleted and a different one being added, instead of being changed in-place. This should work as long as HIR-hashing still incorporates expanded span information. Otherwise we would miss something, if two proc-macro FileMaps are collapsed into one and then split up again.
Disable file-related things in debuginfo and panic messages if they would originate from a synthetic FileMap. This seems a bit too sketchy for me.

I've implemented a proof of concept of solution (3) and it makes the ICE go away in the case of Servo. But option (1) seems rather attractive, since it would allow for removing quite a bit of infrastructure that might not serve an actual purpose after all. We should try to determine whether including expanded spans into the HIR hash is a sufficient correctness condition (as in (1)) and not just a necessary one (as in (3)).

cc @nikomatsakis

michaelwoerister · 2017-05-23T15:04:16Z

I was about to make a PR implementing approach (1) when I found a problem with it: In a cross-crate scenario with crate A depending on crate B, when B is compiled without debuginfo and A is compiled with debuginfo, then functions inlined from B into A can end up with invalid debuginfo. The reason for this is that B, being compiled without debuginfo does not feed spans into the item hashes, so we can change those spans without it being detected by the downstream crate.

One way to solve this would be to always include spans into metadata hashes. But that would make them more sensitive than they need to be. But maybe that's still the way to go.

@nikomatsakis

…x-1, r=nikomatsakis incr.comp.: Track expanded spans instead of FileMaps. This PR removes explicit tracking of FileMaps in response to rust-lang#42101. The reasoning behind being able to just *not* track access to FileMaps is similar to why we don't track access to the `DefId->DefPath` map: 1. One can only get ahold of a `Span` value by accessing the HIR (for local things) or a `metadata::schema::Entry` (for things from external crates). 2. For both of these things we compute a hash that incorporates the *expanded spans*, that is, what we hash is in the (FileMap independent) format `filename:line:col`. 3. Consequently, everything that emits a span should already be tracked via its dependency to something that has the span included in its hash and changes would be detected via that hash. One caveat here is that we have to be conservative when exporting things in metadata. A crate can be built without debuginfo and would thus by default not incorporate most spans into the metadata hashes. However, a downstream crate can make an inline copy of things in the upstream crate and span changes in the upstream crate would then go undetected, even if the downstream uses them (e.g. by emitting debuginfo for an inlined function). For this reason, we always incorporate spans into metadata hashes for now (there might be more efficient ways to handle this safely when red-green tracking is implemented). r? @nikomatsakis

…servo:warn); r=nox (Do not upgrade yet because of rust-lang/rust#42101) Source-Repo: https://github.com/servo/servo Source-Revision: dbd4adf3b266fb8d02cb717bc255c04f0fe41c05

@nikomatsakis

…x-1, r=nikomatsakis incr.comp.: Track expanded spans instead of FileMaps. This PR removes explicit tracking of FileMaps in response to rust-lang#42101. The reasoning behind being able to just *not* track access to FileMaps is similar to why we don't track access to the `DefId->DefPath` map: 1. One can only get ahold of a `Span` value by accessing the HIR (for local things) or a `metadata::schema::Entry` (for things from external crates). 2. For both of these things we compute a hash that incorporates the *expanded spans*, that is, what we hash is in the (FileMap independent) format `filename:line:col`. 3. Consequently, everything that emits a span should already be tracked via its dependency to something that has the span included in its hash and changes would be detected via that hash. One caveat here is that we have to be conservative when exporting things in metadata. A crate can be built without debuginfo and would thus by default not incorporate most spans into the metadata hashes. However, a downstream crate can make an inline copy of things in the upstream crate and span changes in the upstream crate would then go undetected, even if the downstream uses them (e.g. by emitting debuginfo for an inlined function). For this reason, we always incorporate spans into metadata hashes for now (there might be more efficient ways to handle this safely when red-green tracking is implemented). r? @nikomatsakis

@nikomatsakis

…x-1, r=nikomatsakis incr.comp.: Track expanded spans instead of FileMaps. This PR removes explicit tracking of FileMaps in response to rust-lang#42101. The reasoning behind being able to just *not* track access to FileMaps is similar to why we don't track access to the `DefId->DefPath` map: 1. One can only get ahold of a `Span` value by accessing the HIR (for local things) or a `metadata::schema::Entry` (for things from external crates). 2. For both of these things we compute a hash that incorporates the *expanded spans*, that is, what we hash is in the (FileMap independent) format `filename:line:col`. 3. Consequently, everything that emits a span should already be tracked via its dependency to something that has the span included in its hash and changes would be detected via that hash. One caveat here is that we have to be conservative when exporting things in metadata. A crate can be built without debuginfo and would thus by default not incorporate most spans into the metadata hashes. However, a downstream crate can make an inline copy of things in the upstream crate and span changes in the upstream crate would then go undetected, even if the downstream uses them (e.g. by emitting debuginfo for an inlined function). For this reason, we always incorporate spans into metadata hashes for now (there might be more efficient ways to handle this safely when red-green tracking is implemented). r? @nikomatsakis

@nikomatsakis

…x-1, r=nikomatsakis incr.comp.: Track expanded spans instead of FileMaps. This PR removes explicit tracking of FileMaps in response to rust-lang#42101. The reasoning behind being able to just *not* track access to FileMaps is similar to why we don't track access to the `DefId->DefPath` map: 1. One can only get ahold of a `Span` value by accessing the HIR (for local things) or a `metadata::schema::Entry` (for things from external crates). 2. For both of these things we compute a hash that incorporates the *expanded spans*, that is, what we hash is in the (FileMap independent) format `filename:line:col`. 3. Consequently, everything that emits a span should already be tracked via its dependency to something that has the span included in its hash and changes would be detected via that hash. One caveat here is that we have to be conservative when exporting things in metadata. A crate can be built without debuginfo and would thus by default not incorporate most spans into the metadata hashes. However, a downstream crate can make an inline copy of things in the upstream crate and span changes in the upstream crate would then go undetected, even if the downstream uses them (e.g. by emitting debuginfo for an inlined function). For this reason, we always incorporate spans into metadata hashes for now (there might be more efficient ways to handle this safely when red-green tracking is implemented). r? @nikomatsakis

@nikomatsakis

…matsakis incr.comp.: Track expanded spans instead of FileMaps. This PR removes explicit tracking of FileMaps in response to #42101. The reasoning behind being able to just *not* track access to FileMaps is similar to why we don't track access to the `DefId->DefPath` map: 1. One can only get ahold of a `Span` value by accessing the HIR (for local things) or a `metadata::schema::Entry` (for things from external crates). 2. For both of these things we compute a hash that incorporates the *expanded spans*, that is, what we hash is in the (FileMap independent) format `filename:line:col`. 3. Consequently, everything that emits a span should already be tracked via its dependency to something that has the span included in its hash and changes would be detected via that hash. One caveat here is that we have to be conservative when exporting things in metadata. A crate can be built without debuginfo and would thus by default not incorporate most spans into the metadata hashes. However, a downstream crate can make an inline copy of things in the upstream crate and span changes in the upstream crate would then go undetected, even if the downstream uses them (e.g. by emitting debuginfo for an inlined function). For this reason, we always incorporate spans into metadata hashes for now (there might be more efficient ways to handle this safely when red-green tracking is implemented). r? @nikomatsakis

michaelwoerister · 2017-07-07T14:41:56Z

Fixed by #42175, closing.

…servo:warn); r=nox (Do not upgrade yet because of rust-lang/rust#42101) Source-Repo: https://github.com/servo/servo Source-Revision: dbd4adf3b266fb8d02cb717bc255c04f0fe41c05 UltraBlame original commit: f7050ffdfddb21a4b839f81b05bb356481fc4f7c

SimonSapin added a commit to servo/servo that referenced this issue May 19, 2017

Fix warning in a future compiler version.

902d979

(Do not upgrade yet because of rust-lang/rust#42101)

SimonSapin mentioned this issue May 19, 2017

Fix warning in a future compiler version. servo/servo#16950

Merged

Mark-Simulacrum added the I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ label May 20, 2017

king6cong mentioned this issue May 21, 2017

incr.comp.: Hash more pieces of crate metadata to detect changes there. #41709

Merged

2 tasks

michaelwoerister self-assigned this May 23, 2017

michaelwoerister added the A-incr-comp Area: Incremental compilation label May 23, 2017

michaelwoerister added a commit to michaelwoerister/rust that referenced this issue May 23, 2017

solution (3) rust-lang#42101

7343064

michaelwoerister mentioned this issue May 23, 2017

incr.comp.: Track expanded spans instead of FileMaps. #42175

Merged

kennytm mentioned this issue May 26, 2017

Compiler panics with CARGO_INCREMENTAL=1 #42237

Closed

Mark-Simulacrum mentioned this issue May 28, 2017

ICE: simple Serde case #42279

Closed

michaelwoerister closed this as completed Jul 7, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ICE: already have hash for FileMap(DefId {… #42101

ICE: already have hash for FileMap(DefId {… #42101

SimonSapin commented May 19, 2017

CraZySacX commented May 19, 2017

king6cong commented May 21, 2017 •

edited

Loading

michaelwoerister commented May 22, 2017

kennytm commented May 22, 2017

michaelwoerister commented May 23, 2017

michaelwoerister commented May 23, 2017

michaelwoerister commented Jul 7, 2017

ICE: already have hash for FileMap(DefId {… #42101

ICE: already have hash for FileMap(DefId {… #42101

Comments

SimonSapin commented May 19, 2017

CraZySacX commented May 19, 2017

king6cong commented May 21, 2017 • edited Loading

michaelwoerister commented May 22, 2017

kennytm commented May 22, 2017

michaelwoerister commented May 23, 2017

michaelwoerister commented May 23, 2017

michaelwoerister commented Jul 7, 2017

king6cong commented May 21, 2017 •

edited

Loading