Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"called Result::unwrap() on an Err value: Utf8Error { valid_up_to: 2 }" ICE in stable #33778

Closed
netvl opened this issue May 21, 2016 · 22 comments
Assignees
Labels
I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ P-high High priority regression-from-stable-to-nightly Performance or correctness regression from stable to nightly. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@netvl
Copy link
Contributor

netvl commented May 21, 2016

I suddenly got the following error when compiling my crate:

% RUST_BACKTRACE=1 rustup run stable cargo build --verbose
       Fresh num-traits v0.1.32
       Fresh byteorder v0.5.1
       Fresh odds v0.2.12
       Fresh nodrop v0.1.6
       Fresh arrayvec v0.3.15
   Compiling immeta v0.3.4 (file:///home/netvl/dev/lang/rust/projects/immeta)
     Running `rustc src/lib.rs --crate-name immeta --crate-type lib -g --out-dir /home/netvl/dev/lang/rust/projects/immeta/target/debug --emit=dep-info,link -L dependency=/home/netvl/dev/lang/rust/projects/immeta/target/debug -L dependency=/home/netvl/dev/lang/rust/projects/immeta/target/debug/deps --extern arrayvec=/home/netvl/dev/lang/rust/projects/immeta/target/debug/deps/libarrayvec-bcf06b3ef065493b.rlib --extern num_traits=/home/netvl/dev/lang/rust/projects/immeta/target/debug/deps/libnum_traits-a4000e09af4c8117.rlib --extern byteorder=/home/netvl/dev/lang/rust/projects/immeta/target/debug/deps/libbyteorder-4e8da61ace56de4d.rlib`
error: internal compiler error: unexpected panic
note: the compiler unexpectedly panicked. this is a bug.
note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports
thread 'rustc' panicked at 'called `Result::unwrap()` on an `Err` value: Utf8Error { valid_up_to: 2 }', ../src/libcore/result.rs:746
stack backtrace:
   1:     0x7fa66f6de330 - sys::backtrace::tracing::imp::write::h3675b4f0ca767761Xcv
   2:     0x7fa66f6e768b - panicking::default_handler::_$u7b$$u7b$closure$u7d$$u7d$::closure.44519
   3:     0x7fa66f6e71e3 - panicking::default_handler::h18faf4fbd296d909lSz
   4:     0x7fa66f6ab9ac - sys_common::unwind::begin_unwind_inner::hfb5d07d6e405c6bbg1t
   5:     0x7fa66f6ac448 - sys_common::unwind::begin_unwind_fmt::h8b491a76ae84af35m0t
   6:     0x7fa66f6dbd91 - rust_begin_unwind
   7:     0x7fa66f72e7af - panicking::panic_fmt::h98b8cbb286f5298alcM
   8:     0x7fa6674b731b - result::unwrap_failed::h14307315525592478485
   9:     0x7fa6674b728e - Doc<'doc>::as_str_slice::ha91bf39a19faeabdHVa
  10:     0x7fa66d35f45f - loader::Context<'a>::extract_one::h945c702fbde346e3Eem
  11:     0x7fa66d356acf - loader::Context<'a>::find_library_crate::h30971509c344ad3bD3l
  12:     0x7fa66d33bf16 - loader::Context<'a>::load_library_crate::h0ade06e110892b81WSl
  13:     0x7fa66d33ad99 - creader::CrateReader<'a>::resolve_crate::he1542772b02d0896E3j
  14:     0x7fa66d33c58d - creader::CrateReader<'a>::resolve_crate_deps::_$u7b$$u7b$closure$u7d$$u7d$::closure.38792
  15:     0x7fa66d33c33c - collections::hash::map::HashMap<K, V, S>.FromIterator<(K, V)>::from_iter::h12379182329167058746
  16:     0x7fa66d3384dd - creader::CrateReader<'a>::register_crate::h62645edecf14372cTZj
  17:     0x7fa66d33b6e2 - creader::CrateReader<'a>::resolve_crate::he1542772b02d0896E3j
  18:     0x7fa66d33586b - creader::LocalCrateReader<'a, 'b>.Visitor<'hir>::visit_item::hd760834db1c6f482eDj
  19:     0x7fa66d347d7e - creader::LocalCrateReader<'a, 'b>::read_crates::hb36235539ae0e4e04vk
  20:     0x7fa66fc027c7 - driver::phase_3_run_analysis_passes::h10920926650053012691
  21:     0x7fa66fbd8a15 - driver::compile_input::h7ae6a86e23de0774Hca
  22:     0x7fa66fbc7b47 - run_compiler::hb0408bcf47642fe6mPc
  23:     0x7fa66fbc52c1 - sys_common::unwind::try::try_fn::h7614151098073783683
  24:     0x7fa66f6dbd1b - __rust_try
  25:     0x7fa66f6d41fd - sys_common::unwind::inner_try::hadd81c754a64f07ciYt
  26:     0x7fa66fbc5b10 - boxed::F.FnBox<A>::call_box::h18261575856772421581
  27:     0x7fa66f6e5c59 - sys::thread::Thread::new::thread_start::h9bc812305b5e01feFPy
  28:     0x7fa6689af473 - start_thread
  29:     0x7fa66f35d69c - clone
  30:                0x0 - <unknown>

Could not compile `immeta`.

Caused by:
  Process didn't exit successfully: `rustc src/lib.rs --crate-name immeta --crate-type lib -g --out-dir /home/netvl/dev/lang/rust/projects/immeta/target/debug --emit=dep-info,link -L dependency=/home/netvl/dev/lang/rust/projects/immeta/target/debug -L dependency=/home/netvl/dev/lang/rust/projects/immeta/target/debug/deps --extern arrayvec=/home/netvl/dev/lang/rust/projects/immeta/target/debug/deps/libarrayvec-bcf06b3ef065493b.rlib --extern num_traits=/home/netvl/dev/lang/rust/projects/immeta/target/debug/deps/libnum_traits-a4000e09af4c8117.rlib --extern byteorder=/home/netvl/dev/lang/rust/projects/immeta/target/debug/deps/libbyteorder-4e8da61ace56de4d.rlib` (exit code: 101)

Here are the crate sources: immeta.zip

Apparently there is no such error on travis, so I think this may be caused by some weird interaction of rustc with rustup, but I don't know for sure.

rustc version:

% rustup run stable rustc -Vv
rustc 1.8.0 (db2939409 2016-04-11)
binary: rustc
commit-hash: db2939409db26ab4904372c82492cd3488e4c44e
commit-date: 2016-04-11
host: x86_64-unknown-linux-gnu
release: 1.8.0

cargo version:

% rustup run stable cargo --version
cargo 0.9.0-nightly (8fc3fd8 2016-02-29)

rustup version:

% rustup --version
rustup 0.1.12 (c6e430a 2016-05-12)
@Diggsey
Copy link
Contributor

Diggsey commented May 21, 2016

@netvl Try doing a cargo clean, and then rebuilding. It looks like it might be trying to read the metadata from a crate compiled with a previous version of the compiler.

@mattico
Copy link
Contributor

mattico commented May 22, 2016

@Diggsey Can (probably) confirm. I got this same error when switching from nightly-msvc to stable-gnu after aborting a build.

@mkollaro
Copy link

Same here. I'm not sure what caused it, but I was switching around between nightly and stable before. Works in nightly, fails in stable. Once it started happening, it was happening every time. Running cargo clean fixed it.

RUST_BACKTRACE=1 cargo build --verbose
       Fresh log v0.3.6
       Fresh libc v0.2.9
       Fresh cfg-if v0.1.0
       Fresh semver v0.1.20
       Fresh getopts v0.2.14
       Fresh bitflags v0.4.0
       Fresh rustc_version v0.1.7
       Fresh nix v0.5.1-pre (https://github.com/mkollaro/nix#22ae34a7)
   Compiling librdb v0.1.0 (file:///mnt/data/src/rdb)
     Running `rustc src/lib.rs --crate-name librdb --crate-type lib -g --cfg feature=\"default\" --out-dir /mnt/data/src/rdb/target/debug --emit=dep-info,link -L dependency=/mnt/data/src/rdb/target/debug -L dependency=/mnt/data/src/rdb/target/debug/deps --extern libc=/mnt/data/src/rdb/target/debug/deps/liblibc-d2268fc21ed63f2c.rlib --extern log=/mnt/data/src/rdb/target/debug/deps/liblog-30a8a27ec161f1be.rlib --extern nix=/mnt/data/src/rdb/target/debug/deps/libnix-4281f0d4c5a4bab6.rlib --extern getopts=/mnt/data/src/rdb/target/debug/deps/libgetopts-852fe11dd444cf81.rlib -L native=/mnt/data/src/rdb/thirdparty/libdwarf/libdwarf -l static=dwarf -l elf -l z`
error: internal compiler error: unexpected panic
note: the compiler unexpectedly panicked. this is a bug.
note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports
thread 'rustc' panicked at 'called `Result::unwrap()` on an `Err` value: Utf8Error { valid_up_to: 0 }', ../src/libcore/result.rs:746
stack backtrace:
   1:     0x7f96765bc330 - sys::backtrace::tracing::imp::write::h3675b4f0ca767761Xcv
   2:     0x7f96765c568b - panicking::default_handler::_$u7b$$u7b$closure$u7d$$u7d$::closure.44519
   3:     0x7f96765c51e3 - panicking::default_handler::h18faf4fbd296d909lSz
   4:     0x7f96765899ac - sys_common::unwind::begin_unwind_inner::hfb5d07d6e405c6bbg1t
   5:     0x7f967658a448 - sys_common::unwind::begin_unwind_fmt::h8b491a76ae84af35m0t
   6:     0x7f96765b9d91 - rust_begin_unwind
   7:     0x7f967660c7af - panicking::panic_fmt::h98b8cbb286f5298alcM
   8:     0x7f966e36e31b - result::unwrap_failed::h14307315525592478485
   9:     0x7f966e36e28e - Doc<'doc>::as_str_slice::ha91bf39a19faeabdHVa
  10:     0x7f967421945f - loader::Context<'a>::extract_one::h945c702fbde346e3Eem
  11:     0x7f9674210acf - loader::Context<'a>::find_library_crate::h30971509c344ad3bD3l
  12:     0x7f96741f5f16 - loader::Context<'a>::load_library_crate::h0ade06e110892b81WSl
  13:     0x7f96741f4d99 - creader::CrateReader<'a>::resolve_crate::he1542772b02d0896E3j
  14:     0x7f96741f658d - creader::CrateReader<'a>::resolve_crate_deps::_$u7b$$u7b$closure$u7d$$u7d$::closure.38792
  15:     0x7f96741f633c - collections::hash::map::HashMap<K, V, S>.FromIterator<(K, V)>::from_iter::h12379182329167058746
  16:     0x7f96741f24dd - creader::CrateReader<'a>::register_crate::h62645edecf14372cTZj
  17:     0x7f96741f745c - creader::CrateReader<'a>::read_extension_crate::hb5e653870a478718Zak
  18:     0x7f96741f96b5 - creader::CrateReader<'a>::read_exported_macros::h2322639de24bfa4cUek
  19:     0x7f967421ed0d - macro_import::MacroLoader<'a>.Visitor<'v>::visit_item::hf3405d18ebfdd3eakYm
  20:     0x7f967421d88a - macro_import::read_macro_defs::h345dbb1da4180a9foXm
  21:     0x7f9676aced89 - driver::phase_2_configure_and_expand::h0bbd128ff132a80elBa
  22:     0x7f9676ab4b21 - driver::compile_input::h7ae6a86e23de0774Hca
  23:     0x7f9676aa5b47 - run_compiler::hb0408bcf47642fe6mPc
  24:     0x7f9676aa32c1 - sys_common::unwind::try::try_fn::h7614151098073783683
  25:     0x7f96765b9d1b - __rust_try
  26:     0x7f96765b21fd - sys_common::unwind::inner_try::hadd81c754a64f07ciYt
  27:     0x7f9676aa3b10 - boxed::F.FnBox<A>::call_box::h18261575856772421581
  28:     0x7f96765c3c59 - sys::thread::Thread::new::thread_start::h9bc812305b5e01feFPy
  29:     0x7f966f869181 - start_thread
  30:     0x7f967622a47c - __clone
  31:                0x0 - <unknown>

Could not compile `librdb`.

Caused by:
  Process didn't exit successfully: `rustc src/lib.rs --crate-name librdb --crate-type lib -g --cfg feature="default" --out-dir /mnt/data/src/rdb/target/debug --emit=dep-info,link -L dependency=/mnt/data/src/rdb/target/debug -L dependency=/mnt/data/src/rdb/target/debug/deps --extern libc=/mnt/data/src/rdb/target/debug/deps/liblibc-d2268fc21ed63f2c.rlib --extern log=/mnt/data/src/rdb/target/debug/deps/liblog-30a8a27ec161f1be.rlib --extern nix=/mnt/data/src/rdb/target/debug/deps/libnix-4281f0d4c5a4bab6.rlib --extern getopts=/mnt/data/src/rdb/target/debug/deps/libgetopts-852fe11dd444cf81.rlib -L native=/mnt/data/src/rdb/thirdparty/libdwarf/libdwarf -l static=dwarf -l elf -l z` (exit code: 101)

@steveklabnik steveklabnik added the I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ label May 24, 2016
@azerupi
Copy link
Contributor

azerupi commented May 27, 2016

I got the same error running cargo test (stable Rust 1.9) after running the tests with nightly using rustup run nightly cargo test.

It's probably unrelated but just before it panicked I set up the new error format with RUST_NEW_ERROR_FORMAT="true" in my .zshrc file. That's why I was building with nightly.

I have a file in tests/ looking like this

extern crate my_crate;

#[test]
...

And the panic did not occur when I commented out extern crate my_crate;. It also didn't occur when building normally with cargo build (on stable).

When I changed the code and it compiled with nightly I was able to compile with stable again.

error: internal compiler error: unexpected panic
note: the compiler unexpectedly panicked. this is a bug.
note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports
note: run with `RUST_BACKTRACE=1` for a backtrace
thread 'rustc' panicked at 'called `Result::unwrap()` on an `Err` value: Utf8Error { valid_up_to: 0 }', ../src/libcore/result.rs:746
stack backtrace:
   1:     0x7fada89d8570 - std::sys::backtrace::tracing::imp::write::h4c73fcd3363076f5
   2:     0x7fada89e5c6b - std::panicking::default_hook::_$u7b$$u7b$closure$u7d$$u7d$::h0422dbb3077e6747
   3:     0x7fada89e580c - std::panicking::default_hook::haac48fa641db8fa2
   4:     0x7fada89aa79f - std::sys_common::unwind::begin_unwind_inner::h39d40f52add53ef7
   5:     0x7fada89ac888 - std::sys_common::unwind::begin_unwind_fmt::h64c0ff793199cc1b
   6:     0x7fada89d5d81 - rust_begin_unwind
   7:     0x7fada8a2f85f - core::panicking::panic_fmt::h73bf9d7e8e891a73
   8:     0x7fad9ff89a0b - core::result::unwrap_failed::h82c81d60621f70bd
   9:     0x7fad9ff8997e - rbml::Doc::as_str_slice::hd88d17b04c3f2cca
  10:     0x7fada67e0a62 - rustc_metadata::decoder::maybe_get_crate_hash::h1c0939ce4d9ba236
  11:     0x7fada681401b - rustc_metadata::loader::Context::extract_one::h56453781fb65bffa
  12:     0x7fada680aada - rustc_metadata::loader::Context::find_library_crate::h30ea71d32623d713
  13:     0x7fada67f1f96 - rustc_metadata::loader::Context::load_library_crate::h7787c1fa9559d604
  14:     0x7fada67f0e1f - rustc_metadata::creader::CrateReader::resolve_crate::h98decfaedbcfe12a
  15:     0x7fada67eab41 - _<creader..LocalCrateReader<'a, 'b> as rustc..hir..intravisit..Visitor<'hir>>::visit_item::h2f4b84ef1536be92
  16:     0x7fada67ff4be - rustc_metadata::creader::LocalCrateReader::read_crates::h7c2989bf6ef8974b
  17:     0x7fada8f21c86 - rustc_driver::driver::phase_3_run_analysis_passes::h83da042ec4b8ea10
  18:     0x7fada8ef6f9f - rustc_driver::driver::compile_input::h6491aaddd9e61258
  19:     0x7fada8edd4e4 - rustc_driver::run_compiler::h80b2ba5e4d787c5f
  20:     0x7fada8eda941 - std::sys_common::unwind::try::try_fn::h34e27823ddd1d5e9
  21:     0x7fada89d5d0b - __rust_try
  22:     0x7fada89d5c9d - std::sys_common::unwind::inner_try::h9eebd8dc83f388a6
  23:     0x7fada8edb18a - _<F as std..boxed..FnBox<A>>::call_box::h3d5d78986dfac5b2
  24:     0x7fada89e3e04 - std::sys::thread::Thread::new::thread_start::h471ad90789353b5b
  25:     0x7fada13ca473 - start_thread
  26:     0x7fada864269c - clone
  27:                0x0 - <unknown>

@alexcrichton
Copy link
Member

cc @rust-lang/compiler

This seems to be getting lots of hits, is something that's fixed on nightly? Perhaps something that can be backported to beta?

@alexcrichton alexcrichton added I-nominated T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels May 31, 2016
@nikomatsakis
Copy link
Contributor

@alexcrichton I'm not sure what's causing this -- you think it's a rustc bug?

@alexcrichton
Copy link
Member

@nikomatsakis yes:

$ cargo new foo
$ cd foo
$ echo 'libc = "0.2"' >> Cargo.toml
$ rustup run nightly cargo build
...
$ rustup run stable cargo build
   Compiling libc v0.2.11
error: internal compiler error: unexpected panic
note: the compiler unexpectedly panicked. this is a bug.
note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports
thread 'rustc' panicked at 'called `Result::unwrap()` on an `Err` value: Utf8Error { valid_up_to: 1 }', ../src/libcore/result.rs:746
note: Run with `RUST_BACKTRACE=1` for a backtrace.

error: Could not compile `libc`.

To learn more, run the command again with --verbose.

This may be fixed on both nightly and beta, as the only transition I can get to break is nightly to stable. I basically just want to confirm that we know what happened here and that it is indeed fixed and future-proofed.

@nikomatsakis
Copy link
Contributor

@alexcrichton hmm maybe I'm just being dense but I at least don't know what caused this and/or if it is fixed. =)

@nikomatsakis
Copy link
Contributor

I suspect the problem is due to the fact that I changed the crate hash from being a string to being a u64. It seems to me to be sort of a cargo bug -- like, why are we seeing stale metadata from a different version of the compiler? But I don't quite know how this system interacts. @alexcrichton let's chat. :)

@nikomatsakis
Copy link
Contributor

triage: P-high

@rust-highfive rust-highfive added the P-high High priority label Jun 2, 2016
@alexcrichton
Copy link
Member

The stale metadata may be sticking around because Cargo doesn't clean out old libraries, and when the compiler is specifically looking for the libc library transitively it'll find the old crate compiled with a newer version of Rust.

If it's the case that this is the crate hash, there's probably nothing we can do about this other than issuing a point release, and it doesn't seem worth it to do that...

@Diggsey
Copy link
Contributor

Diggsey commented Jun 2, 2016

Could rustc make sure to only use panic-free code up until the point where it's read the metadata version, and give a helpful error message instead of ICEing?
(I realise this would only fix future instances of this bug)

@nikomatsakis
Copy link
Contributor

@alexcrichton I could go back to putting a string in metadata instead of just writing a u64, I guess

@nikomatsakis
Copy link
Contributor

nikomatsakis commented Jun 3, 2016

I'll have to go poke at the code

@nikomatsakis
Copy link
Contributor

@alexcrichton hmm for some reason I'm not seeing that failure you suggested locally...

@plumenator
Copy link

This can also be reproduced using rustup.

$ rustup run nightly -- cargo build
# Suceeds
$ rustup run stable -- cargo build
# ICE

@tafia
Copy link
Contributor

tafia commented Jun 8, 2016

Same error, on stable only. A cargo clean fixes it.

@cholcombe973
Copy link

I'm hitting this also but a cargo clean doesn't fix it.

@brson brson added the regression-from-stable-to-nightly Performance or correctness regression from stable to nightly. label Jun 23, 2016
@brson
Copy link
Contributor

brson commented Jun 23, 2016

@rust-lang/lang Can somebody be assigned to this one? It's P-high.

@pnkfelix
Copy link
Member

arguably this is perhaps not a regression... @arielb1 says the problem is caused by new versions of the compiler generating metadata that old versions of rustc cannot handle.

(We are supposed to have a metadata guard but the issue arises before the metadata guard fires.)

@pnkfelix
Copy link
Member

Issues here:

  1. what to do about this specific bug? Is it worth issuing a point-release to fix this? (We may be issuing a point release anyway though, due to e.g. ICE in stable 1.9 - ERROR:rbml::reader: failed to find block with tag 32 | [Avoided by removing type decl from closure scope?] #34027
  2. how do we enforce our supposed policy that reading old versions of metadata should not cause the current compiler to ICE.

@pnkfelix
Copy link
Member

simplest thing may be to just go back to writing a string instead of a u64, if someone knows the precise place that @nikomatsakis was referring to in his comment above.

arielb1 added a commit to arielb1/rust that referenced this issue Jul 2, 2016
previously, only .so files included a metadata encoding version, *outside*
of the zlib compressed area. This adds an encoding version inside the metadata
itself, in both .so and .rlib files.

Fixes rust-lang#33778.
arielb1 added a commit to arielb1/rust that referenced this issue Jul 2, 2016
previously, only .so files included a metadata encoding version, *outside*
of the zlib compressed area. This adds an encoding version inside the metadata
itself, in both .so and .rlib files.

Fixes rust-lang#33778.
bors added a commit that referenced this issue Jul 2, 2016
Make the metadata lock more robust

Fixes #33778 and friends.

I also needed to add a metadata encoding version to rlibs, as they did not have it before. To keep it backwards-compatible, I added 4 zeroes to the start of the metadata, which are treated as an empty length field by older rustcs.

r? @alexcrichton
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ P-high High priority regression-from-stable-to-nightly Performance or correctness regression from stable to nightly. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests