compiletest failures are close to unreadable because of JSON #36516

brson · 2016-09-16T01:17:29Z

Here's one I hit recently:

---- [run-pass] run-pass/hello.rs stdout ----                                                                                                                                             [124/9070]

error: compilation failed!
status: exit code: 101
command: /mnt2/dev/rust/build/x86_64-unknown-linux-gnu/stage2/bin/rustc /mnt2/dev/rust/src/test/run-pass/hello.rs -L /mnt2/dev/rust/build/x86_64-unknown-linux-gnu/test/run-pass --target=asmjs-unkn
own-emscripten --error-format json -L /mnt2/dev/rust/build/x86_64-unknown-linux-gnu/test/run-pass/hello.stage2-asmjs-unknown-emscripten.run-pass.libaux -C prefer-dynamic -o /mnt2/dev/rust/build/x8
6_64-unknown-linux-gnu/test/run-pass/hello.stage2-asmjs-unknown-emscripten.js -Crpath -O -Lnative=/mnt2/dev/rust/build/asmjs-unknown-emscripten/rust-test-helpers
stdout:
------------------------------------------

------------------------------------------
stderr:
------------------------------------------
{"message":"linking with `emcc` failed: exit code: 1","code":null,"level":"error","spans":[],"children":[{"message":"\"emcc\" \"-L\" \"/mnt2/dev/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rust
lib/asmjs-unknown-emscripten/lib\" \"/mnt2/dev/rust/build/x86_64-unknown-linux-gnu/test/run-pass/hello.0.o\" \"-o\" \"/mnt2/dev/rust/build/x86_64-unknown-linux-gnu/test/run-pass/hello.stage2-asmjs
-unknown-emscripten.js\" \"-Wl,--gc-sections\" \"-Wl,-O1\" \"-nodefaultlibs\" \"-L\" \"/mnt2/dev/rust/build/x86_64-unknown-linux-gnu/test/run-pass\" \"-L\" \"/mnt2/dev/rust/build/x86_64-unknown-li
nux-gnu/test/run-pass/hello.stage2-asmjs-unknown-emscripten.run-pass.libaux\" \"-L\" \"/mnt2/dev/rust/build/asmjs-unknown-emscripten/rust-test-helpers\" \"-L\" \"/mnt2/dev/rust/build/x86_64-unknow
n-linux-gnu/stage2/lib/rustlib/asmjs-unknown-emscripten/lib\" \"-Wl,-Bstatic\" \"-Wl,-Bdynamic\" \"/mnt2/dev/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/asmjs-unknown-emscripten/lib/lib
std-c5ae712d21ff1b88.rlib\" \"/mnt2/dev/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/asmjs-unknown-emscripten/lib/libpanic_unwind-0aed89c908dbae00.rlib\" \"/mnt2/dev/rust/build/x86_64-un
known-linux-gnu/stage2/lib/rustlib/asmjs-unknown-emscripten/lib/libunwind-4406c7bac693a305.rlib\" \"/mnt2/dev/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/asmjs-unknown-emscripten/lib/li
brand-4bf39e147f01f701.rlib\" \"/mnt2/dev/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/asmjs-unknown-emscripten/lib/libcollections-94042b4fb30df9b8.rlib\" \"/mnt2/dev/rust/build/x86_64-u
nknown-linux-gnu/stage2/lib/rustlib/asmjs-unknown-emscripten/lib/librustc_unicode-e09cf12a6737fbe8.rlib\" \"/mnt2/dev/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/asmjs-unknown-emscripte
n/lib/liballoc-e2faee10ccc53a4e.rlib\" \"/mnt2/dev/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/asmjs-unknown-emscripten/lib/liballoc_system-57a84a70354893c6.rlib\" \"/mnt2/dev/rust/buil
d/x86_64-unknown-linux-gnu/stage2/lib/rustlib/asmjs-unknown-emscripten/lib/liblibc-5f7c0b0b6e65da09.rlib\" \"/mnt2/dev/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/asmjs-unknown-emscript
en/lib/libcore-380af50841193d3f.rlib\" \"/mnt2/dev/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/asmjs-unknown-emscripten/lib/libcompiler_builtins-3720833c50b88cf0.rlib\" \"-l\" \"c\" \"-
s\" \"ERROR_ON_UNDEFINED_SYMBOLS=1\"","code":null,"level":"note","spans":[],"children":[],"rendered":null},{"message":"/mnt2/dev/emsdk_portable/clang/fastcomp/build_incoming_64/bin/llvm-nm: /mnt2/
dev/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/asmjs-unknown-emscripten/lib/libstd-c5ae712d21ff1b88.rlib(rust.metadata.bin) The end of the file was unexpectedly encountered\n\n/mnt2/de
v/emsdk_portable/clang/fastcomp/build_incoming_64/bin/llvm-nm: /mnt2/dev/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/asmjs-unknown-emscripten/lib/libpanic_unwind-0aed89c908dbae00.rlib(r
ust.metadata.bin) The end of the file was unexpectedly encountered\n\n/mnt2/dev/emsdk_portable/clang/fastcomp/build_incoming_64/bin/llvm-nm: /mnt2/dev/rust/build/x86_64-unknown-linux-gnu/stage2/li
b/rustlib/asmjs-unknown-emscripten/lib/libunwind-4406c7bac693a305.rlib(rust.metadata.bin) The end of the file was unexpectedly encountered\n\n/mnt2/dev/emsdk_portable/clang/fastcomp/build_incoming
_64/bin/llvm-nm: /mnt2/dev/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/asmjs-unknown-emscripten/lib/librand-4bf39e147f01f701.rlib(rust.metadata.bin) The end of the file was unexpectedly
 encountered\n\n/mnt2/dev/emsdk_portable/clang/fastcomp/build_incoming_64/bin/llvm-nm: /mnt2/dev/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/asmjs-unknown-emscripten/lib/libcollections-
94042b4fb30df9b8.rlib(rust.metadata.bin) The end of the file was unexpectedly encountered\n\n/mnt2/dev/emsdk_portable/clang/fastcomp/build_incoming_64/bin/llvm-nm: /mnt2/dev/rust/build/x86_64-unkn
own-linux-gnu/stage2/lib/rustlib/asmjs-unknown-emscripten/lib/librustc_unicode-e09cf12a6737fbe8.rlib(rust.metadata.bin) The end of the file was unexpectedly encountered\n\n/mnt2/dev/emsdk_portable
/clang/fastcomp/build_incoming_64/bin/llvm-nm: /mnt2/dev/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/asmjs-unknown-emscripten/lib/liballoc-e2faee10ccc53a4e.rlib(rust.metadata.bin) The e
nd of the file was unexpectedly encountered\n\n/mnt2/dev/emsdk_portable/clang/fastcomp/build_incoming_64/bin/llvm-nm: /mnt2/dev/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/asmjs-unknown
-emscripten/lib/liballoc_system-57a84a70354893c6.rlib(rust.metadata.bin) The end of the file was unexpectedly encountered\n\n/mnt2/dev/emsdk_portable/clang/fastcomp/build_incoming_64/bin/llvm-nm:
/mnt2/dev/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/asmjs-unknown-emscripten/lib/liblibc-5f7c0b0b6e65da09.rlib(rust.metadata.bin) The end of the file was unexpectedly encountered\n\n/
mnt2/dev/emsdk_portable/clang/fastcomp/build_incoming_64/bin/llvm-nm: /mnt2/dev/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/asmjs-unknown-emscripten/lib/libcore-380af50841193d3f.rlib(ru
st.metadata.bin) The end of the file was unexpectedly encountered\n\n/mnt2/dev/emsdk_portable/clang/fastcomp/build_incoming_64/bin/llvm-nm: /mnt2/dev/rust/build/x86_64-unknown-linux-gnu/stage2/lib
/rustlib/asmjs-unknown-emscripten/lib/libcompiler_builtins-3720833c50b88cf0.rlib(rust.metadata.bin) The end of the file was unexpectedly encountered\n\nerror: unresolved symbol: _Unwind_GetRegionS
tart\nerror: unresolved symbol: _Unwind_SetGR\nerror: unresolved symbol: _Unwind_GetLanguageSpecificData\nerror: unresolved symbol: _Unwind_SetIP\nerror: unresolved symbol: _Unwind_GetTextRelBase\
nerror: unresolved symbol: _Unwind_GetDataRelBase\nAborting compilation due to previous errors | undefined\nTraceback (most recent call last):\n  File \"/mnt2/dev/emsdk_portable/emscripten/incomin
g/emcc\", line 13, in <module>\n    emcc.run()\n  File \"/mnt2/dev/emsdk_portable/emscripten/incoming/emcc.py\", line 1510, in run\n    final = shared.Building.emscripten(final, append_ext=False,
extra_args=extra_args)\n  File \"/mnt2/dev/emsdk_portable/emscripten/incoming/tools/shared.py\", line 1707, in emscripten\n    call_emscripten(cmdline)\n  File \"/mnt2/dev/emsdk_portable/emscripte
n/incoming/emscripten.py\", line 1756, in _main\n    temp_files.run_and_clean(lambda: main(\n  File \"/mnt2/dev/emsdk_portable/emscripten/incoming/tools/tempfiles.py\", line 78, in run_and_clean\n
    return func()\n  File \"/mnt2/dev/emsdk_portable/emscripten/incoming/emscripten.py\", line 1761, in <lambda>\n    DEBUG=DEBUG,\n  File \"/mnt2/dev/emsdk_portable/emscripten/incoming/emscripten
.py\", line 1662, in main\n    temp_files=temp_files, DEBUG=DEBUG)\n  File \"/mnt2/dev/emsdk_portable/emscripten/incoming/emscripten.py\", line 91, in emscript\n    glue, forwarded_data = compiler
_glue(metadata, settings, libraries, compiler_engine, temp_files, DEBUG)\n  File \"/mnt2/dev/emsdk_portable/emscripten/incoming/emscripten.py\", line 286, in compiler_glue\n    cwd=path_from_root(
'src'), error_limit=300)\n  File \"/mnt2/dev/emsdk_portable/emscripten/incoming/tools/jsrun.py\", line 76, in run_js\n    raise Exception('Expected the command ' + str(command) + ' to finish with
return code ' + str(assert_returncode) + ', but it returned with code ' + str(proc.returncode) + ' instead! Output: ' + str(ret)[:error_limit])\nException: Expected the command ['/mnt2/dev/emsdk_p
ortable/node/4.1.1_64bit/bin/node', '/mnt2/dev/emsdk_portable/emscripten/incoming/src/compiler.js', '/tmp/tmpnGlx_B.txt', '/mnt2/dev/emsdk_portable/emscripten/incoming/src/library_pthread_stub.js'
] to finish with return code 0, but it returned with code 1 instead! Output: // The Module object: Our interface to the outside world. We import\n// and export values on it, and do the work to get
 that through\n// closure compiler if necessary. There are various ways Module can be used:\n// 1. Not defined. We create it here\n// 2. A function parameter, function(Module) { ..gener\n","code":
null,"level":"note","spans":[],"children":[],"rendered":null}],"rendered":null}
{"message":"aborting due to previous error","code":null,"level":"error","spans":[],"children":[],"rendered":null}

------------------------------------------

thread '[run-pass] run-pass/hello.rs' panicked at 'explicit panic', src/tools/compiletest/src/runtest.rs:2359
note: Run with `RUST_BACKTRACE=1` for a backtrace.

compiletest uses the --error-format=json switch to capture more precise error information then dumps that out for humans to read.

Instead we might either: have the compiler output the json to a side file; reformat the JSON into something that makes more sense.

The text was updated successfully, but these errors were encountered:

brson · 2016-09-16T01:19:37Z

@nikomatsakis suggests that the compile-fail output is better and json is only used for that case, so compiletest could be modified to only use json mode there.

brson · 2016-09-16T01:54:56Z

I've started a patch.

brson · 2016-09-16T02:50:13Z

Simply disabling json errors for rpass isn't sufficient because there are rpass errors that use the new precise error checking to check warnings.

nikomatsakis · 2016-09-16T13:34:56Z

@brson example? Perhaps those tests should be made into ui tests? I forget if those work for run-pass tests or not.

In general, as discussed on this thread, I'd like to move over to ui testing more and more. One of the things I had hoped to do was to generalize the concept of "run-pass" vs "compile-fail" test so that we just had "tests", where you could specify in the test how it was expected to behave (compile successfully, compile with errors, etc) and then use a reference file to check the compiler's output. I don't claim to have this vision 100% worked out yet though.

brson · 2016-09-16T16:57:13Z

I don't think I'll get to fixing this after all because of the aforementioned issue.

brson · 2016-09-16T16:58:00Z

@nikomatsakis here are the rpass failures that check warnings:

    [run-pass] run-pass/deprecated-macro_escape-inner.rs
    [run-pass] run-pass/deprecated-macro_escape.rs
    [run-pass] run-pass/deriving-meta-empty-trait-list.rs
    [run-pass] run-pass/enum-size-variance.rs
    [run-pass] run-pass/issue-19100.rs

alexcrichton · 2016-09-27T19:37:57Z

@brson do you have an update on this, just hit this again with run-pass-fulldeps tests unfortunately :(

@nikomatsakis I think the problem here is that the tests aren't UI tests, just standard run-pass tests. Everything is json errors now which makes it harder to diagnose failures.

nikomatsakis · 2016-09-27T20:31:46Z

@alexcrichton my point is that I think we could switch run-pass to not use JSON. If the sticking point is those 5 tests, we could either fix them somehow, or convert those 5 into ui tests instead.

brson · 2016-09-27T21:06:03Z

@alexcrichton No, I'm not working on this.

alexcrichton · 2016-09-27T21:40:11Z

@nikomatsakis agreed! (that we can switch)

…ust-lang#36516. The 'run-pass' header cause a 'ui' test to execute the result. It is used to test the lint output, at the same time ensure those lints won't cause the source code to become compile-fail. 12 run-pass/run-pass-fulldeps tests gained the header and are moved to ui/ui-fulldeps. After this move, no run-pass/run-pass-fulldeps tests should rely on the compiler's JSON message. This allows us to stop passing `--error-format json` in run-pass tests, thus fixing rust-lang#36516.

…tput-36516, r=nikomatsakis Introduce 'run-pass' header to 'ui' tests in compiletest. Fix issue rust-lang#36516. <del>`ui-run` test is a combination of `ui` test and `run-pass` test. It is used to test lint output.</del> Added support of `// run-pass` header to `ui` tests. The compiler message of each test must match the corresponding `*.stderr` file like the traditional `ui` tests. Additionally, the compiled output must be executed successfully like the `run-pass` test. 12 `run-pass`/`run-pass-fulldeps` tests are moved to `ui`/`ui-fulldeps` plus the headers. After this move, no `run-pass`/`run-pass-fulldeps` tests should rely on the compiler's JSON message. This allows us to stop passing `--error-format json` in run-pass tests, thus fixing rust-lang#36516.

est31 · 2017-06-02T04:59:22Z

Is this still an issue? I seem to get much nicer failures now.

kennytm · 2017-06-02T06:46:04Z

@est31 This issue only affects "run-pass" tests (maybe "incremental" as well). #41968 fixes it.

…ust-lang#36516. The 'run-pass' header cause a 'ui' test to execute the result. It is used to test the lint output, at the same time ensure those lints won't cause the source code to become compile-fail. 12 run-pass/run-pass-fulldeps tests gained the header and are moved to ui/ui-fulldeps. After this move, no run-pass/run-pass-fulldeps tests should rely on the compiler's JSON message. This allows us to stop passing `--error-format json` in run-pass tests, thus fixing rust-lang#36516.

… r=nikomatsakis Introduce 'run-pass' header to 'ui' tests in compiletest. Fix issue #36516. <del>`ui-run` test is a combination of `ui` test and `run-pass` test. It is used to test lint output.</del> Added support of `// run-pass` header to `ui` tests. The compiler message of each test must match the corresponding `*.stderr` file like the traditional `ui` tests. Additionally, the compiled output must be executed successfully like the `run-pass` test. 12 `run-pass`/`run-pass-fulldeps` tests are moved to `ui`/`ui-fulldeps` plus the headers. After this move, no `run-pass`/`run-pass-fulldeps` tests should rely on the compiler's JSON message. This allows us to stop passing `--error-format json` in run-pass tests, thus fixing #36516.

pnkfelix · 2018-11-05T11:24:55Z

We've changed run-pass to all be ui tests. I do not think there's any more action items on this issue.

brson added the A-testsuite Area: The testsuite used to check the correctness of rustc label Sep 16, 2016

kennytm mentioned this issue May 13, 2017

Introduce 'run-pass' header to 'ui' tests in compiletest. Fix issue #36516. #41968

Merged

Mark-Simulacrum added the C-enhancement Category: An issue proposing an enhancement or a PR with one. label Jul 26, 2017

sanmai-NL mentioned this issue Feb 17, 2018

rustc output hygiene wrt. stderr/stdout #48301

Closed

pnkfelix closed this as completed Nov 5, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

compiletest failures are close to unreadable because of JSON #36516

compiletest failures are close to unreadable because of JSON #36516

brson commented Sep 16, 2016

brson commented Sep 16, 2016

brson commented Sep 16, 2016

brson commented Sep 16, 2016

nikomatsakis commented Sep 16, 2016

brson commented Sep 16, 2016

brson commented Sep 16, 2016

alexcrichton commented Sep 27, 2016

nikomatsakis commented Sep 27, 2016

brson commented Sep 27, 2016

alexcrichton commented Sep 27, 2016

est31 commented Jun 2, 2017

kennytm commented Jun 2, 2017

pnkfelix commented Nov 5, 2018

compiletest failures are close to unreadable because of JSON #36516

compiletest failures are close to unreadable because of JSON #36516

Comments

brson commented Sep 16, 2016

brson commented Sep 16, 2016

brson commented Sep 16, 2016

brson commented Sep 16, 2016

nikomatsakis commented Sep 16, 2016

brson commented Sep 16, 2016

brson commented Sep 16, 2016

alexcrichton commented Sep 27, 2016

nikomatsakis commented Sep 27, 2016

brson commented Sep 27, 2016

alexcrichton commented Sep 27, 2016

est31 commented Jun 2, 2017

kennytm commented Jun 2, 2017

pnkfelix commented Nov 5, 2018