Regular incremental (?) build failures during stage 0 build #99538

RalfJung · 2022-07-21T00:56:29Z

Regularly, when I do a git pull that skips over a few days worth of commits, or switch between branches based on different master commits, a ./x.py check fails afterwards. It fails with errors like "lifetime may not live long enough" or "this associated function takes 2 generic arguments but 1 generic argument was supplied" -- regular build errors, not ICEs. The errors are completely bogus though. Here's an example of the most recent one that I experienced today:

Checking stage0 std test/bench/example targets (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
warning: profiles for the non root package will be ignored, specify profiles at the workspace root:
package:   /home/r/src/rust/rustc.2/src/tools/miri/Cargo.toml
workspace: /home/r/src/rust/rustc.2/Cargo.toml
    Checking core v0.0.0 (/home/r/src/rust/rustc.2/library/core)
error[E0107]: this associated function takes 2 generic arguments but 1 generic argument was supplied
   --> library/core/tests/any.rs:145:14
    |
145 |             .provide_value::<String>(|| "bye".to_owned());
    |              ^^^^^^^^^^^^^   ------ supplied 1 generic argument
    |              |
    |              expected 2 generic arguments
    |
help: add missing generic argument
    |
145 |             .provide_value::<String, F>(|| "bye".to_owned());
    |                                    +++

error[E0107]: this function takes 2 generic arguments but 1 generic argument was supplied
   --> library/core/tests/any.rs:185:9
    |
185 |         request_ref::<T>(self)
    |         ^^^^^^^^^^^   - supplied 1 generic argument
    |         |
    |         expected 2 generic arguments
    |
help: add missing generic argument
    |
185 |         request_ref::<T, P>(self)
    |                        +++

error[E0107]: this function takes 2 generic arguments but 1 generic argument was supplied
   --> library/core/tests/any.rs:154:19
    |
154 |     assert_eq!(&**request_ref::<String>(obj).unwrap(), "hello");
    |                   ^^^^^^^^^^^   ------ supplied 1 generic argument
    |                   |
    |                   expected 2 generic arguments
    |
help: add missing generic argument
    |
154 |     assert_eq!(&**request_ref::<String, P>(obj).unwrap(), "hello");
    |                                       +++

error[E0107]: this function takes 2 generic arguments but 1 generic argument was supplied
   --> library/core/tests/any.rs:155:18
    |
155 |     assert_eq!(&*request_value::<String>(obj).unwrap(), "bye");
    |                  ^^^^^^^^^^^^^   ------ supplied 1 generic argument
    |                  |
    |                  expected 2 generic arguments
    |
help: add missing generic argument
    |
155 |     assert_eq!(&*request_value::<String, P>(obj).unwrap(), "bye");
    |                                        +++

error[E0107]: this function takes 2 generic arguments but 1 generic argument was supplied
   --> library/core/tests/any.rs:156:16
    |
156 |     assert_eq!(request_value::<u8>(obj), None);
    |                ^^^^^^^^^^^^^   -- supplied 1 generic argument
    |                |
    |                expected 2 generic arguments
    |
help: add missing generic argument
    |
156 |     assert_eq!(request_value::<u8, P>(obj), None);
    |                                  +++

error[E0107]: this function takes 2 generic arguments but 1 generic argument was supplied
   --> library/core/tests/any.rs:164:19
    |
164 |     assert_eq!(&**request_ref::<String>(&*obj).unwrap(), "hello");
    |                   ^^^^^^^^^^^   ------ supplied 1 generic argument
    |                   |
    |                   expected 2 generic arguments
    |
help: add missing generic argument
    |
164 |     assert_eq!(&**request_ref::<String, P>(&*obj).unwrap(), "hello");
    |                                       +++

error[E0107]: this function takes 2 generic arguments but 1 generic argument was supplied
   --> library/core/tests/any.rs:165:18
    |
165 |     assert_eq!(&*request_value::<String>(&*obj).unwrap(), "bye");
    |                  ^^^^^^^^^^^^^   ------ supplied 1 generic argument
    |                  |
    |                  expected 2 generic arguments
    |
help: add missing generic argument
    |
165 |     assert_eq!(&*request_value::<String, P>(&*obj).unwrap(), "bye");
    |                                        +++

error[E0107]: this function takes 2 generic arguments but 1 generic argument was supplied
   --> library/core/tests/any.rs:166:16
    |
166 |     assert_eq!(request_value::<u8>(&*obj), None);
    |                ^^^^^^^^^^^^^   -- supplied 1 generic argument
    |                |
    |                expected 2 generic arguments
    |
help: add missing generic argument
    |
166 |     assert_eq!(request_value::<u8, P>(&*obj), None);
    |                                  +++

error[E0107]: this function takes 2 generic arguments but 1 generic argument was supplied
   --> library/core/tests/any.rs:174:19
    |
174 |     assert_eq!(&**request_ref::<String>(&obj).unwrap(), "hello");
    |                   ^^^^^^^^^^^   ------ supplied 1 generic argument
    |                   |
    |                   expected 2 generic arguments
    |
help: add missing generic argument
    |
174 |     assert_eq!(&**request_ref::<String, P>(&obj).unwrap(), "hello");
    |                                       +++

error[E0107]: this function takes 2 generic arguments but 1 generic argument was supplied
   --> library/core/tests/any.rs:175:18
    |
175 |     assert_eq!(&*request_value::<String>(&obj).unwrap(), "bye");
    |                  ^^^^^^^^^^^^^   ------ supplied 1 generic argument
    |                  |
    |                  expected 2 generic arguments
    |
help: add missing generic argument
    |
175 |     assert_eq!(&*request_value::<String, P>(&obj).unwrap(), "bye");
    |                                        +++

error[E0107]: this function takes 2 generic arguments but 1 generic argument was supplied
   --> library/core/tests/any.rs:176:16
    |
176 |     assert_eq!(request_value::<u8>(&obj), None);
    |                ^^^^^^^^^^^^^   -- supplied 1 generic argument
    |                |
    |                expected 2 generic arguments
    |
help: add missing generic argument
    |
176 |     assert_eq!(request_value::<u8, P>(&obj), None);
    |                                  +++

For more information about this error, try `rustc --explain E0107`.
error: could not compile `core` due to 11 previous errors

These always go away when I do ./x.py clean. This started some time not too long before Jun 28 and has been going on since then.

The text was updated successfully, but these errors were encountered:

Bump to latest beta bootstrap compiler Hopefully this will address rust-lang#99538, but I'm not sure we can confirm that without rolling it out. Should be safe in general, so likely little risk to just landing this.

ehuss · 2022-07-21T05:25:04Z

Can you share your config.toml? I tried building every commit with incremental for the last 3 weeks in sequence and randomly, and it didn't seem to have any issues.

The particular error you posted seems to imply that it is loading an outdated rmeta or rlib file when running the second cargo with --all-targets. I can't imagine how that might be happening, since the stage0-sysroot gets rebuilt.

RalfJung · 2022-07-21T12:29:40Z

Here re the non-comment non-empty lines of my config.toml:

changelog-seen = 2
[llvm]
download-ci-llvm = true
ninja = true
[build]
submodules = false
extended = true
tools = ["miri"]
[rust]
codegen-units = 0
debug-assertions = true
debug-assertions-std = false
debuginfo-level = 1
incremental = true
deny-warnings = false

RalfJung · 2022-07-23T14:23:54Z

Now that the bootstrap compiler was bumped, I am getting new exciting errors:

error[E0514]: found crate `unic_langid_impl` compiled by an incompatible version of rustc
 --> /home/r/.cargo/registry/src/github.com-1ecc6299db9ec823/unic-langid-macros-0.9.0/src/lib.rs:2:9
  |
2 | pub use unic_langid_impl::{subtags, LanguageIdentifier};
  |         ^^^^^^^^^^^^^^^^
  |
  = help: please recompile that crate using this compiler (rustc 1.63.0-beta.6 (efd358333 2022-07-16)) (consider running `cargo clean` first)
  = note: the following crate versions were found:
          crate `unic_langid_impl` compiled by rustc 1.63.0-beta.2 (6c1f14289 2022-06-28): /home/r/src/rust/rustc/build/x86_64-unknown-linux-gnu/stage0-rustc/release/deps/libunic_langid_impl-ca98cc3868906402.rmeta

error[E0514]: found crate `unicode_xid` compiled by an incompatible version of rustc
  --> /home/r/.cargo/registry/src/github.com-1ecc6299db9ec823/proc-macro2-1.0.37/src/fallback.rs:16:5
   |
16 | use unicode_xid::UnicodeXID;
   |     ^^^^^^^^^^^
   |
   = help: please recompile that crate using this compiler (rustc 1.63.0-beta.6 (efd358333 2022-07-16)) (consider running `cargo clean` first)
   = note: the following crate versions were found:
           crate `unicode_xid` compiled by rustc 1.63.0-beta.2 (6c1f14289 2022-06-28): /home/r/src/rust/rustc/build/x86_64-unknown-linux-gnu/stage0-rustc/release/deps/libunicode_xid-9976faca336fa7c2.rmeta

ehuss · 2022-07-23T16:10:08Z

I don't suppose you could look back through your history just before the error to see what steps you took? Like did you run commands A, B, C, switch from commit X to commit Y, and then run D and immediately there was an error?

Also, it can be helpful to post the entire output from x.py, to at least know what steps it is taking. Like, did it try to build unicode_xid? Or, did for some reason, did it think it wasn't out-of-date? Why did that happen but for some reason it thought proc-macro2 was out-of-date?

Is this reproducible? Like, if you run ./x.py check again, do you get the exact same errors? If so, it could be helpful to collect some debugging information (like running with -v, using CARGO_LOG, and the contents of the deps directories).

Are you using something like rust-analyzer? If so, how is that set up? Perhaps that is interfering?

I tried for a while with your config to reproduce what you are seeing, but I haven't been able to. I'm having a hard time imagining how that could happen. For example, the second error:

It was compiling proc-macro2 for some proc-macro. Presumably because the compiler version changed.
proc-macro2 depends on unicode_xid.
unicode_xid was not recompiled, and rustc picked up a stale copy.

Normally, when the compiler version changes, cargo detects the fingerprint change from a hash of rustc -Vv. Since the two beta versions are different, it should have noticed that unicode_xid was out of date and rebuilt it.

Perhaps it is possible that the crate loader is rejecting the correct unicode_xid, but only reporting the error for some other stale file? For example, if for some reason there was a "stale" .rlib and a "current" .rmeta, rustc will (IIRC) prefer the rlib over the rmeta. However, your error message is mentioning the .rmeta is stale... Regardless, it could really help to know roughly which steps were taken to lead to this point. Like, are you running combinations of ./x.py check and ./x.py build? Does it happen immediately after you update git? etc.

RalfJung · 2022-07-23T17:52:29Z

Here is the full output that ./x.py check compiler/rustc produced:

Building rustbuild
    Finished dev [unoptimized] target(s) in 0.04s
Checking stage0 std artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
    Finished release [optimized + debuginfo] target(s) in 0.26s
Checking stage0 std test/bench/example targets (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
    Finished release [optimized + debuginfo] target(s) in 0.16s
Checking stage0 compiler artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
    Checking proc-macro2 v1.0.37
    Checking unic-langid-macros v0.9.0
    Checking rustc_ast v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_ast)
    Checking rustc_error_messages v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_error_messages)
    Checking rustc_target v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_target)
    Checking rustc_feature v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_feature)
    Checking rustc_log v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_log)
error[E0514]: found crate `unic_langid_impl` compiled by an incompatible version of rustc
 --> /home/r/.cargo/registry/src/github.com-1ecc6299db9ec823/unic-langid-macros-0.9.0/src/lib.rs:2:9
  |
2 | pub use unic_langid_impl::{subtags, LanguageIdentifier};
  |         ^^^^^^^^^^^^^^^^
  |
  = help: please recompile that crate using this compiler (rustc 1.63.0-beta.6 (efd358333 2022-07-16)) (consider running `cargo clean` first)
  = note: the following crate versions were found:
          crate `unic_langid_impl` compiled by rustc 1.63.0-beta.2 (6c1f14289 2022-06-28): /home/r/src/rust/rustc/build/x86_64-unknown-linux-gnu/stage0-rustc/release/deps/libunic_langid_impl-ca98cc3868906402.rmeta

error[E0514]: found crate `unicode_xid` compiled by an incompatible version of rustc
  --> /home/r/.cargo/registry/src/github.com-1ecc6299db9ec823/proc-macro2-1.0.37/src/fallback.rs:16:5
   |
16 | use unicode_xid::UnicodeXID;
   |     ^^^^^^^^^^^
   |
   = help: please recompile that crate using this compiler (rustc 1.63.0-beta.6 (efd358333 2022-07-16)) (consider running `cargo clean` first)
   = note: the following crate versions were found:
           crate `unicode_xid` compiled by rustc 1.63.0-beta.2 (6c1f14289 2022-06-28): /home/r/src/rust/rustc/build/x86_64-unknown-linux-gnu/stage0-rustc/release/deps/libunicode_xid-9976faca336fa7c2.rmeta

error: could not compile `unic-langid-macros` due to previous error
warning: build failed, waiting for other jobs to finish...
error: could not compile `proc-macro2` due to previous error
Build completed unsuccessfully in 0:00:04

Is this reproducible? Like, if you run ./x.py check again, do you get the exact same errors?

Yes. But I have no idea how to get my checkout into that state. (And this checkout has been ./x.py cleaned since then. I needed to get a patch written...)

Are you using something like rust-analyzer? If so, how is that set up? Perhaps that is interfering?

I am increasingly suspecting that this might be the case. That would explain why it never happens when I just chain some commands on the terminal. rust-analyzer does some weird things for its proc-macro support (useRustcWrapper is a thing, it injects itself deep into the build process it seems), maybe that is going wrong.

My config pretty much matches the rustc-dev-guide recommendations, I think:

{
    "rust-analyzer.rustc.source": "./Cargo.toml",
    "rust-analyzer.linkedProjects": [
        "./Cargo.toml",
        //"./src/bootstrap/Cargo.toml"
    ],
    "rust-analyzer.checkOnSave.overrideCommand": [
        "python3",
        "x.py",
        "check",
        "--json-output",
        "library/std",
        "compiler/rustc",
    ],
    // This also affects proc macros
    "rust-analyzer.cargo.buildScripts.overrideCommand": [
        "python3",
        "x.py",
        "check",
        "--json-output",
        "compiler/rustc",
    ],
    "rust-analyzer.rustfmt.overrideCommand": [
        "./build/x86_64-unknown-linux-gnu/stage0/bin/rustfmt",
        "--edition=2021"
    ],
    "files.watcherExclude": {
        "*rustc*/src/llvm-project/**": true,
        "*rustc*/build/**": true,
    },
    "files.exclude": {
        "src/llvm-project/**": true,
        "build/**": true
    },
}

Today I also globally disabled build scrips and proc macros, but it is possible that I did that after I got the error mentioned above.

Regardless, it could really help to know roughly which steps were taken to lead to this point.

What I did today in that checkout is to switch from some old branch (forgot which one) to latest master, open vscode, and I also ran ./x.py fmt. Then I wondered why vscode didn't show compilation errors for clearly ill-typed code, and ran the command I quoted above.

bjorn3 · 2022-07-23T17:56:15Z

I had the same issue for cg_clif in rust-lang/rust-analyzer#12845

ehuss · 2022-07-23T19:27:04Z

OK, I think this almost certainly an issue with rust-analyzer. Here are some reproduction steps using the config you posted:

Exit VSCode
git co be9cfb307e6928600a9769695e02d6296e8e1174
./x.py clean
./x.py check compiler
git co 22d25f21dc008785f52e7c2833de4f4236b1066b
code .
Wait for r-a to do its indexing stuff.
./x.py check compiler

RalfJung · 2022-08-08T13:17:07Z

I have not seen this problem for a while, since I disabled proc macro and build script support in RA.

jyn514 · 2023-04-09T02:58:41Z

The upstream issue has been closed; I think this particular bug is no longer a problem. Thank you for reporting incremental bugs ❤️ I know it's easy to dismiss them as "rust-lang/rust has a weird build system" but the reports really are helpful.

Mark-Simulacrum mentioned this issue Jul 21, 2022

Bump to latest beta bootstrap compiler #99540

Merged

RalfJung mentioned this issue Aug 8, 2022

The no-op always-success RUSTC_WRAPPER breaks build scripts rust-lang/rust-analyzer#12973

Closed

jyn514 added A-incr-comp Area: Incremental compilation C-bug Category: This is a bug. labels Apr 9, 2023

jyn514 closed this as completed Apr 9, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Regular incremental (?) build failures during stage 0 build #99538

Regular incremental (?) build failures during stage 0 build #99538

RalfJung commented Jul 21, 2022 •

edited

Loading

ehuss commented Jul 21, 2022

RalfJung commented Jul 21, 2022

RalfJung commented Jul 23, 2022

ehuss commented Jul 23, 2022

RalfJung commented Jul 23, 2022 •

edited

Loading

bjorn3 commented Jul 23, 2022

ehuss commented Jul 23, 2022

RalfJung commented Aug 8, 2022

jyn514 commented Apr 9, 2023

Regular incremental (?) build failures during stage 0 build #99538

Regular incremental (?) build failures during stage 0 build #99538

Comments

RalfJung commented Jul 21, 2022 • edited Loading

ehuss commented Jul 21, 2022

RalfJung commented Jul 21, 2022

RalfJung commented Jul 23, 2022

ehuss commented Jul 23, 2022

RalfJung commented Jul 23, 2022 • edited Loading

bjorn3 commented Jul 23, 2022

ehuss commented Jul 23, 2022

RalfJung commented Aug 8, 2022

jyn514 commented Apr 9, 2023

RalfJung commented Jul 21, 2022 •

edited

Loading

RalfJung commented Jul 23, 2022 •

edited

Loading