Run x test --stage 1 in CI

[jyn514]

Similar to #95996, we expect contributors to run this frequently, so it should be tested in CI.

@eddyb suggested using the defaults in

rust/src/bootstrap/defaults/config.compiler.toml

Line 1 in 7443cc2

# These defaults are meant for contributors to the compiler who do not modify codegen or LLVM

but I'm a little hesitant to do that because some of them are very wrong for CI (e.g. download-ci-llvm) and others shouldn't affect tests but make the compiler slower (e.g. debug-logging = true). I do agree we should enable incremental for these tests, though.

Not sure which builder to add these to - adding it to mingw-check would make it significantly slower, and I don't know which of the full builders has spare capacity. @pietroalbini do you have suggestions?

_Originally posted by @eddyb in #98660 (comment)

[eddyb]

To be clear, I was mistakenly under the impression that only stage1 tests were ran for PR CI.
But I could see how that may be an issue for ignore-stage1 tests (though, is it easy to only run those in stage2?).

[jyn514]

To be clear, I was mistakenly under the impression that only stage1 tests were ran for PR CI.

ah yeah no, we currently run x test --stage 2 in PR CI. I think it might make sense to only run --stage 1 now that you mention it though! would be a lot faster and in practice there are very few tests that only fail in stage 2.

though, is it easy to only run those in stage2?

not currently, no - I think it wouldn't be impossible to add to compiletest though, it already does filtering and parses the headers, so it should be possible to combine the two. That would mean we have to build the stage 2 compiler though, so if we switch to stage 1 I think it would be ok to only run those in a full bors merge, not in PR CI.

[pietroalbini]

@jyn514 just add another builder, we have capacity for that.

[eddyb]

If we do get another builder, can we have incremental enabled on it (or rather, whichever dev profile enables that)?

Some codegen tests haven't been passing for a short while because of the difference, and might be good to be tracking something that strongly resembles local workflows.

[jyn514]

@eddyb

I'm a little hesitant to do that because some of them are very wrong for CI (e.g. download-ci-llvm) and others shouldn't affect tests but make the compiler slower (e.g. debug-logging = true). I do agree we should enable incremental for these tests, though.

[rustbot]

Error: Parsing assign command in comment failed: ...'bot assign' | error: specify user to assign to at >| ''...

Please file an issue on GitHub at triagebot if there's a problem with this bot, or reach out on #t-infra on Zulip.

[Milo123459]

@rustbot claim

[Milo123459]

What is the correct course of action here? Adding another builder?

[jyn514]

Yes, adding another builder is enough. I want to investigate running --stage 1 in PR CI instead of --stage 2 long term, but that's harder and shouldn't block testing this.

[RalfJung]

This would have caught #99619, I assume?

[eddyb]

@RalfJung Yes, despite not being filed as an issue, that's exactly the kind of test failure I was referencing in #99135 (comment) (it does require that we do enable incremental, not just test at stage 1).