Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

$DIR and $WORKSPACE removed in output, causing CI failure #141

Closed
AzureMarker opened this issue Nov 28, 2021 · 2 comments
Closed

$DIR and $WORKSPACE removed in output, causing CI failure #141

AzureMarker opened this issue Nov 28, 2021 · 2 comments

Comments

@AzureMarker
Copy link

AzureMarker commented Nov 28, 2021

Due to #132 the output format changed but the version was kept at 1.0.x, causing many trybuild tests to fail when upgraded to the latest version. Since libraries don't normally check in Cargo.lock, this caused my UI tests to fail in CI. This isn't a big deal, but it's unexpected and seems to be unintended based on the comment I reference below.

This is related to #140, though different because the only change in the output is the switch from $DIR to relative path.

According to this comment, it sounds like trybuild should not raise a mismatch in this case:

The normalizations that trybuild applies are always changing to smooth over differences in rustc's diagnostic output across rustc versions, and in general we do normalization in a way that makes an old captured *.stderr file succeed if it matches the normalization logic of any previous trybuild release.

#140 (comment)

Link to CI where the mismatch occurs:
https://app.circleci.com/pipelines/github/AzureMarker/shaku/8/workflows/0db93ae3-9ee0-4b77-ac18-94008f9aa5b3/jobs/17

AzureMarker added a commit to AzureMarker/shaku that referenced this issue Nov 28, 2021
trybuild updated their output format in a backwards-incompatible way.

See dtolnay/trybuild#141
@dtolnay
Copy link
Owner

dtolnay commented Nov 28, 2021

It's expected that the test outputs occasionally change. This is the nature of ui tests. We make some effort to prevent things changing spuriously, but as long as the rate of change from trybuild normalization logic is small relative to the rate of change from diagnostics changes upstream in rustc, this doesn't seem like an issue.

@dtolnay dtolnay closed this as completed Nov 28, 2021
@AzureMarker
Copy link
Author

I took another look at the test outputs that changed, and they all included $WORKSPACE. The tests that only included $DIR did not change. So there was some effort made to avoid changing the output, but $WORKSPACE was missed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants