Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

meta_test_multiple_versions_strategy is spuriously failing #6613

Closed
ehuss opened this issue Jan 30, 2019 · 5 comments
Closed

meta_test_multiple_versions_strategy is spuriously failing #6613

ehuss opened this issue Jan 30, 2019 · 5 comments
Labels
A-dependency-resolution Area: dependency resolution and the resolver A-testing-cargo-itself Area: cargo's tests C-bug Category: bug

Comments

@ehuss
Copy link
Contributor

ehuss commented Jan 30, 2019

meta_test_multiple_versions_strategy has failed a few times during CI:

with errors that look like:

thread 'support::resolver::meta_test_multiple_versions_strategy' panicked at 'In 1280 tries we did not see a wide enough distribution of multiple versions of the same library! dis: [1099, 45, 31, 16, 20, 15, 3, 3, 0, 9]', tests/testsuite/support/resolver.rs:564:5

thread 'support::resolver::meta_test_multiple_versions_strategy' panicked at 'In 1280 tries we did not see a wide enough distribution of multiple versions of the same library! dis: [1026, 75, 30, 24, 14, 5, 4, 7, 0, 19]', src/tools/cargo/tests/testsuite/support/resolver.rs:564:5

@ehuss
Copy link
Contributor Author

ehuss commented Feb 1, 2019

Hit again today.
---- support::resolver::meta_test_multiple_versions_strategy stdout ----
thread 'support::resolver::meta_test_multiple_versions_strategy' panicked at 'In 1280 tries we did not see a wide enough distribution of multiple versions of the same library! dis: [1086, 55, 16, 11, 9, 10, 3, 10, 0, 22]', tests/testsuite/support/resolver.rs:564:5

@Eh2406 Eh2406 added A-dependency-resolution Area: dependency resolution and the resolver A-testing-cargo-itself Area: cargo's tests labels Feb 2, 2019
@Eh2406
Copy link
Contributor

Eh2406 commented Feb 2, 2019

So with the number of different things running Cargos test sweet flaky tests are not acceptable. I am sorry for adding them! It does make me wonder if something changed to make the distribution worse. The real solution is to switch to using a fixed seed so as to be sure we have consistent tests.

@ehuss
Copy link
Contributor Author

ehuss commented Feb 15, 2019

Also occurs with meta_test_deep_trees_from_strategy
https://travis-ci.com/rust-lang/cargo/jobs/178165928

thread 'support::resolver::meta_test_deep_trees_from_strategy' panicked at 'In 1280 tries we did not see a wide enough distribution of dependency trees! dis: [49, 852, 94, 43, 32, 23, 21, 24, 16, 16, 17, 10, 9, 7, 5, 9, 8, 4, 4, 0, 23]', tests/testsuite/support/resolver.rs:524:5

@mati865
Copy link
Contributor

mati865 commented Mar 21, 2019

Fixed by #6753?

@Eh2406
Copy link
Contributor

Eh2406 commented Mar 21, 2019

Yes! Thanks!

@Eh2406 Eh2406 closed this as completed Mar 21, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-dependency-resolution Area: dependency resolution and the resolver A-testing-cargo-itself Area: cargo's tests C-bug Category: bug
Projects
None yet
Development

No branches or pull requests

3 participants