Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Race Condition in Console Runner #512

Closed
jugglinmike opened this issue Feb 19, 2016 · 0 comments
Closed

Race Condition in Console Runner #512

jugglinmike opened this issue Feb 19, 2016 · 0 comments

Comments

@jugglinmike
Copy link
Contributor

A race condition in the console runner's threading logic causes it to run some
test cases more than once and to not run others at all.

This example demonstrates a test run using 2 parallel threads (via -j 2). I'm
using a shell pipeline to highlight repeated runs:

$ ./tools/packaging/test262.py --command js language/reserved-words -j 2 | sort | uniq -c | sort | tail
2 language/reserved-words/7.6.1-7-10 passed in non-strict mode
2 language/reserved-words/7.6.1-7-11 passed in non-strict mode
2 language/reserved-words/7.6.1-7-1 passed in non-strict mode
2 language/reserved-words/7.6.1-7-8 passed in non-strict mode
2 language/reserved-words/7.6.1-7-8 passed in strict mode
2 language/reserved-words/7.6.1-8-3 passed in strict mode
2 language/reserved-words/7.6.1-8-6 passed in strict mode
2 language/reserved-words/S7.6.1_A1.1 failed in strict mode as expected
2 language/reserved-words/S7.6.1_A2 passed in non-strict mode
3 language/reserved-words/7.6.1-8-1 passed in strict mode

Executing the tests within a single thread (via -j 1) does not present this
problem:

$ ./tools/packaging/test262.py --command js language/reserved-words -j 1 | sort | uniq -c | sort | tail
1 language/reserved-words/7.6.1-8-9 passed in non-strict mode
1 language/reserved-words/7.6.1-8-9 passed in strict mode
1 language/reserved-words/S7.6.1_A1.1 failed in non-strict mode as expected
1 language/reserved-words/S7.6.1_A1.1 failed in strict mode as expected
1 language/reserved-words/S7.6.1_A1.2 failed in non-strict mode as expected
1 language/reserved-words/S7.6.1_A1.2 failed in strict mode as expected
1 language/reserved-words/S7.6.1_A1.3 failed in non-strict mode as expected
1 language/reserved-words/S7.6.1_A1.3 failed in strict mode as expected
1 language/reserved-words/S7.6.1_A2 passed in non-strict mode
1 language/reserved-words/S7.6.1_A2 passed in strict mode

We have a couple options for how to handle this: fix the bug or remove the
optimization. Patches and rationale incoming.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant