-
Notifications
You must be signed in to change notification settings - Fork 122
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
tests: faster testing with less duplication #460
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You'll need to get the CI to pass but otherwise looks alright.
This is broken on main too - pypy + pytest-rerunfailures + Windows blows up. I'll need to play with this and probably report it to PyPy. Maybe we can turn off rerun failure on Windows PyPy... I didn't realize you could link issues directly in the UI without mentioning them, interesting. |
f6f64e2
to
15afaf7
Compare
Shall we mark it then xfail and report it upstream, when they fix it we can re-enable it 🤔 |
@mattip, any idea why this is INTERNALERROR'ing pytest on all versions of PyPy on Windows? It didn't used to before the last release as far as I can tell. I've been trying to set up to test this off and on for the last three weeks, but it's hard to recreate locally - at least a simple pytest + pypy + windows + pytest-rerunfailures loaded without the INTERNALERROR on a demo test suite. (PyCon and a bunch of other things keep getting in the way, and I specifically have to be at home to access my Windows box). |
It seems on PyPy |
My guess is that any failing test (maybe PyPy only?) triggers some bad path through the report parsing. I submitted #467 to fix the test failures, which were actually tests marked xfail that are passing. |
@henryiii I think if you rebase now it should pass 👍 |
14ba3e5
to
22c3320
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Signed-off-by: Henry Schreiner <[email protected]>
22c3320
to
0ce89ee
Compare
This replaces the SDist and wheel tests with tests of the file contents of the SDist and wheel. Every file is checked - over-including or forgetting a file will be detected. This also simplifies the path test and runs it on only one matrix element (all OS's) in order to reduce our testing load - currently we fill and block the entire PyPI's quota when running. If something is added that breaks build's ability to run from the path, it should break that matrix element.
With the newly reduced testing load, I've then enabled testing PyPy 3.8 and 3.9 as well.