Add initial support for pytest plugins #18217
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Instead of completely ignoring plugin-generated tests, handle plugin-generated tests that are instances of `pytest.Item`.
Add a test to make sure we handle plugin-generated tests correctly. There are a couple of things this tries to test that (can) happen in plugin-generated tests: - an instance of pytest.Item that isn't one of the classes that gets special cased - a test location that's different from the file that the test was created - a suite name with dots in it - an expected output path that doesn't have a dotted path to a function
Each SingleTestInfo instance is created with the default value of kind, which is "function", even when that should really be "doctest" or, as of this PR, "other".
|
|
|
Hi @pranavrajpal, thank you for your PR! We are in the process of revamping test discovery for both pytest and unittest (#17242, January iteration plan at #18214), and we are going to replace the code being modified in this PR soon. Even though I am closing this PR, thank you for bringing this use case to our attention, we'll make sure to cover it in the testing rewrite 🙂 |
|
@kimadeline Would it be possible to reconsider reviewing this PR? Work on #17242 seems to have stalled (or at the very least I can't find any reference to PRs refactoring or rewriting the pytest adapter) and this is a relatively simple but still useful change, even if it will get replaced soon. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds initial support for pytest plugin-generated tests, including running and debugging tests, and opening tests in the editor (even if the location is a non-Python file).
Debugging non-Python files still isn't supported.
This PR has the consequence of making nodeids not always be unique, because a plugin could always generate multiple tests with the same name in the same suite. Most of the code seems to assume that they will be unique, though, because the UI only shows only one test even when multiple tests have the same nodeid.
I don't know what the ideal solution for that is, because the current behavior has some strange consequences when interacting with tests with repeated nodeids, but repeating nodeids seems like a bug considering that the pytest docs say that nodeids are supposed to be unique:
Work on #16742 and #16852
Both of those tests mention crashes with specific plugins, but I haven't tested any of those plugins so I don't know if those crashes are fixed.