Try to import module before creating dummy modules with 'importmode=importlib' #9681
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The dummy modules we introduce in
insert_missing_modules
(due to #7856 and #7859)would cause problems if the dummy modules end up replacing modules
which could be imported normally because they are available in
PYTHONPATH
.Now we attempt to first import the module via normal mechanisms, and only
introduce the dummy modules if the intermediary modules don't actually exist.
Close #9645
Original PR text, when opened as a draft, for reference:
I spent some time investigating this and narrowed it down to this function:
pytest/src/_pytest/pathlib.py
Lines 595 to 612 in c01a5c1
which is called during
import_path
whenimportmode=importlib
.The function was introduced because of #7856 and #7859, and it did feel like a hack at the time, but since everything seemed to work fine, it was merged in.
Stepping into the debugger, I can see that the function is called with
"tests.conftest"
, with"test.conftest"
already being insys.modules
by then, then it creates this empty/dummy module for"tests"
.OK the reported error makes sense: we are telling
tests
is a plain module, it should be a package according to the message.I dig around a bit in the import lib resources, looking for the code which generated that error message (
"is not a pacakge"
) to understand whatimportlib
looked for in a module to recognize it as a package, and found this code in the standard library:So it seems it expects the parent module (
tests
in our case) to have a__path__
attribute, so I went ahead and quickly hacked this together to see how it behaves:This did not work however, and we get the exact same error.
Then I had another thought, what if we try to use
__import__
first, and only if that fails we attempt generate the dummy intermediate modules?The code in this PR follows that approach and works with the original issue and the rest of the test suite 1, but I'm opening this as a draft to gather the opinion of people more knowledgeable of the import machinery (@asottile).
If this looks good enough, I will add tests and a changelog entry.
Footnotes
Except for one test but that's because we call
__import__
with an emptysys.meta_path
, which raises a warning, this can easily be fixed by checkingsys.meta_path
first before calling__import__
. ↩