Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New approach to lazy loading of pygame submodules (surfarray, sndarray) #3249

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

aatle
Copy link
Contributor

@aatle aatle commented Dec 1, 2024

A new approach to lazy loading pygame submodules after #3232 had a problem with slower attribute accesses (__getattr__).

Implementation explanation:
The implementation is twofold, where the second part has many alternatives.

First, pygame checks if numpy exists.
Then, each submodule is loaded if numpy exists and the submodule's other pygame module dependencies (e.g. mixer) load successfully. If numpy is missing or a module dependency fails, then the submodule becomes MissingModule. It is necessary to get an error to use its message in MissingModule.reason, so try-except is used.

If successful, the submodule is created and put into sys.modules, using the documented LazyLoader implementation, relying on import machinery.
This makes it so that the submodule exists, but its contents are not executed until the first attribute or method access on the submodule.
It does this by changing the module class temporarily to _LazyModule; after the module is actually loaded, the module class is set back to the normal module type.

Potential differences from current behavior:
The initial checks are not completely perfect. It is assumed that if numpy is importable, it will not raise an ImportError nor OSError.
Obviously, numpy not being loaded by pygame automatically is an intended difference. Instead, it is loaded upon the first attribute access of either submodule. I do not expect this to cause any lag spikes in-game: the user probably imports numpy themselves, and it will also load if directly importing one of the submodule functions, or using one during game loading.
After a submodule is loaded successfully, there should not be any noticeable difference from current behavior. The module class is the same, the original loader is put back, the module code was not changed.
https://docs.python.org/3/library/importlib.html#importlib.util.LazyLoader

Note: For projects where startup time is critical, this class allows for potentially minimizing the cost of loading a module if it is never used. For projects where startup time is not essential then use of this class is heavily discouraged due to error messages created during loading being postponed and thus occurring out of context.

There are no expected errors during postponed loading of these submodules, as they should have all been handled at the start.

Lazy loading of numpy has great potential benefits, so it's worth the extra complexity.

Closes #3232

@aatle aatle requested a review from a team as a code owner December 1, 2024 23:36
@@ -23,6 +23,8 @@ Each sample is an 8-bit or 16-bit integer, depending on the data format. A
stereo sound file has two values per sample, while a mono sound file only has
one.

.. versionchanged:: 2.5.3 sndarray module is lazily loaded to avoid loading NumPy needlessly
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This still doesn't explain why the action is actually being taken.

Why is it bad to load NumPy?

@Starbuck5
Copy link
Member

Starbuck5 commented Dec 6, 2024

@damusss You cast some doubt about this PR working with PyInstaller on discord but did not elaborate.

If there's a problem here can you explain what you did please.

@damusss
Copy link
Member

damusss commented Dec 6, 2024

If there's a problem here can you explain what you did please.

🫡 I tested both normal pygame-ce and this branch, and it looks like both imports are successful. when I got that error I must have been using an incomplete version of the pull request
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants