Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow custom allocators for PyObject memory #69

Open
nascheme opened this issue Aug 3, 2023 · 1 comment
Open

Allow custom allocators for PyObject memory #69

nascheme opened this issue Aug 3, 2023 · 1 comment

Comments

@nascheme
Copy link

nascheme commented Aug 3, 2023

The current API allows a lot of flexibly in how PyObject memory is allocated. It can be statically allocated (non-heap types, for example), allocated with CPython's allocator or allocated with some custom allocator (provided by the extension module). This flexibility must provide some advantages (I'm not really sure what they are exactly) but it also places a pretty significant constraint on the implementation of Python. Typically in a garbage collected language implementation, object memory is allocated by the runtime. For best performance, the garbage collector is intimately familiar or integrated with the object allocator. This can reduce memory overhead per-object and speed up the GC process.

We should decide if this is a required feature of the API. An alternative would be to require that PyObject memory be allocated by the runtime (which is already the case for objects with PyGC_Head). Hopefully people with compelling use cases for custom allocated can comment and explain their situation.

@gvanrossum
Copy link

This is particularly relevant since noGIL (currently) relies on mimalloc, making custom allocators either impossible or at least expensive to use. I presume something like Python's own tracemalloc continues to be used. But just providing three pointers to alternative implementations of malloc, free and realloc? That may be a challenge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants