Skip to content

dev call 20221006

Florian Angerer edited this page Oct 6, 2022 · 12 revisions

Dev Call: 6 October 2022

Present

Antonio, Stepan, Petr V., Florian, Tim, Matti, Ronan, Mohaned

Agenda

  • Discussion item: Mixed mode binary verification step? https://github.com/hpyproject/hpy/issues/366

  • Discussion item: Limited contexts for tp_traverse and tp_destroy

    Mark has proposed limited contexts for these slots: https://github.com/markshannon/New-C-API-for-Python/blob/main/DesignRules.md#all-functions-to-take-an-opaque-context

    It would help with the numpy port if we could have e.g. ctx->Mem_Free available to tp_destroy; or ctx->Field_Load, ctx->Global_Load, ctx->Type and ctx->Is available to tp_traverse

  • Discussion item: Clarify encoding used for C strings in the API. #355 How much should we start to deviate from CPython towards the "ideal" API already now?

  • Discussion item: 1.0 release? Should we aim for one since we'll have the ABI version check very soon?

    • PyPy 0.0.4 seems ready. Review before merge? Can we add it to CI?
  • HPyBytes_AsString/AS_STRING: should we remove them and just use the buffer protocol for that?

  • Multi-phase module init

  • Module state vs HPyGlobal

Misc

  • HPyBytes_AsString/AS_STRING: * Currently, pointer is valid as long as the handle. The buffer protocol could be * There is consens that we should keep since there is not much difference to the buffer protocol and it is easier to use. * top4000 uses it 630 times * CPython might want to remove it because the underlying storage might be changed * Mark will probably remove it from his new API proposal
  • PyPy/HPy 0.0.4 changes are ready to merge; Matti asked for review * Ronan pointed out the the mixed-mode seems to be broken
  • Encoding in C strings: let's just do that and make it explicit. Everyone agreed on that.

Release 1.0

  • What do we want to achieve? Visibility?
  • Antonio's concern: what if we change the API/ABI soon after the release? This should be covered
  • Matti: let's add milestones (and add issues)
  • Antonio doesn't feel convinced that HPy is already there. He suggested to have NumPy fully ported (tests passing) before we do that.
  • The idea of having a 1.0 release is to bootstrap everything.
  • We should probably go for something like 0.9 by the end of the year.
  • There is consens about having milestones.
  • Matti suggests to have a fixed release schedule regardless of how many changes are in the release.

Binary verification

  • Tim, Florian suggest to do a verification when creating the stub loader
  • Antonio suggests to also allow mixed-mode binaries. But there would probably not be much difference to a C API extension.
  • Antonio thinks that we should do that at the compilation level.
  • Python.h has a guard against multiple inclusions; we could just define that
  • We can have both: the compilation guard and the binary verfication.

Multi-phase module init

  • Tim asks: how is multi-phase module init related to subinterpreters?
  • Petr: all modules should be sub-interpreter ready. The multi-phase module init basically asks for that. Current doc says: you shouldn't use multi-phase module init if not subinterpreter ready.
    • What does subinterpreter ready mean? Modules might be initialized multiple times. Simplest way: just have a global flag that already initialized.
  • Stepan: it looks like that multi-phase module init and subinterpreters are still orthogonal. We could just have multi-phase module init and have a flags for subinterpreter-readiness
  • Petr strongly recommends to use multi-phase module init from the beginning

Minimal context for tp_traverse, tp_destroy

  • Antonio thinks it opens a can of worms
  • Q: What is PyMem_Malloc/Free actually good for? A: This is for memory allocation tracking. Victor should know more about it. CPython's debug build is doing some memory tracking and padding which allows to detect use-after-free and out-of-bounds checking.
  • Are those functions meant to be used by extensions authors? Petr would recommend using it because it since it allows easier debugging.
  • Numpy also uses those functions in some places (we don't remember exactly where).
  • Antonio would be fine to have some separate context like HPyDestroyContext.
  • Apart memory management, PyPy will have problems to support anything like HPyField_Load in such a context.
Clone this wiki locally