Use of PYBIND11_EMBEDDED_MODULE prevents extensions being loaded in a standalone interpreter

[bruce-optibrium]

I had a strange problem writing a python C extension (not using pybind) which was very hard to track down. The extension was failing to load a dll.

After running it in a debugger I eventually tracked the cause to embed.h

    if (Py_IsInitialized() != 0) {
        pybind11_fail("Can't add new modules after the interpreter has been initialized");
    }

Basically any use of PYBIND11_EMBEDDED_MODULE results in std::runtime_error being thrown when a standalone python interpreter attempts to load that dll because the python interpreter is of course already initialised at this point.

It is not possible to catch this exception from without modifying the dll as it is raised during static initialisation code run when the dll is loaded.

The dll happens to use pybind embedded modules for an embedded python interpreter it uses for other purposes.
The modules work fine in this environment where the dll is already loaded.
What I want is to be able to have this dll as a dependency of my C extension but as it stands this doesn't work.

I can make my code work by replacing pybind11_fail() with a return statement here but that is obviously not a good solution.

There is register_exception_translator() but I not sure how you might use it from within python rather than in the embedded context.

So my question is, is there an alternative (short of conditional compiltion) to using PYBIND11_EMBEDDED_MODULE that would allow modules to be loaded on demand and prevent the python interpreter from picking them up when it loads and scans the offending dll?
If not would it be worth raising this as a bug/feature request?
and if that is the answer what would be the best approach to changing pybind to allow for this kind of thing?

[bruce-optibrium]

Suggested a possible solution in #5155