You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Doing this in C++ is easy enough, but now I would like Python code to be able to inherit from Polymorphic as well. Inheritance using “trampolines” as explained in the docs works well, but I get into trouble when adding these objects to the Wrapper, because the Python state is lost in the process.
My first unsuccessful attempt was as follows:
#include<cxxabi.h>
#include<iostream>
#include<memory>
#include<string>
#include<typeinfo>
#include<pybind11/pybind11.h>namespacepy= pybind11;
/// Just a class with a virtual method and virtual destructorstructPolymorphicBase {
virtual~PolymorphicBase() = default;
virtual std::string to_string() const = 0;
};
// Trampoline for inheriting/overriding in PythonstructPolymorphicTrampoline : PolymorphicBase {
std::string to_string() constoverride { PYBIND11_OVERRIDE_PURE(std::string, PolymorphicBase, to_string, ); }
};
// Derived class in C++structPolymorphicDerived : PolymorphicBase {
std::string to_string() constoverride { return"PolymorphicDerived"; }
};
// Some other class that stores a shared_ptr to PolymorphicBasestructWrapper {
std::shared_ptr<PolymorphicBase> ptr;
std::string to_string() const { return"Wrapper: " + ptr->to_string(); }
};
staticinline std::string demangled_type_name(const std::type_info &ti) {
int status = 0;
std::unique_ptr<char, decltype(&std::free)> cstr(abi::__cxa_demangle(ti.name(), nullptr, nullptr, &status), std::free);
return cstr.get();
}
// Returns a function that makes a Wrapper from the (polymorphic) argumenttemplate <classT>
autoWrapperConstructor() {
return [](std::shared_ptr<T> t) -> Wrapper {
std::cout << __PRETTY_FUNCTION__ << "\r\n"
<< " --> to_string: " << t->to_string() << "\r\n"
<< " --> C++ type: " << demangled_type_name(typeid(*t))
<< std::endl;
return {std::dynamic_pointer_cast<PolymorphicBase>(t)};
};
}
PYBIND11_MODULE(_core, m) {
py::class_<PolymorphicBase, std::shared_ptr<PolymorphicBase>,
PolymorphicTrampoline>(m, "PolymorphicBase")
.def(py::init())
.def("__str__", &PolymorphicBase::to_string);
py::class_<PolymorphicDerived, std::shared_ptr<PolymorphicDerived>,
PolymorphicBase>(m, "PolymorphicDerived")
.def(py::init())
.def("__str__", &PolymorphicDerived::to_string);
py::class_<Wrapper>(m, "Wrapper")
.def(py::init(WrapperConstructor<PolymorphicDerived>()))
.def(py::init(WrapperConstructor<PolymorphicTrampoline>()))
.def("__str__", &Wrapper::to_string);
}
Testing this module using the following Python script:
WrapperConstructor<PolymorphicDerived>::<lambda(std::shared_ptr<PolymorphicDerived>)>
--> to_string: PolymorphicDerived
--> C++ type: PolymorphicDerived
WrapperConstructor<PolymorphicTrampoline>::<lambda(std::shared_ptr<PolymorphicTrampoline>)>
--> to_string: PolymorphicPython 1
--> C++ type: PolymorphicTrampoline
WrapperConstructor<PolymorphicTrampoline>::<lambda(std::shared_ptr<PolymorphicTrampoline>)>
--> to_string: PolymorphicPython 2
--> C++ type: PolymorphicTrampoline
Wrapper: PolymorphicDerived
Traceback (most recent call last):
File "test.py", line 14, in <module>
print(b)
RuntimeError: Tried to call pure virtual function "PolymorphicBase::to_string"
At the moment the Wrapper is constructed, calling the virtual function works (see lines 5 and 8 in the output), but when accessing it through the Wrapper later, the virtual PolymorphicPython::to_string() function is gone.
This somewhat makes sense, since I only keep the PolymorphicTrampoline object alive, not necessarily the Python state associated with it.
Following this issue (#1049 (comment)), I replaced my WrapperConstructor function by the following:
// Returns a function that makes a copy of the (polymorphic) argument and// returns a Wrapper for ittemplate <classT>
autoWrapperConstructor() {
return [](T &t) -> Wrapper {
std::cout << __PRETTY_FUNCTION__ << "\r\n"
<< " --> to_string: " << t.to_string() << "\r\n"
<< " --> C++ type: " << demangled_type_name(typeid(t))
<< std::endl;
auto full_python_copy = std::make_shared<py::object>(py::cast(t));
auto base_copy = full_python_copy->templatecast<PolymorphicBase *>();
return {std::shared_ptr<PolymorphicBase>(full_python_copy, base_copy)};
};
}
Now for my main question: How can py::cast(t) preserve the Python state?
The argument t is simply a reference to PolymorphicTrampoline, and its dynamic type is PolymorphicTrampoline as well, so it doesn't seem like there are any pybind11 subclasses that store any extra state. So how then does py::cast manage to “find” the Python state in order to make a copy of it (or increase the ref count)?
I've tried going through the pybind11::cast documentation and source code, and stepped through it in a debugger, but it's not clear to me what's going on under the hood.
A second question: is the second version of WrapperConstructor the correct approach to do things like this, or are there better ways to save polymorphic Python objects in the wrapper?
Maybe it's possible to pass the actual Python object to WrapperConstructor (not the reference to PolymorphicTrampoline), so the extra py::cast and make_shared<py::object> can be avoided? Can this be done while still having pybind11 match the correct type? I imagine that adding a def(py::init([](py::object o) { ... })) constructor would match any Python object, not just trampolines?
Finally, I also tried compiling and using this test module using the Address Sanitizer, and it reports many memory leaks in pybind11 code (most of them in pybind11::cpp_function::strdup_guard::operator()(char const*), pybind11::cpp_function::initialize_generic(std::unique_ptr<pybind11::detail::function_record, pybind11::cpp_function::InitializingFunctionRecordDeleter>&&, char const*, std::type_info const* const*, unsigned long) and pybind11::cpp_function::make_function_record()). Is this because of an issue with my code? Are they false positives? The complete output is rather long, but I'll be happy attach it if you think this is useful.
The text was updated successfully, but these errors were encountered:
(Main question is in bold at the bottom)
I'm trying to get a struct to store (a pointer to) an instance of a polymorphic type as a member. Basically:
Doing this in C++ is easy enough, but now I would like Python code to be able to inherit from
Polymorphic
as well. Inheritance using “trampolines” as explained in the docs works well, but I get into trouble when adding these objects to theWrapper
, because the Python state is lost in the process.My first unsuccessful attempt was as follows:
Testing this module using the following Python script:
Results in the following error:
At the moment the Wrapper is constructed, calling the virtual function works (see lines 5 and 8 in the output), but when accessing it through the Wrapper later, the virtual
PolymorphicPython::to_string()
function is gone.This somewhat makes sense, since I only keep the
PolymorphicTrampoline
object alive, not necessarily the Python state associated with it.Following this issue (#1049 (comment)), I replaced my
WrapperConstructor
function by the following:Now the Python script works correctly:
Now for my main question: How can
py::cast(t)
preserve the Python state?The argument
t
is simply a reference toPolymorphicTrampoline
, and its dynamic type isPolymorphicTrampoline
as well, so it doesn't seem like there are any pybind11 subclasses that store any extra state. So how then doespy::cast
manage to “find” the Python state in order to make a copy of it (or increase the ref count)?I've tried going through the
pybind11::cast
documentation and source code, and stepped through it in a debugger, but it's not clear to me what's going on under the hood.A second question: is the second version of
WrapperConstructor
the correct approach to do things like this, or are there better ways to save polymorphic Python objects in the wrapper?Maybe it's possible to pass the actual Python object to WrapperConstructor (not the reference to PolymorphicTrampoline), so the extra
py::cast
andmake_shared<py::object>
can be avoided? Can this be done while still having pybind11 match the correct type? I imagine that adding adef(py::init([](py::object o) { ... }))
constructor would match any Python object, not just trampolines?Finally, I also tried compiling and using this test module using the Address Sanitizer, and it reports many memory leaks in pybind11 code (most of them in
pybind11::cpp_function::strdup_guard::operator()(char const*)
,pybind11::cpp_function::initialize_generic(std::unique_ptr<pybind11::detail::function_record, pybind11::cpp_function::InitializingFunctionRecordDeleter>&&, char const*, std::type_info const* const*, unsigned long)
andpybind11::cpp_function::make_function_record()
). Is this because of an issue with my code? Are they false positives? The complete output is rather long, but I'll be happy attach it if you think this is useful.The text was updated successfully, but these errors were encountered: