nb::reference_internal with nb::tensor as return type

[qnzhou]

Hi Wenzel,

I am trying to expose an internal buffer owned by a C++ class as numpy array using the new nb::tensor class. However, I am not able to get nb::rv_policy::reference_internal or nb::keep_alive<0, 1> working properly. Here is a minimal example illustrating the problem:

struct A {
    std::vector<int> buffer; // The buffer managed by C++ but we want to expose it as numpy array in python
};

nb::class_<A>(m, "A")
    .def(nb::init<>())
    .def_property_readonly("buffer", [](A& A){
        size_t shape[] = {A.buffer.size()};
        return nb::tensor<nb::numpy, int, nb::shape<nb::any>>(A.buffer.data(), 1, shape);
    });  // `nb::rv_policy::reference_internal` is applied by default

In python, a memory error can be generated with the following:

def create():
    a = A()
    return a.buffer()

buffer = create() 

I thought the reference_internal return policy should keep the parent object a valid as long as there is a reference to the buffer object. However, it seems to be not the case, and address sanitizer shows that buffer points to invalid memory. Change the return policy to nb::keep_alive<0, 1> also does not work, it trigger the following error:

nanobind/src/nb_type.cpp

Lines 725 to 727 in f3044cf

	fail("nanobind::detail::keep_alive(): could not create a weak "
	"reference! Likely, the 'nurse' argument you specified is not "
	"a weak-referenceable type!");

I am a bit confused as it is mentioned in the README that "it is no longer necessary that bound types are weak-referenceable". Did I miss anything? Thanks!

[wjakob]

The nb::reference_internal feature is intended for cases when you are returning a nanobind-bound object (i.e. something created using nb::class_<>).

The nb::tensor provides an owner parameter specifically for this purpose. Could you try if this works?

[qnzhou]

Actually, from the pybind11 doc, owner = nb::cast(&a) seems to work pretty well. Please let me know if this is what you have in mind.

Thanks a lot for the help!

[wjakob]

Yeah, that should do it.

[brettc]

Sorry for jumping in, but I'm very interested in the new tensor interface in nanobind (see #42).

Adding the owner parameter here fixes the problem, but the numpy array returned is read-only. I believe these flags are set by numpy's from_dlpack in the tensor_wrap function. So I'm not sure how to return an array to python that is writeable. Have I missed something obvious here?

[qnzhou]

I have the same question in #46. I think numpy's ndarray.from_dlpack() disabled the write bit by design, so there may not be anything we can do on nanobind side.

[brettc]

ah, I did not see #46. I'll continue the conversation there.