Lock objects with exported buffers

[QuLogic]

pybind11 provides a reasonable interface to the buffer protocol using the buffer_protocol tag and def_buffer. However, it does not, AFAICT, provide a way to lock a mutable object for the duration of the buffer's existence.

For example, if you use class from the docs, and add a mutating method resize:

class Matrix {
public:
    Matrix(size_t rows, size_t cols) : m_rows(rows), m_cols(cols) {
        m_data = new float[rows*cols];
    }
    float *data() { return m_data; }
    size_t rows() const { return m_rows; }
    size_t cols() const { return m_cols; }
    void resize(size_t rows, size_t cols) {
        m_rows = rows;
        m_cols = cols;
        delete[] m_data;
        m_data = new float[rows*cols];
    }
private:
    size_t m_rows, m_cols;
    float *m_data;
};

then if you create a buffer and resize it:

mat = Matrix(2, 2)
view = memoryview(m)
mat.resize(10, 10)

the view will now be pointing to invalid memory:

view.tobytes()  # <- will crash or read garbage, and raises an error in valgrind.

In the standard library, the closest analogue is array, and if you try to modify it, it will fail instead:

>>> from array import array
>>> a = array('d', [1, 2, 3])
>>> mv = memoryview(a)
>>> a.extend([4])
Traceback (most recent call last):
  File "<python-input-5>", line 1, in <module>
    a.extend([4])
    ~~~~~~~~^^^^^
BufferError: cannot resize an array that is exporting buffers

In the Python standard library, they simply increment a counter in bf_getbuffer and decrement the counter in bf_releasebuffer.

However, since def_buffer returns static data in the buffer_info struct, and there is no (AFAICT) analogue to bf_releasebuffer, I don't see how one can do the same in pybind11.

I would suggest that pybind11 handles this buffer counting itself, and then add a tag for functions that might modify the buffer structure. So for the above example, if you did something like:

py::class_<Matrix>(m, "Matrix", py::buffer_protocol())
    .def(py::init<size_t, size_t>())
    .def_buffer(....)
    .def("resize", &Matrix::resize, py::lock_buffer())  # Name to be determined.

then pybind11 could check on calls to Matrix.resize that the buffer count is 0 and if not, raise an error like array.extend does.