Can the pt3d vectors be exposed to the Python API?

[Helveg]

Currently I think there's only the x3d(n), y3d(n), ... functions to collect single values about the n-th 3d point. This implementation suggests that there are underlying vectors for x, y, z, diam, arc. Could these hypothesized vectors be exposed in their entirety if they exist?

Currently I solve the problem like this:

def vectorize(f):
    def sink():
        i = 0
        while True:
            try:
                yield f(i)
            except Exception:
                break
            i += 1

    return np.fromiter(sink(), dtype=float)

x = vectorize(cell.branch[n].x3d)

But if these vectors exist it's a bit of a waste to call a function to turn them into elements that then get stored into a vector :) I think numpy can construct arrays rapidly around contingent C arrays

[ramcdougal]

In principle, I love this idea.

In practice, I see two challenges:

Minor: Internally it's an array of structs (points are stored together not x-values), see for example how we get x3d values in src/nrnpython/nrnpy_nrn.cpp#L679. This, I think is not a huge deal as numpy is happy with arrays that have strides.

Major: It would either still have to be a copy anyways or we'd have to check the shape at every advance to see if anything has changed. Changing the 3D points potentially changes the section length and diameters, and thus requires recalculating a number of things.

Out of curiosity, what are the advantages of your approach as opposed to

x = [cell.branch[n].x3d(i) for i in range(cell.branch[n].n3d()]

Thoughts?

[Helveg]

I would need only read-only access, but without making a copy there would be nothing stopping numpy from writing to the original data I guess, is that the concern?

To answer your curiosity, I didn't know if n3d was O(n) or O(1) and it saves me a call to NEURON per Section (again unsure how much overhead this incurs). I'm also hoping that using np.fromiter saves me constructing an intermediate list that I won't use but x = np.fromiter(cell.branch[n].x3d(i) for i in range(cell.branch[n].n3d())) would do that as well.

[ramcdougal]

It looks like there's a numpy WRITEABLE flag that can be used to make an array read-only. (Changing the data would be bad because derived values are cached for faster simulation.) (Edit: it looks like this flag can be flipped from Python via e.g. x.setflags(write=True) making it not particularly robust.)

There's another consideration: if we don't make a copy, there's nothing stopping NEURON from changing the memory location of the pt3d array (e.g. if there are more points added after you grab the vector), and this is an easy way of getting a segfault or garbage data.

Just for the record in case anybody stumbles across this in the future, sec.n3d() is O(1). Also I corrected a copy-paste error in my earlier post in the call to n3d.