Numpy Array Support and Work Group Configuration in Python Kompute

[jomiq]

Hello. For the last few days I've been experimenting with porting some pyopencl code to kompute. I'm posting this here mostly to get a sense of the general state of the vulkan-kompute Python implementation, and to find out where effort is needed and discuss the overall goals of the project.

As I understand buffers can only be constructed from a vector of floats, but there is functionality to read back directly to a numpy.Array. Is tight integration with numpy a goal of the project? Is support for other math libraries under consideration?

What is the envisioned method to handle alignment and buffers of composite data types? lava has a neat trick that gets the buffer directly from shader bytecode, but I'm not sure how it works in all situations. pyopencl has functions to convert numpy datatypes to aligned C equivalents, and a mechanism to generate C struct definitions that can be added to the shader source before compiling. I prefer the latter method.

As a final note, I have not found a way to specify the dimensions of a kernel explicitly, am I missing something?

[axsaucedo]

@jomiq great questions! Absolutely fantastic to hear this, we woudl be very keen to explore how to make it work on your efforts porting pyopencl code to kompute. In regards to your questions, coincidentally a lot has been addressed in the 0.5.2 build (currently not released yet but you can install from master). Namely:

• We have adopted Numpy.Array as first-class citizen via https://github.com/EthicalML/vulkan-kompute/pull/109/files
  • This actually performs a very efficient copy by passing the underlying C buffer
• Python now exposes workgroups / kernel dim via Workgroups for Python #116

I am certainly keen to support further types of buffers for storage - we currently have #5 and #99 to encompass this. Having said this, we could open another issue to also encompass composite types. Having said that, so far I haven't really found tangible use-cases where this would be a hard requirement, as most compute use-cases can be addressed via buffer (eg array data structures) and work groups (to simplify multi-dimensional arrays).

Do you have examples where composite data types would be a requirement? I would certainly be keen to explore this further with an example that could extend the capabilities and functionality of the Kompute package.

Thank you @jomiq looking forward to hear your thoughts.

[jomiq]

Python now exposes workgroups / kernel dim via #116

Nice work, makes a lot more sense now. 👍

Having said that, so far I haven't really found tangible use-cases where this would be a hard requirement

I mostly agree, but one use case would be to integrate with code written in terms of some struct. Doing the conversion at some later stage could be inefficient, and rewriting the rest of the code may be infeasible. My kernel is computing closest pairs between two sets of points in 3D space. I return the results as

typedef struct {
  float4 v;  // unit vector and distance
  int i;     // index
} res_type;

My case is absolutely not a hard requirement, it just makes code more concise and manageable. At the very least being able to send/retrieve something like a vector of float vectors would be nice. That is really not that different from passing around vectors of arbritrary structs, in a C sense, but I confess I don't know Vulkan enough to tell how hard it would be to actually implement. Maybe image datatypes can be used for many of these types of things but it feels kludgy. And eventually some bastard will show up with a bucket of 5D vectors...

[axsaucedo]

makes a lot more sense now.

Great

I mostly agree, but one use case would be to integrate with code written in terms of some struct.

I definitely see what you mean. So far I have been thinking to implement it, however when I start exploring further it does seem that it does make more sense to go towards the numpy (and matrix) capability specialisation route, as the objective of Kompute is primarily to optimize towards scientific type applications. I definitely see the point that even for scientific applications having more alignment with structs could make sense, but for example when dealing with numpy as a core foundation, the abstractions normally come at a higher level once you build tools like pandas on top. This is more or less the direction I see, which is also why I am more inclined towards investing the initial bulk of time towards supporting further types beyond floats (ints, uint, etc) as well as potentially multidimensional storages like image2D / image3D (but even then having a single array with strides may make mores sense as this is how numpy is implemented under the hood).

I think this prioritisation will certainly be guided based on demand from the user-base, ensuring the higher priority blockers are addressed, and then identifying the bigger bottlenecks in development workflow.

[jomiq]

Ok. Thanks for clarifying these things and I definitely agree with your priorities. Just wanted to share my thoughts as these decisions will probably have lots of impact down the line. Looking forward to test all this stuff!

[axsaucedo]

Your thoughts are very much appreciated! This has made me think about the priorities around these components. I will be looking at the roadmap this week so will certianly take these points into consideration. Thank you @jomiq looking forward to hear your thoughts once you test all the functionality.