Add support for executing OSD class methods

[phlogistonjohn]

In ceph/ceph-csi#1071 we started a side conversation about further improvements to the approach used by csi to work with omaps.

There @dillaman wrote:

As a further (future) optimization, you could eliminate the journal lock and use a RADOS class call to execute the cmpomp.cmp_set_vals method which would allow you to atomically reserve the UUID and then, combined w/ a second omap setvals call, store all the values with a single round-trip to the OSD.

and

You need to expose this API [1] in go-ceph to support calling arbitrary OSD-side cls methods. The particular cls method I was referring to is located here [2] and the blob (bufferlist) for the input parameters is here [3].

[1] https://github.com/ceph/ceph/blob/master/src/include/rados/librados.h#L3004
[2] https://github.com/ceph/ceph/blob/master/src/cls/cmpomap/client.cc#L21
[3] https://github.com/ceph/ceph/blob/master/src/cls/cmpomap/ops.h#L29

Looking at the links I see that wrapping the linked function and those similar to it (see #307 #306 #305 #304 etc) would be generally straightforward†.

The cls and method arguments are strings so that's a matter of choosing the correct inputs.

It's the last item that has induced me to file a specific issue to track the topic.
Looking at the linked code I see serialization code that appears to be C++ outside of the librados C wrapper. It looks like it relies on the encoding functions also in ceph's src/include/encoding.h.
(Is this ceph's wire format?)

If this is useful functionality for our users I am wondering how we ought to make this available to our users as, by default, Go doesn't wrap C++ and this appears outside of librados.

Options I see:

• Add additional data serialization functions to librados C layer
• Implement the encoding/decoding in pure-Go
• Somehow try to start consuming C++ outside of librados

Am I missing anything?
@dillaman I'd like your input on this, especially if I've overlooked or misunderstood how this is meant to be used. Later, we can discuss feasibility & effort of these various approaches.

There doesn't seem to be wrappers for this in pybind so I can't steal^W borrow ideas from there, at the moment. :-)

† - I want to do more proper support for write op and read op, and I started looking into it but I haven't followed up much on that. Fie on me.

[phlogistonjohn]

Poking around a bit more, and it appears this isn't in any released versions of ceph yet, not even octopus, so it would appear we'd need to wait a while for csi to be able to consume it anways.

[dillaman]

Poking around a bit more, and it appears this isn't in any released versions of ceph yet, not even octopus, so it would appear we'd need to wait a while for csi to be able to consume it anways.

Good catch -- so this is just a backlog item for the time being then.