Feature: add rkyv support

[df-oss]

Checklist

• Updated guide with pertinent info (may not always apply).
• Squash down commits to one or two logical commits which clearly describe the work you've done.
• Unittest is a friend:)

• Fix rkyv support #316
• No support for errors, but that's not necessary according to discussion in rkyv support #316 (comment)
• Adding rkyv for errors would require first adding rkyv to AnyError

---

This change is 

[drmingdrmer]

@df-oss

The tests for rkyv includes:

• a storage implementation, e.g., another store in openraft/stores/, that ensures storage layer works as expected;
  The storage implementation should pass all tests defined in openraft::testing::suite:

  pub fn test_rocks_store() -> Result<(), StorageError<RocksNodeId>> {
    Suite::test_all(RocksBuilder {})?;
    Ok(())
  }

• and an example app that ensures network implementation works as expected, similar to examples/raft-kv-memstore.

https://github.com/datafuselabs/openraft/blob/79588014ab9effc1ac8da5dafbdcef4c51283f4c/examples/raft-kv-memstore/tests/cluster/test_cluster.rs#L171-L177

[drmingdrmer]

PR summary(by Claude):

Accept changes to Cargo.toml and enable rkyv serialization / deserialization support on schema types

[df-oss]

I've looked into this issue some more after taking a break from it due to several ICEs encountered during development :-( Here are my findings so far: * RaftMetrics `running_state` field uses `Fatal` which implies that several errors need to implement the rkyv traits which includes the following two types: * `rkyv` traits are not implemented for `core::ops::range`. I've created an issue rkyv/rkyv#404. * AnyError would require rkyv support (via a feature flag). This is non-trivial due to it being self-referencial. I think I've solved it but would like to contact the `rkyv` maintainer to look at a PR once I've got a complete example working. * I don't agree with the comment in #316 (comment)

But AFAIK, only types that need to be persisted on disk need rkyv support. Errors in openraft are only used for transport. Does it have to be encoded in the rkyv format for transport? E.g., for `send_append_entries():

Due to RaftMetrics and Forward-ing requests requiring rkyv serialization also for errors when used in transport. Maybe the tracking issue #880 should be updated to mention/include rkyv support for transport and/or storage and not just storage. I'll continue to work on a complete example with rkyv used as transport over tonic gRPC.

…

On Fri, Jun 30, 2023 at 12:24 PM 张炎泼 ***@***.***> wrote: PR summary(by Claude): Accept changes to Cargo.toml and enable rkyv serialization / deserialization support on schema types — Reply to this email directly, view it on GitHub <#879 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AVZNEQI775PJGK52LCENT53XN2SMHANCNFSM6AAAAAAZVFHKJU> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[drmingdrmer]

• RaftMetrics running_state field uses Fatal which implies that several errors need to implement the rkyv traits which includes the following two types:
  rkyv traits are not implemented for core::ops::range. I've created an issue rkyv/rkyv#404.

Where is core::ops::range used in error types? I can not remember. Please give a link to the code where it is used.

IMHO, errors and RaftMetrics can be serialized in another way other than rkyv. Why do you have to serialize them in rkyv?

Maybe the tracking issue #880
should be updated to mention/include rkyv support for transport and/or
storage and not just storage.

Agree. These three pairs of RPC struct also needs rkyv support:

• AppendEntriesRequest
• AppendEntriesResponse
• VoteRequest
• VoteResponse
• InstallSnapshotRequest
• InstallSnapshotResponse