EPIC: Optimize Block Stream processing

[a-saksena]

Epic Goal

Rewrite all Block Node components to accept the repeated BlockItem message data as a raw Bytes object. Add low-level data inspection to determine message types and boundaries, then pass data through the ring buffer as the same raw bytes. All services (persistence, verification, client pass-through, state management, etc...) will be rewritten to likewise avoid parsing the binary protobuf data.

Modify Receiver

Preview Give feedback

1. Modify interfaces to accept List<BlockItemUnparsed> instead of PublishStreamRequest
2. Modify "mediators" to work with Bytes
3. Modify the ring buffer interfaces
4. Modify stream publishers
5. Modify single block publishers
6. Modify persistence service

A byte-level inspector does not parse objects, all items detected or extracted are detected via byte-level manipulation and extracted as a byte array or Bytes object.

Modify Ring Buffer interface(s)

Preview Give feedback

1. Modify the Ring Buffer to manage List<BlockItemUnparsed>.
2. Modify all writers/readers to process List<BlockItemUnparsed>.

Modify Stream Publishers

Preview Give feedback

1. Modify consumers to accept BlockItem from the ring buffer as a Bytes object.
2. Modify publishers to combine a requested or configured number of BlockItem bytes into a single BlockItemSet, and wrap that in a SubscribeStreamResponse.

Modify Single Block Publishers

Preview Give feedback

1. Design the publishers to accept the block file bytes
2. Design the publishers to wrap the raw file bytes in a SingleBlockResponse for transmission to a client.

Modify Persistence Service

Preview Give feedback

1. Modify persistence implementation(s) to accept partially parsed protos.
2. Modify persistence to combine all partially parsed protos for each BlockItem starting with a BlockHeader and ending with a BlockProof into a single partially parsed protos.

Tasks

Preview Give feedback

1. Evaluate "Unparsed" schema approach #353
   Block Node Improvement +2
   
2. Repair unit tests #356
   Block Node P1 Phase 2 dependencies +4
   

[a-saksena]

@jsync-swirlds , @mattp-swirldslabs -- Please refine this Epic with task breakdown.

[mattp-swirldslabs]

All tasks have been completed/merged or moved to other epics.