[PIP 96] Add message payload processor for Pulsar client

[BewareMyPower]

Master Issue: #12087

Motivation

See #12087 for details.

Modifications

• Add MessagePayloadContext, MessagePayload, MessagePayloadProcessor interfaces following the PIP 96.
• Add MessagePayloadImpl, which uses ByteBuf as the backed buffer, as the default implementation of MessagePayload.
• Add MessagePayloadFactory, the DEFAULT field is an instance that can wrap a byte array or NIO buffer to a MessagePayloadImpl object.
• Add MessagePayloadUtils class to convert MessagePayload to ByteBuf. No bytes copy happens if the MessagePayload is MessagePayloadImpl.
• Refactor ConsumerImpl to provide methods to create a single message in batch or a normal message based on metadata and payload. Some repeated code is also removed.
• Add MessagePayloadContextImpl as the default implementation of MessagePayloadContext. It uses methods from ConsumerImpl to create a Message based on a MessagePayload.
• Support configuring MessagePayloadProcessor in ConsumerBuilder.
• Add consumeMessagesFromConverter method to ConsumerImpl when a payload converter is configured.

Verifying this change

• Make sure that the change passes the CI checks.

• Add MessagePayloadTest to test converting MessagePayload to ByteBuf, it covers the cases for both the default MessagePayloadImpl and another implementation based on a NIO ByteBuffer.

• Add MessagePayloadProcessorTest, which includes two tests:

  • testDefaultProcessor: Test the default processor could work. In addition, it checks the refCnt to verify after MessagePayloadContext creates a Message<T>, the ByteBuf will be released.
  • testCustomProcessor: Test the processor for custom message format that represents a batched message.
    • CustomBatchFormat: Define the custom message format.
    • CustomBatchProducer: A producer that writes the custom formatted message to bookie directly. When the batch is full, send the batch.

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): (no)
• The public API: (yes)
• The schema: (no)
• The default values of configurations: (no)
• The wire protocol: (no)
• The rest endpoints: (no)
• The admin cli options: (no)
• Anything that affects deployment: (no)

[BewareMyPower]

Now this PR is ready for review. And the PR description was updated as well.

Currently there's two +1 in vote email, you can take a look first, @eolivelli @codelipenghui

[BewareMyPower]

@eolivelli @315157973 @codelipenghui @hangc0276 @merlimat @rdhabalia PTAL when you have time since Pulsar 2.9.0 release cut is coming.

[eolivelli]

Great work

I left some minor comments PTAL

[BewareMyPower]

Comments were addressed, PTAL again @eolivelli

[eolivelli]

Lgtm

[Anonymitaet]

Confirmed with @BewareMyPower, this PR adds docs in JavaDoc