metricbeat: fetch offset info from primary partiton instead of replica partition

[wangdisdu]

fetch topic offset info from primary partition, because of the follow reasons:

• 1. The offset information of primary partiton is enough for learning about topic offset;
• 2. When the replica count of partiton is 0, you can not get anything from replica partiton;
• 3. When the replica count of partiton is more than 1, you will generate replica record from replica partiton list;
• 4. You do not need to set replica_id parameter in ListOffsets request and just set it -1 appending on the Kafka protocol guide

[elasticmachine]

Can one of the admins verify this patch?

[ruflin]

@wangdisdu Thanks for starting to tackle issues in our Kafka module. It's definitively something that needs improvement. Will make sure I get soonish some time to review but also want to wait on @urso for some feedback.

[ruflin]

@urso I think this one still needs your comment.

[urso]

What's the exact purpose of this PR?

To me it seems this change removes all replica information. Replica details can be nice to have in case replicas are not in sync, but also to learn which brokers are replicas (e.g. depending on required ACK settings, a partition might become unavailable for producing events if too many replicas are unavailable).

One can use processors to drop unwanted events and fields.

[jsoriano]

On #7767 a new field kafka.partition.partition.is_leader is added to indicate if an event belongs to the primary partition or not. This could be used to easily drop events from non-primary replicas if wanted as proposed by @urso. I also think that it can be useful to collect offsets of all replicas in any case.

I am closing this issue by now as is inactive and there seems to be alternatives for the same purpouse as dropping events. Please feel free to reopen if you think otherwise.