Improve Azure Event Hub scaling

[troydn]

This PR improves the Event Hub unprocessed event count calculation to prevent negative and large values.

There are two issues in the current implementation:

1. Stale partition runtime information results in very large values:
   The scaling calculation uses the SequenceNumber from the partition runtime information and the checkpoint store.
   Partition information could be stale compared to the checkpoint, for example when the partition runtime information is 10 and the checkpoint is 15. This should result in 0 and not in 9223372036854775802.

2. When the sum of all unprocessed events per partition is greater than the Int64 max value the result is negative:
   When there are a lot of unprocessed events (mostly caused by the first issue), the sum of all unprocessed events per partition is greater than the Int64 max value, resulting in a negative value.
   Using the first example: Partition 1 (9223372036854775802) + Partition 2 (100) = -9223372036854775714
   In this case the result should be the Int64 max value, which will result in the max value for lagRelatedToPartitionCount: (partitionCount * threshold).

To solve the first issue I introduced a new parameter stalePartitionInfoThreshold to configure the stale partition information threshold.
This configures the range to decide if the partition information is stale or if the Event Hub went almost through the full circular buffer.

Using the first example to explain the new implementation:

• Partition information sequence number: 10
• Checkpoint store sequence number: 15
• stalePartitionInfoThreshold: 10000
  • Results in 9223372036854765807 (Int64 max value - threshold) max unprocessed events for the partition info not to be considered stale.
• Distance between the checkpoint and the partition info: 9223372036854775802
• 9223372036854775802 > 9223372036854765807, so the partition info is stale. Return 0 as the unprocessed event count for the partition because all data is processed.

Visualization:

Checklist

• I have verified that my change is according to the deprecations & breaking changes policy
• Tests have been added
• Changelog has been updated and is aligned with our changelog requirements
• Commits are signed with Developer Certificate of Origin (DCO - learn more)

Fixes #4250

[github-actions]

Thank you for your contribution! 🙏 We will review your PR as soon as possible.

While you are waiting, make sure to:

• Add an entry in our changelog in alphabetical order and link related issue
• Update the documentation, if needed
• Add unit & e2e tests for your changes
• GitHub checks are passing
• Is the DCO check failing? Here is how you can fix DCO issues

Learn more about:

• Our contribution guide

[JorTurFer]

@tomkerkhove , could you pull a revision from any folk expert in event hub?

[zroubalik]

@tomkerkhove FYI

[JorTurFer]

I think is ready to merge. Could you solve merge conflicts please @troydn ? 🙏

[JorTurFer]

/run-e2e azure
Update: You can check the progress here