[fix] [ml] Fix cursor metadata compatability issue when switching the config unackedRangesOpenCacheSetEnabled

[poorbarcode]

Motivation

• [fix][broker] Support large number of unack message store for cursor recovery #9292 introduced a new serialization about cursor metadata, but it will be lost. information when switching the config unackedRangesOpenCacheSetEnabled. You can reproduce the issue with the new test testUnackmessagesAndRecoveryCompatibility
• [DISCUSS] Add an optional config to disable the feature that compresses cursor metadata, which was contributed by #9292

Modifications

• Fix the issue
• Add an optional config managedLedgerPersistIndividualAckAsLongArray to enable or disable the feature that [fix][broker] Support large number of unack message store for cursor recovery #9292 introduced, and set the default value to false.

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

Matching PR in forked repository

PR in forked repository: x

[poorbarcode]

/pulsarbot rerun-failure-checks

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 56.66667% with 13 lines in your changes missing coverage. Please review.

Project coverage is 74.13%. Comparing base (bbc6224) to head (f2a2441).
Report is 815 commits behind head on master.

Files with missing lines	Patch %	Lines
...che/bookkeeper/mledger/impl/ManagedCursorImpl.java	31.57%	13 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #23759      +/-   ##
============================================
+ Coverage     73.57%   74.13%   +0.56%     
+ Complexity    32624    31763     -861     
============================================
  Files          1877     1853      -24     
  Lines        139502   143391    +3889     
  Branches      15299    16279     +980     
============================================
+ Hits         102638   106308    +3670     
+ Misses        28908    28705     -203     
- Partials       7956     8378     +422     

Flag	Coverage Δ
inttests	26.72% <43.33%> (+2.13%)	⬆️
systests	23.14% <43.33%> (-1.18%)	⬇️
unittests	73.67% <56.66%> (+0.82%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...apache/bookkeeper/mledger/ManagedLedgerConfig.java	96.42% <100.00%> (+0.13%)	⬆️
...pache/bookkeeper/mledger/impl/RangeSetWrapper.java	81.25% <100.00%> (-13.09%)	⬇️
...org/apache/pulsar/broker/ServiceConfiguration.java	98.14% <100.00%> (-1.25%)	⬇️
...rg/apache/pulsar/broker/service/BrokerService.java	84.87% <100.00%> (+4.09%)	⬆️
...che/bookkeeper/mledger/impl/ManagedCursorImpl.java	79.30% <31.57%> (+<0.01%)	⬆️

... and 1017 files with indirect coverage changes

[codelipenghui]

It's better to move to the internal of the getPreviousPosition() method.

[poorbarcode]

It's better to move to the internal of the getPreviousPosition() method.

getPreviousPosition returns a meaningful position, in other words, it returns a position that exactly exists. And other methods rely on this feature, such as https://github.com/apache/pulsar/blob/master/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java#L3621-L3623. Only cursor individual ack range does not allow crossing ledgers so far.

if (specifiedLac.getEntryId() < 0) {
     // Calculate the meaningful LAC.
     Position actLac = getPreviousPosition(specifiedLac);
    ...
}

[lhotari]

Does change the existing Pulsar 4.0 behavior when this defaults to false?

[poorbarcode]

Yes, will rollback the default behavior to 3.0.x, in other words, removed the default behavior changes that introduced by #9292