Made PendingAddOp thread safe

[merlimat]

Motivation

PendingAddOp is not thread-safe and the BK client instead follows a pattern of "jumping" to a particular thread for both the requests and the callback path.

This is in practice much more expensive than acquiring a mutex since in most cases the caller will have already serialized the calls to ledger.append().

With this change, we can avoid 2 context switches and the very big contention on the executor queues used between the threads.

[codecov-commenter]

Codecov Report

Merging #3784 (2985456) into master (2381d9b) will increase coverage by 0.10%.
The diff coverage is 77.77%.

@@             Coverage Diff              @@
##             master    #3784      +/-   ##
============================================
+ Coverage     68.08%   68.18%   +0.10%     
- Complexity     6712     6723      +11     
============================================
  Files           473      473              
  Lines         40952    40930      -22     
  Branches       5236     5235       -1     
============================================
+ Hits          27882    27909      +27     
+ Misses        10808    10762      -46     
+ Partials       2262     2259       -3     

Flag	Coverage Δ
bookie	39.71% <33.33%> (+<0.01%)	⬆️
client	44.27% <77.77%> (+0.01%)	⬆️
remaining	29.59% <33.33%> (-0.03%)	⬇️
replication	41.31% <33.33%> (-0.01%)	⬇️
tls	21.03% <22.22%> (+0.03%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...ava/org/apache/bookkeeper/client/PendingAddOp.java	88.83% <60.00%> (+1.09%)	⬆️
...ava/org/apache/bookkeeper/client/LedgerHandle.java	80.05% <100.00%> (-0.12%)	⬇️
.../org/apache/bookkeeper/client/LedgerHandleAdv.java	80.59% <100.00%> (+3.13%)	⬆️
.../org/apache/bookkeeper/proto/BookieClientImpl.java	64.36% <100.00%> (+0.30%)	⬆️
...pache/bookkeeper/proto/PerChannelBookieClient.java	79.86% <100.00%> (+0.38%)	⬆️
...g/apache/bookkeeper/client/MetadataUpdateLoop.java	80.00% <0.00%> (-7.15%)	⬇️
...apache/bookkeeper/proto/WriteEntryProcessorV3.java	82.79% <0.00%> (-4.31%)	⬇️
...keeper/meta/AbstractHierarchicalLedgerManager.java	72.97% <0.00%> (-4.06%)	⬇️
...g/apache/bookkeeper/bookie/DefaultEntryLogger.java	79.93% <0.00%> (-1.20%)	⬇️
...ain/java/org/apache/bookkeeper/bookie/Journal.java	80.56% <0.00%> (-1.16%)	⬇️
... and 25 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[hangc0276]

Good job!

[hangc0276]

Hi @merlimat, I found one test failed related to this PR. #3805