Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[CELEBORN-875][FOLLOWUP] Enhance
DataPushQueueSuiteJ
for thread saf…
…ety and prevent `NullPointerException` ### What changes were proposed in this pull request? 1. replaced the usage of `HashMap` with `ConcurrentHashMap` for `partitionBatchIdMap` to ensure thread safety during parallel data processing 2. put the partition id and batch id into the `partitionBatchIdMap` before adding the task to prevent the possibility of a NPE ### Why are the changes needed? to fix NPE https://github.com/apache/incubator-celeborn/actions/runs/5734532048/job/15540863715?pr=1785 ``` xception in thread "DataPusher-0" java.lang.NullPointerException at org.apache.celeborn.client.write.DataPushQueueSuiteJ$1.pushData(DataPushQueueSuiteJ.java:121) at org.apache.celeborn.client.write.DataPusher$1.run(DataPusher.java:125) Error: The operation was canceled. ``` ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Pass GA Closes apache#1789 from cfmcgrady/celeborn-875-followup. Authored-by: Fu Chen <[email protected]> Signed-off-by: zky.zhoukeyong <[email protected]>
- Loading branch information