Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[SPARK-43378][CORE] Properly close stream objects in deserializeFromC…
…hunkedBuffer ### What changes were proposed in this pull request? Fixes a that SerializerHelper.deserializeFromChunkedBuffer does not call close on the deserialization stream. For some serializers like Kryo this creates a performance regressions as the kryo instances are not returned to the pool. ### Why are the changes needed? This causes a performance regression in Spark 3.4.0 for some workloads. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Existing tests. Closes #41049 from eejbyfeldt/SPARK-43378. Authored-by: Emil Ejbyfeldt <[email protected]> Signed-off-by: Sean Owen <[email protected]>
- Loading branch information