[CELEBORN-1467] celeborn.worker.storage.dirs should support soft link

### What changes were proposed in this pull request? `celeborn.worker.storage.dirs` supports soft link ### Why are the changes needed? NPE will be thrown when StorageManager.createWriter if `celeborn.worker.storage.dirs` contains soft link. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Existing uts. Closes apache#2576 from leixm/CELEBORN-1467. Authored-by: Xianming Lei <[email protected]> Signed-off-by: mingji <[email protected]>
FMX · Jun 19, 2024 · 759638e · 759638e
1 parent 627ee8c
commit 759638e
Showing 1 changed file with 3 additions and 1 deletion.
diff --git a/common/src/main/scala/org/apache/celeborn/common/meta/DeviceInfo.scala b/common/src/main/scala/org/apache/celeborn/common/meta/DeviceInfo.scala
@@ -288,7 +288,9 @@ object DeviceInfo {
           val deviceInfo = mountPointToDeviceInfo.get(mountPoint)
           val diskInfo = new DiskInfo(
             mountPoint,
-            dirs.map(_._1).toList,
+            dirs.map { workingDir =>
+              new File(workingDir._1.getCanonicalPath)
+            }.toList,
             deviceInfo,
             conf)
           val (_, maxUsableSpace, threadCount, storageType) = dirs(0)