[CELEBORN-1467] celeborn.worker.storage.dirs should support soft link

[leixm]

What changes were proposed in this pull request?

celeborn.worker.storage.dirs supports soft link

Why are the changes needed?

NPE will be thrown when StorageManager.createWriter if celeborn.worker.storage.dirs contains soft link.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Existing uts.

[leixm]

We have a soft link

/mnt/ssd/0 -> /mnt/ssd/3

worker conf

celeborn.worker.storage.dirs                    /mnt/ssd/0,/mnt/ssd/1,/mnt/ssd/2

df -h

/dev/sda3       1.1T  568G  486G   54% /
/dev/sdb        3.5T   92G  3.2T    3% /mnt/ssd/1
/dev/sdc        3.5T  110G  3.2T    4% /mnt/ssd/2
/dev/sdd        3.5T   88G  3.2T    3% /mnt/ssd/3

ls /sys/block/

sda  sdb  sdc  sdd

StorageManager#diskInfos

/mnt/ssd/3=DiskInfo(maxSlots: 0, committed shuffles 0, running applications 0, shuffleAllocations: Map(), mountPoint: /mnt/ssd/3, usableSpace: 0.0 B, avgFlushTime: 0 ns, avgFetchTime: 0 ns, activeSlots: 0, storageType: HDD) status: HEALTHY dirs /mnt/ssd/0/celeborn-worker/shuffle_data

We should keep dirs and mount point consistent, so above dirs should be /mnt/ssd/3/celeborn-worker/shuffle_data

[AngersZhuuuu]

ping @waitinfuture @FMX

[FMX]

LGTM. Thanks. After some tests, this PR is correct. Merged into main(v0.6.0), branch-0.5(v0.5.0), branch-0.4(v0.4.2).

[RexXiong]

LGTM