[Fix #2506] Specify multiprocessing context in DataLoader

[MengqingCao]

Fix #2506

P.S. The indentation adjustment of the code is to pass the format check of yapf.

[xingchensong]

thx!

[xingchensong]

Sorry , I have to revert this PR, I cannot laungh deepspeed engine after this PR.

[MengqingCao]

Sorry , I have to revert this PR, I cannot laungh deepspeed engine after this PR.

Hi @xingchensong, could you describe in more detail the error you get when launching deepspeed? If it is solvable, I would like to do this work and remerge this pr if possible.

[xingchensong]

Support for the KUNPENG CPU is not a top priority; you can treat it as a patch and explore DeepSpeed issue when using an Intel CPU.

[MengqingCao]

Support for the KUNPENG CPU is not a top priority; you can treat it as a patch and explore DeepSpeed issue when using an Intel CPU.

I tried to run training pipeline in aishell/whisper/run.sh with deepspeed, using mpDataLoader as a patch. Nothing went wrong with it. Maybe more details for reproducing the error you met could help.

The cpu test with:

Intel(R) Xeon(R) Gold 6151 CPU @ 3.00GHz

[xingchensong]

我在使用自己diy代码，抄本在线hotfix (wenet-e2e/WenetSpeech#54 )，deespeed会一直重复进行初始化

https://paste.ubuntu.com/p/4FmD3342Dv/

听你描述，应该去掉这段diy代码可以正常跑，没有时间探究原因，你有时间可以尝试解决下

[MengqingCao]

听你描述，应该去掉这段diy代码可以正常跑，没有时间探究原因，你有时间可以尝试解决下

ok 我尝试下