Bug: SteamingDataset Processor data config mismatch

[cauyxy]

Bug description

When the number of processes is less than the number of files, some processes do not read or write data. At this time, the data_format written to process_idx_index.json is inconsistent, which leads to the failure of synthesizing the final index.json.

What version are you seeing the problem on?

v2.2

How to reproduce the bug

No response

Error messages and logs

Environment

Current environment

#- Lightning Component (e.g. Trainer, LightningModule, LightningApp, LightningWork, LightningFlow):
#- PyTorch Lightning Version (e.g., 1.5.0):
#- Lightning App Version (e.g., 0.5.2):
#- PyTorch Version (e.g., 2.0):
#- Python version (e.g., 3.9):
#- OS (e.g., Linux):
#- CUDA/cuDNN version:
#- GPU models and configuration:
#- How you installed Lightning(`conda`, `pip`, source):
#- Running environment of LightningApp (e.g. local, cloud):

More info

No response