Some issues with the behavior of the caption_column in the Parquet Dataloader.

[Jannchie]

I am using the Parquet Dataloader to load a large amount of data. I think this feature is really cool, but there are some aspects I don’t understand...

1. Since the statement if not caption: is used to check for emptiness, it seems the caption cannot be an empty string. Perhaps having an empty string as a caption might also be a reasonable requirement?

2. I’ve noticed that the value of the caption_column field can be an array. What happens if a dataset has two caption fields?

My training dataset uses tags, and I prefer using an array to store them, as this better utilizes the advantages of Parquet, resulting in smaller files. It seems that the values in the caption_column do not support arrays.

I can simply add code in the Loader to convert captions that are np.array into strings, but it might be better if arrays were supported, and even better if the array order could be partially shuffled during training, similar to sd-scripts.

[bghira]

well, an array of captions isnt going to behave like tags. they each will be their own caption. i despise tag based prompts really..

[Jannchie]

How does a set of captions work?
Please forgive me for not looking closely at the source code. If there is only one image in the dataset with two captions, would it be considered as two same images in the dataset, each with a different caption?

[bghira]

same image will get random chance of one of the captions. to sample all captions you would probably need to do N number of epochs/repeats.