Add laion5b-example with Dataloader2

[SvenDS9]

This is an example that uses Datapipes to download and preprocess the laion5b-dataset (to be more precise this subset). Also uses Dataloader2 for multiprocessing.

Changes

• Load metadata from Huggingface and filter
• Load images from the urls
• access metadata of image and print out label and copyright information

Unfortunately I made a mistake while rebasing in #1017 so I had to reopen the PR.

[SvenDS9]

To use the Dataloader2 I have to use r.content instead of:

data/torchdata/datapipes/iter/load/online.py

Line 41 in a3b34a0

return url, StreamWrapper(r.raw)

This causes the following problem:
Traceback (most recent call last):
File "\lib\multiprocessing\queues.py", line 244, in _feed
obj = _ForkingPickler.dumps(obj)
File "\lib\multiprocessing\reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
TypeError: cannot pickle '_io.BufferedReader' object
r.raw doesn't serialize very well. We probably need to change this in HTTPReader in the future. It was originally introduced in #51 to allow using seek in TarArchiveLoader (#42)

[ejguan]

Thank you. LGTM

[facebook-github-bot]

@ejguan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[ejguan]

To use the Dataloader2 I have to use r.content instead of:

data/torchdata/datapipes/iter/load/online.py

Line 41 in a3b34a0

return url, StreamWrapper(r.raw)

This causes the following problem:
Traceback (most recent call last):
File "\lib\multiprocessing\queues.py", line 244, in _feed
obj = _ForkingPickler.dumps(obj)
File "\lib\multiprocessing\reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
TypeError: cannot pickle '_io.BufferedReader' object
r.raw doesn't serialize very well. We probably need to change this in HTTPReader in the future. It was originally introduced in #51 to allow using seek in TarArchiveLoader (#42)

I am not sure why r.raw is sent through mp.queue. The whole pipeline is supposed to stay in worker processes, then r.raw has been processed by load_image. The only thing being passed from worker process to main process is the batch of data, right?

[SvenDS9]

I am not sure why r.raw is sent through mp.queue. The whole pipeline is supposed to stay in worker processes, then r.raw has been processed by load_image. The only thing being passed from worker process to main process is the batch of data, right?

Previously load_image() didn't call PIL.Image.open() but just returned the result of _get_response_from_http() or None if it failed to load. Therefore the batches still contained the r.raw objects. If the batches contain r.content instead, you do not run into any issues.

So one needs to be careful when using HTTPReader with multiprocessing and process the result before passing it to another process or else one will run into issues.

[ejguan]

Previously load_image() didn't call PIL.Image.open() but just returned the result of _get_response_from_http() or None if it failed to load. Therefore the batches still contained the r.raw objects. If the batches contain r.content instead, you do not run into any issues.

I see. IIRC, r.content basically would download every byte into memory, which would eliminate this problem.

So one needs to be careful when using HTTPReader with multiprocessing and process the result before passing it to another process or else one will run into issues.

Make sense. TBH, in the majority of use cases, I would expect users run all decoding operations in worker processes. So, I would assume they don't have to pass r.raw from worker to main.

[facebook-github-bot]

@ejguan merged this pull request in 6ca4402.