replace_sampler_ddp + batch_sampler #3455
Unanswered
sshleifer
asked this question in
DDP / multi-GPU / multi-node
Replies: 1 comment
-
@sshleifer
|
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Is it possible to make a distributed-friendly
batch_sampler
that gets passed toDataLoader
.I have a working single-gpu version that produces an iterator where each
.next
is list of ids, but when I pass it withI get too few batches and each batch is way to small.
Has anyone seen a working example of this?
Beta Was this translation helpful? Give feedback.
All reactions