Skip to content

Commit

Permalink
Small doc fixes
Browse files Browse the repository at this point in the history
  • Loading branch information
SvenDS9 committed Mar 5, 2023
1 parent 03379e6 commit e85ad28
Showing 1 changed file with 5 additions and 4 deletions.
9 changes: 5 additions & 4 deletions torchdata/datapipes/iter/transform/callable.py
Original file line number Diff line number Diff line change
Expand Up @@ -619,7 +619,7 @@ class ThreadPoolMapperIterDataPipe(IterDataPipe[T_co]):
- Key is used for dict. New key is acceptable.
scheduled_tasks: How many tasks will be scheduled at any given time (Default value: 64)
max_workers: Maximum number of threads to execute function calls. (Default value: None)
max_workers: Maximum number of threads to execute function calls
**threadpool_kwargs: additional arguments to be given to the ``ThreadPoolExecutor``
Note:
Expand All @@ -628,7 +628,8 @@ class ThreadPoolMapperIterDataPipe(IterDataPipe[T_co]):
Note:
For optimal use of all threads, we recommend ``scheduled_tasks`` > ``max_workers``. High value of ``scheduled_tasks``
might lead to long waiting period until the first element is yielded as tasks are executed out of order.
might lead to long waiting period until the first element is yielded as ``next`` is called
``scheduled_tasks`` many times on ``source_datapipe`` before yielding.
Example:
Expand Down Expand Up @@ -683,7 +684,7 @@ def mul_ten(x):

def __init__(
self,
datapipe: IterDataPipe,
source_datapipe: IterDataPipe,
fn: Callable,
input_col=None,
output_col=None,
Expand All @@ -692,7 +693,7 @@ def __init__(
**threadpool_kwargs,
) -> None:
super().__init__()
self.datapipe = datapipe
self.datapipe = source_datapipe

_check_unpickable_fn(fn)
self.fn = fn # type: ignore[assignment]
Expand Down

0 comments on commit e85ad28

Please sign in to comment.