Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix type annotations in datalad_next/itertools/itemize.py #550

Merged
merged 2 commits into from
Nov 28, 2023
Merged

Fix type annotations in datalad_next/itertools/itemize.py #550

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6d578e9
fix type annotations in itemize.py
christian-monch Nov 27, 2023
70225c4
adjust doc-strings to type annotations
christian-monch Nov 27, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 17 additions & 13 deletions datalad_next/itertools/itemize.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,18 +5,22 @@
from typing import (
Generator,
Iterable,
TypeVar,
)


__all__ = ['itemize']


T = TypeVar('T', str, bytes, bytearray)


def itemize(
iterable: Iterable[bytes | str],
sep: str | bytes | None,
iterable: Iterable[T],
sep: T | None,
*,
keep_ends: bool = False,
) -> Generator[bytes | str, None, None]:
) -> Generator[T, None, None]:
"""Yields complete items (only), assembled from an iterable

This function consumes chunks from an iterable and yields items defined by
Expand All @@ -27,8 +31,8 @@ def itemize(

Items are defined by a separator given via ``sep``. If ``sep`` is ``None``,
the line-separators built into ``str.splitlines()`` are used, and each
yielded item will be a line. If ``sep`` is not `None`, its type must match
the type of the elements in ``iterable``.
yielded item will be a line. If ``sep`` is not `None`, its type must be
compatible to the type of the elements in ``iterable``.

A separator could, for example, be ``b'\\n'``, in which case the items
would be terminated by Unix line-endings, i.e. each yielded item is a
Expand All @@ -51,9 +55,9 @@ def itemize(

Parameters
----------
iterable: Iterable[bytes | str]
iterable: Iterable[str | bytes | bytearray]
The iterable that yields the input data
sep: str | bytes | None
sep: str | bytes | bytearray | None
The separator that defines items. If ``None``, the items are
determined by the line-separators that are built into
``str.splitlines()``.
Expand All @@ -64,7 +68,7 @@ def itemize(

Yields
------
bytes | str
str | bytes | bytearray
The items determined from the input iterable. The type of the yielded
items depends on the type of the first element in ``iterable``.

Expand Down Expand Up @@ -97,10 +101,10 @@ def itemize(
)


def _split_items_with_separator(iterable: Iterable[bytes | str],
sep: str | bytes,
def _split_items_with_separator(iterable: Iterable[T],
sep: T,
keep_ends: bool = False,
) -> Generator[bytes | str, None, None]:
) -> Generator[T, None, None]:
assembled = None
for chunk in iterable:
if not assembled:
Expand All @@ -126,9 +130,9 @@ def _split_items_with_separator(iterable: Iterable[bytes | str],
yield assembled


def _split_lines(iterable: Iterable[bytes | str],
def _split_lines(iterable: Iterable[T],
keep_ends: bool = False,
) -> Generator[bytes | str, None, None]:
) -> Generator[T, None, None]:
assembled = None
for chunk in iterable:
if not assembled:
Expand Down
Loading