Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support non-multiple and non-named values in Yields and Receives #322

Merged
merged 3 commits into from
Sep 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 70 additions & 2 deletions docs/reference/docstrings.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,9 +141,14 @@ The parser accepts a few options:
- `ignore_init_summary`: Ignore the first line in `__init__` methods' docstrings.
Useful when merging `__init__` docstring into class' docstrings
with mkdocstrings-python's [`merge_init_into_class`][merge_init] option. Default: false.
- `returns_multiple_items`: Parse [Returns sections](#google-section-returns) as if they contain multiple items.
- `returns_multiple_items`: Parse [Returns sections](#google-section-returns) and [Yields sections](#google-section-yields) as if they contain multiple items.
It means that continuation lines must be indented. Default: true.
- `returns_named_value`: Whether to parse `thing: Description` in [Returns sections](#google-section-returns) as a name and description,
- `returns_named_value`: Whether to parse `thing: Description` in [Returns sections](#google-section-returns) and [Yields sections](#google-section-yields) as a name and description,
rather than a type and description. When true, type must be wrapped in parentheses: `(int): Description.`.
When false, parentheses are optional but the items cannot be named: `int: Description`. Default: true.
- `receives_multiple_items`: Parse [Receives sections](#google-section-receives) as if they contain multiple items.
It means that continuation lines must be indented. Default: true.
- `receives_named_value`: Whether to parse `thing: Description` in [Receives sections](#google-section-receives) as a name and description,
rather than a type and description. When true, type must be wrapped in parentheses: `(int): Description.`.
When false, parentheses are optional but the items cannot be named: `int: Description`. Default: true.
- `returns_type_in_property_summary`: Whether to parse the return type of properties
Expand Down Expand Up @@ -580,6 +585,22 @@ def foo() -> Iterator[tuple[float, float, datetime]]:
...
```

You have to indent each continuation line when documenting yielded values,
even if there's only one value yielded:

```python
"""Foo.

Yields:
partial_result: Some partial result.
A longer description of details and other information
for this partial result.
"""
```

If you don't want to indent continuation lines for the only yielded value,
use the [`returns_multiple_items=False`](#google-options) parser option.

Type annotations can as usual be overridden using types in parentheses
in the docstring itself:

Expand All @@ -593,6 +614,22 @@ Yields:
"""
```

If you want to specify the type without a name, you still have to wrap the type in parentheses:

```python
"""Foo.

Yields:
(int): Absissa.
(int): Ordinate.
(int): Timestamp.
"""
```

If you don't want to wrap the type in parentheses,
use the [`returns_named_value=False`](#google-options) parser option.
Setting it to false will disallow specifying a name.

TIP: **Types in docstrings are resolved using the docstrings' parent scope.**
See previous tips for types in docstrings.

Expand Down Expand Up @@ -663,6 +700,22 @@ def foo() -> Generator[int, tuple[str, bool], None]:
...
```

You have to indent each continuation line when documenting received values,
even if there's only one value received:

```python
"""Foo.

Receives:
data: Input data.
A longer description of what this data actually is,
and what it isn't.
"""
```

If you don't want to indent continuation lines for the only received value,
use the [`receives_multiple_items=False`](#google-options) parser option.

Type annotations can as usual be overridden using types in parentheses
in the docstring itself:

Expand All @@ -675,6 +728,21 @@ Receives:
"""
```

If you want to specify the type without a name, you still have to wrap the type in parentheses:

```python
"""Foo.

Receives:
(ModeEnum): Some mode.
(int): Some flag.
"""
```

If you don't want to wrap the type in parentheses,
use the [`receives_named_value=False`](#google-options) parser option.
Setting it to false will disallow specifying a name.

TIP: **Types in docstrings are resolved using the docstrings' parent scope.**
See previous tips for types in docstrings.

Expand Down
204 changes: 128 additions & 76 deletions src/_griffe/docstrings/google.py
Original file line number Diff line number Diff line change
Expand Up @@ -442,6 +442,76 @@ def _read_warns_section(
return DocstringSectionWarns(warns), new_offset


def _read_block_items_maybe(
docstring: Docstring,
*,
offset: int,
multiple: bool = True,
**options: Any,
) -> _ItemsBlock:
if multiple:
return _read_block_items(docstring, offset=offset, **options)
one_block, new_offset = _read_block(docstring, offset=offset, **options)
return [(new_offset, one_block.splitlines())], new_offset


def _get_name_annotation_description(
docstring: Docstring,
line_number: int,
lines: list[str],
*,
named: bool = True,
) -> tuple[str | None, Any, str]:
if named:
match = _RE_NAME_ANNOTATION_DESCRIPTION.match(lines[0])
if not match:
docstring_warning(
docstring,
line_number,
f"Failed to get name, annotation or description from '{lines[0]}'",
)
raise ValueError
name, annotation, description = match.groups()
else:
name = None
if ":" in lines[0]:
annotation, description = lines[0].split(":", 1)
annotation = annotation.lstrip("(").rstrip(")")
else:
annotation = None
description = lines[0]
description = "\n".join([description.lstrip(), *lines[1:]]).rstrip("\n")
return name, annotation, description


def _unpack_generators(
annotation: Any,
generator_pos: int,
*,
mandatory: bool = False,
) -> Any:
if annotation.is_generator:
return annotation.slice.elements[generator_pos]
if annotation.is_iterator:
return annotation.slice
if mandatory:
raise ValueError(f"must be a Generator: {annotation!r}")
return annotation


def _maybe_destructure_annotation(
annotation: Any,
index: int,
*,
multiple: bool = True,
) -> Any:
if isinstance(annotation, ExprName):
return annotation
if multiple and annotation.is_tuple:
return annotation.slice.elements[index]
return annotation


def _read_returns_section(
docstring: Docstring,
*,
Expand All @@ -452,32 +522,23 @@ def _read_returns_section(
) -> tuple[DocstringSectionReturns | None, int]:
returns = []

if returns_multiple_items:
block, new_offset = _read_block_items(docstring, offset=offset, **options)
else:
one_block, new_offset = _read_block(docstring, offset=offset, **options)
block = [(new_offset, one_block.splitlines())]
block, new_offset = _read_block_items_maybe(
docstring,
offset=offset,
multiple=returns_multiple_items,
**options,
)

for index, (line_number, return_lines) in enumerate(block):
if returns_named_value:
match = _RE_NAME_ANNOTATION_DESCRIPTION.match(return_lines[0])
if not match:
docstring_warning(
docstring,
line_number,
f"Failed to get name, annotation or description from '{return_lines[0]}'",
)
continue
name, annotation, description = match.groups()
else:
name = None
if ":" in return_lines[0]:
annotation, description = return_lines[0].split(":", 1)
annotation = annotation.lstrip("(").rstrip(")")
else:
annotation = None
description = return_lines[0]
description = "\n".join([description.lstrip(), *return_lines[1:]]).rstrip("\n")
try:
name, annotation, description = _get_name_annotation_description(
docstring,
line_number,
return_lines,
named=returns_named_value,
)
except ValueError:
continue

if annotation:
# try to compile the annotation to transform it into an expression
Expand All @@ -491,22 +552,11 @@ def _read_returns_section(
annotation = docstring.parent.annotation # type: ignore[union-attr]
else:
raise ValueError
if len(block) > 1:
if annotation.is_tuple:
annotation = annotation.slice.elements[index]
else:
if annotation.is_iterator:
return_item = annotation.slice
elif annotation.is_generator:
return_item = annotation.slice.elements[2]
else:
raise ValueError
if isinstance(return_item, ExprName):
annotation = return_item
elif return_item.is_tuple:
annotation = return_item.slice.elements[index]
else:
annotation = return_item
annotation = _maybe_destructure_annotation(
_unpack_generators(annotation, 2),
index,
multiple=returns_multiple_items,
)

if annotation is None:
returned_value = repr(name) if name else index + 1
Expand All @@ -521,43 +571,42 @@ def _read_yields_section(
docstring: Docstring,
*,
offset: int,
returns_multiple_items: bool = True,
returns_named_value: bool = True,
**options: Any,
) -> tuple[DocstringSectionYields | None, int]:
yields = []
block, new_offset = _read_block_items(docstring, offset=offset, **options)

block, new_offset = _read_block_items_maybe(
docstring,
offset=offset,
multiple=returns_multiple_items,
**options,
)

for index, (line_number, yield_lines) in enumerate(block):
match = _RE_NAME_ANNOTATION_DESCRIPTION.match(yield_lines[0])
if not match:
docstring_warning(
try:
name, annotation, description = _get_name_annotation_description(
docstring,
line_number,
f"Failed to get name, annotation or description from '{yield_lines[0]}'",
yield_lines,
named=returns_named_value,
)
except ValueError:
continue

name, annotation, description = match.groups()
description = "\n".join([description.lstrip(), *yield_lines[1:]]).rstrip("\n")

if annotation:
# try to compile the annotation to transform it into an expression
annotation = parse_docstring_annotation(annotation, docstring)
else:
# try to retrieve the annotation from the docstring parent
with suppress(AttributeError, IndexError, KeyError, ValueError):
annotation = docstring.parent.annotation # type: ignore[union-attr]
if annotation.is_iterator:
yield_item = annotation.slice
elif annotation.is_generator:
yield_item = annotation.slice.elements[0]
else:
raise ValueError
if isinstance(yield_item, ExprName):
annotation = yield_item
elif yield_item.is_tuple:
annotation = yield_item.slice.elements[index]
else:
annotation = yield_item
annotation = _maybe_destructure_annotation(
_unpack_generators(annotation, 0, mandatory=True),
index,
multiple=returns_multiple_items,
)

if annotation is None:
yielded_value = repr(name) if name else index + 1
Expand All @@ -572,39 +621,42 @@ def _read_receives_section(
docstring: Docstring,
*,
offset: int,
receives_multiple_items: bool = True,
receives_named_value: bool = True,
**options: Any,
) -> tuple[DocstringSectionReceives | None, int]:
receives = []
block, new_offset = _read_block_items(docstring, offset=offset, **options)

block, new_offset = _read_block_items_maybe(
docstring,
offset=offset,
multiple=receives_multiple_items,
**options,
)

for index, (line_number, receive_lines) in enumerate(block):
match = _RE_NAME_ANNOTATION_DESCRIPTION.match(receive_lines[0])
if not match:
docstring_warning(
try:
name, annotation, description = _get_name_annotation_description(
docstring,
line_number,
f"Failed to get name, annotation or description from '{receive_lines[0]}'",
receive_lines,
named=receives_named_value,
)
except ValueError:
continue

name, annotation, description = match.groups()
description = "\n".join([description.lstrip(), *receive_lines[1:]]).rstrip("\n")

if annotation:
# try to compile the annotation to transform it into an expression
annotation = parse_docstring_annotation(annotation, docstring)
else:
# try to retrieve the annotation from the docstring parent
with suppress(AttributeError, KeyError):
annotation = docstring.parent.returns # type: ignore[union-attr]
if annotation.is_generator:
receives_item = annotation.slice.elements[1]
if isinstance(receives_item, ExprName):
annotation = receives_item
elif receives_item.is_tuple:
annotation = receives_item.slice.elements[index]
else:
annotation = receives_item
annotation = _maybe_destructure_annotation(
_unpack_generators(annotation, 1, mandatory=True),
index,
multiple=receives_multiple_items,
)

if annotation is None:
received_value = repr(name) if name else index + 1
Expand Down
Loading
Loading