From 13678bdc25ee6670ca54d7c7cb12d5d605256c56 Mon Sep 17 00:00:00 2001 From: Danny Yang Date: Thu, 7 Nov 2024 12:48:16 -0500 Subject: [PATCH 01/13] add content from typeshed/CONTRIBUTING.md --- docs/guides/writing_stubs.rst | 162 ++++++++++++++++++++++++++++++---- 1 file changed, 145 insertions(+), 17 deletions(-) diff --git a/docs/guides/writing_stubs.rst b/docs/guides/writing_stubs.rst index 71d13a71..5f0e5d76 100644 --- a/docs/guides/writing_stubs.rst +++ b/docs/guides/writing_stubs.rst @@ -93,6 +93,15 @@ Liskov substitutability or detecting problematic overloads. It may be instructive to examine `typeshed `__'s `setup for testing stubs `__. +To suppress type errors on stubs: +* use mypy error codes for mypy-specific `# type: ignore` annotations, + e.g. `# type: ignore[override]` for Liskov Substitution Principle violations. +* use pyright error codes for pyright-specific suppressions, + e.g. `# pyright: ignore[reportGeneralTypeIssues]`. + - pyright is configured to discard `# type: ignore` annotations. + If you need both on the same line, mypy's annotation needs to go first, + e.g. `# type: ignore[override] # pyright: ignore[reportGeneralTypeIssues]`. + .. TODO: consider adding examples and configurations for specific type checkers @@ -113,18 +122,6 @@ Stub Content This section documents best practices on what elements to include or leave out of stub files. -Modules excluded fom stubs --------------------------- - -Not all modules should be included in stubs. - -It is recommended to exclude: - -1. Implementation details, with `multiprocessing/popen_spawn_win32.py `_ as a notable example -2. Modules that are not supposed to be imported, such as ``__main__.py`` -3. Protected modules that start with a single ``_`` char. However, when needed protected modules can still be added (see :ref:`undocumented-objects` section below) -4. Tests - Public Interface ---------------- @@ -138,9 +135,17 @@ The following should always be included: * All objects included in ``__all__`` (if present). Other objects may be included if they are not prefixed with an underscore -or if they are being used in practice. (See the next section.) +or if they are being used in practice. + +Modules excluded from stubs +--------------------------- -.. _undocumented-objects: +The following should not be included in stubs: + +1. Implementation details, with `multiprocessing/popen_spawn_win32.py `_ as a notable example +2. Modules that are not supposed to be imported, such as ``__main__.py`` +3. Protected modules that start with a single ``_`` char. However, when needed protected modules can still be added (see :ref:`undocumented-objects` section below) +4. Tests Undocumented Objects -------------------- @@ -417,6 +422,28 @@ and the :ref:`best-practices`. There are a few exceptions, outlined below, that different structure of stub files into account and aim to create more concise files. +Syntax Example +-------------- + +The below is an excerpt from the types for the `datetime` module. + + MAXYEAR: int + MINYEAR: int + + class date: + def __new__(cls, year: SupportsIndex, month: SupportsIndex, day: SupportsIndex) -> Self: ... + @classmethod + def fromtimestamp(cls, timestamp: float, /) -> Self: ... + @classmethod + def today(cls) -> Self: ... + @classmethod + def fromordinal(cls, n: int, /) -> Self: ... + @property + def year(self) -> int: ... + def replace(self, year: SupportsIndex = ..., month: SupportsIndex = ..., day: SupportsIndex = ...) -> Self: ... + def ctime(self) -> str: ... + def weekday(self) -> int: ... + Maximum Line Length ------------------- @@ -448,14 +475,14 @@ No:: def time_func() -> None: ... - def date_func() -> None: ... # do no leave unnecessary empty lines + def date_func() -> None: ... # do not leave unnecessary empty lines def ip_func() -> None: ... class Foo: # leave only one empty line above x: int - class MyError(Exception): ... # leave an empty line between the classes + class MyError(Exception): ... Module Level Attributes ----------------------- @@ -575,6 +602,14 @@ No:: ... def to_int3(x: str) -> int: pass +Avoid invariant collection types (`list`, `dict`) for function parameters, +in favor of covariant types like `Mapping` or `Sequence`. + +Avoid union return types. See https://github.com/python/mypy/issues/1693 + +Use `float` instead of `int | float` for parameter annotations. +See [PEP 484](https://peps.python.org/pep-0484/#the-numeric-tower). + Language Features ----------------- @@ -604,6 +639,14 @@ No:: class OtherClass: ... +Use variable annotations instead of type comments, even for stubs that target +older versions of Python. + +Platform-dependent APIs +----------------------- + +Use platform checks like `if sys.platform == 'win32'` to denote platform-dependent APIs. + NamedTuple and TypedDict ------------------------ @@ -631,7 +674,7 @@ No:: Built-in Generics ----------------- -:pep:`585` built-in generics are supported and should be used instead +:pep:`585` built-in generics (such as `list`, `dict`, `tuple`, `set`) are supported and should be used instead of the corresponding types from ``typing``:: from collections import defaultdict @@ -654,3 +697,88 @@ all type checkers:: def foo(x: int | str) -> int | None: ... # recommended def foo(x: Union[int, str]) -> Optional[int]: ... # ok + +Using `Any` and `object` +------------------------ + +When adding type hints, avoid using the `Any` type when possible. Reserve +the use of `Any` for when: +* the correct type cannot be expressed in the current type system; and +* to avoid union returns (see above). + +Note that `Any` is not the correct type to use if you want to indicate +that some function can accept literally anything: in those cases use +`object` instead. + +When using `Any`, document the reason for using it in a comment. Ideally, +document what types could be used. + +Context Managers +---------------- + +When adding type annotations for context manager classes, annotate +the return type of `__exit__` as bool only if the context manager +sometimes suppresses exceptions -- if it sometimes returns `True` +at runtime. If the context manager never suppresses exceptions, +have the return type be either `None` or `bool | None`. If you +are not sure whether exceptions are suppressed or not or if the +context manager is meant to be subclassed, pick `bool | None`. +See https://github.com/python/mypy/issues/7214 for more details. + +`__enter__` methods and other methods that return instances of the +current class should be annotated with `typing_extensions.Self` +([example](https://github.com/python/typeshed/blob/3581846/stdlib/contextlib.pyi#L151)). + +Naming +------ + +Type variables and aliases you introduce purely for legibility reasons +should be prefixed with an underscore to make it obvious to the reader +they are not part of the stubbed API. + +A few guidelines for protocol names below. In cases that don't fall +into any of those categories, use your best judgement. + +* Use plain names for protocols that represent a clear concept + (e.g. `Iterator`, `Container`). +* Use `SupportsX` for protocols that provide callable methods (e.g. + `SupportsInt`, `SupportsRead`, `SupportsReadSeek`). +* Use `HasX` for protocols that have readable and/or writable attributes + or getter/setter methods (e.g. `HasItems`, `HasFileno`). + +`@deprecated` +------------- + +The `@typing_extensions.deprecated` decorator (`@warnings.deprecated` +since Python 3.13) can be used to mark deprecated functionality; see +[PEP 702](https://peps.python.org/pep-0702/). + +A few guidelines for how to use it: + +* In the standard library, apply the decorator only in Python versions + where an appropriate replacement for the deprecated functionality + exists. If in doubt, apply the decorator only on versions where the + functionality has been explicitly deprecated, either through runtime + warnings or in the documentation. Use `if sys.version_info` checks to + apply the decorator only to some versions. +* Keep the deprecation message concise, but try to mention the projected + version when the functionality is to be removed, and a suggested + replacement. + +Imports +------- + +Imports in stubs are considered private (not part of the exported API) +unless: +* they use the form ``from library import name as name`` (sic, using + explicit ``as`` even if the name stays the same); or +* they use the form ``from library import *`` which means all names + from that library are exported. + +Forward References +------------------ + +Stub files support forward references natively. In other words, the +order of class declarations and type aliases does not matter in +a stub file. Unlike regular Python files, you can use the name of the class within its own +body without writing it as a comment. From 9db1d7c36f9a786f3fc42274f5e8a89e24a4ff6d Mon Sep 17 00:00:00 2001 From: Danny Yang Date: Thu, 7 Nov 2024 12:54:48 -0500 Subject: [PATCH 02/13] fix build errors --- docs/guides/writing_stubs.rst | 20 ++++++++------------ 1 file changed, 8 insertions(+), 12 deletions(-) diff --git a/docs/guides/writing_stubs.rst b/docs/guides/writing_stubs.rst index 5f0e5d76..c3e94071 100644 --- a/docs/guides/writing_stubs.rst +++ b/docs/guides/writing_stubs.rst @@ -94,13 +94,9 @@ It may be instructive to examine `typeshed `setup for testing stubs `__. To suppress type errors on stubs: -* use mypy error codes for mypy-specific `# type: ignore` annotations, - e.g. `# type: ignore[override]` for Liskov Substitution Principle violations. -* use pyright error codes for pyright-specific suppressions, - e.g. `# pyright: ignore[reportGeneralTypeIssues]`. - - pyright is configured to discard `# type: ignore` annotations. - If you need both on the same line, mypy's annotation needs to go first, - e.g. `# type: ignore[override] # pyright: ignore[reportGeneralTypeIssues]`. +* use mypy error codes for mypy-specific `# type: ignore` annotations, e.g. `# type: ignore[override]` for Liskov Substitution Principle violations. +* use pyright error codes for pyright-specific suppressions, e.g. `# pyright: ignore[reportGeneralTypeIssues]`. Pyright is configured to discard `# type: ignore` annotations. +* If you need both on the same line, mypy's annotation needs to go first, e.g. `# type: ignore[override] # pyright: ignore[reportGeneralTypeIssues]`. .. TODO: consider adding examples and configurations for specific type checkers @@ -147,6 +143,8 @@ The following should not be included in stubs: 3. Protected modules that start with a single ``_`` char. However, when needed protected modules can still be added (see :ref:`undocumented-objects` section below) 4. Tests +.. _undocumented-objects: + Undocumented Objects -------------------- @@ -532,7 +530,7 @@ Yes:: class Color(Enum): # An assignment with no type annotation is a convention used to indicate - # an enum member. + # an enum member. RED = 1 No:: @@ -770,10 +768,8 @@ Imports Imports in stubs are considered private (not part of the exported API) unless: -* they use the form ``from library import name as name`` (sic, using - explicit ``as`` even if the name stays the same); or -* they use the form ``from library import *`` which means all names - from that library are exported. +* they use the form ``from library import name as name`` (sic, using explicit ``as`` even if the name stays the same); or +* they use the form ``from library import *`` which means all names from that library are exported. Forward References ------------------ From cfc2827a630973e8624235f2c3339716856c5b11 Mon Sep 17 00:00:00 2001 From: yangdanny97 Date: Fri, 6 Dec 2024 21:26:40 -0500 Subject: [PATCH 03/13] address review comments --- docs/guides/writing_stubs.rst | 46 +++++++++++------------------------ 1 file changed, 14 insertions(+), 32 deletions(-) diff --git a/docs/guides/writing_stubs.rst b/docs/guides/writing_stubs.rst index 828db6e1..cebb5222 100644 --- a/docs/guides/writing_stubs.rst +++ b/docs/guides/writing_stubs.rst @@ -93,10 +93,8 @@ Liskov substitutability or detecting problematic overloads. It may be instructive to examine `typeshed `__'s `setup for testing stubs `__. -To suppress type errors on stubs: -* use mypy error codes for mypy-specific `# type: ignore` annotations, e.g. `# type: ignore[override]` for Liskov Substitution Principle violations. -* use pyright error codes for pyright-specific suppressions, e.g. `# pyright: ignore[reportGeneralTypeIssues]`. Pyright is configured to discard `# type: ignore` annotations. -* If you need both on the same line, mypy's annotation needs to go first, e.g. `# type: ignore[override] # pyright: ignore[reportGeneralTypeIssues]`. +To suppress type errors on stubs, use a `# type: ignore` comment. Refer to the style guide for +error suppression formats specific to individual typecheckers. .. TODO: consider adding examples and configurations for specific type checkers @@ -538,7 +536,7 @@ No:: class Foo: # leave only one empty line above x: int - class MyError(Exception): ... + class MyError(Exception): ... # leave an empty line between the classes Module Level Attributes ----------------------- @@ -802,6 +800,14 @@ into any of those categories, use your best judgement. * Use `HasX` for protocols that have readable and/or writable attributes or getter/setter methods (e.g. `HasItems`, `HasFileno`). +Type Checker Error Suppression formats +-------------------------------------- + +* Use mypy error codes for mypy-specific `# type: ignore` annotations, e.g. `# type: ignore[override]` for Liskov Substitution Principle violations. +* Use pyright error codes for pyright-specific suppressions, e.g. `# pyright: ignore[reportGeneralTypeIssues]`. Pyright is configured to discard `# type: ignore` annotations. +* If you need both on the same line, mypy's annotation needs to go first, e.g. `# type: ignore[override] # pyright: ignore[reportGeneralTypeIssues]`. + + `@deprecated` ------------- @@ -809,30 +815,6 @@ The `@typing_extensions.deprecated` decorator (`@warnings.deprecated` since Python 3.13) can be used to mark deprecated functionality; see [PEP 702](https://peps.python.org/pep-0702/). -A few guidelines for how to use it: - -* In the standard library, apply the decorator only in Python versions - where an appropriate replacement for the deprecated functionality - exists. If in doubt, apply the decorator only on versions where the - functionality has been explicitly deprecated, either through runtime - warnings or in the documentation. Use `if sys.version_info` checks to - apply the decorator only to some versions. -* Keep the deprecation message concise, but try to mention the projected - version when the functionality is to be removed, and a suggested - replacement. - -Imports -------- - -Imports in stubs are considered private (not part of the exported API) -unless: -* they use the form ``from library import name as name`` (sic, using explicit ``as`` even if the name stays the same); or -* they use the form ``from library import *`` which means all names from that library are exported. - -Forward References ------------------- - -Stub files support forward references natively. In other words, the -order of class declarations and type aliases does not matter in -a stub file. Unlike regular Python files, you can use the name of the class within its own -body without writing it as a comment. +Keep the deprecation message concise, but try to mention the projected +version when the functionality is to be removed, and a suggested +replacement. From 5c33d201ee1d5c1be56dbc3271540a973186006d Mon Sep 17 00:00:00 2001 From: Danny Yang Date: Tue, 10 Dec 2024 06:57:35 -0800 Subject: [PATCH 04/13] Update docs/guides/writing_stubs.rst Co-authored-by: Rebecca Chen --- docs/guides/writing_stubs.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/guides/writing_stubs.rst b/docs/guides/writing_stubs.rst index cebb5222..cb092bb9 100644 --- a/docs/guides/writing_stubs.rst +++ b/docs/guides/writing_stubs.rst @@ -93,7 +93,7 @@ Liskov substitutability or detecting problematic overloads. It may be instructive to examine `typeshed `__'s `setup for testing stubs `__. -To suppress type errors on stubs, use a `# type: ignore` comment. Refer to the style guide for +To suppress type errors in stubs, use `# type: ignore` comments. Refer to the style guide for error suppression formats specific to individual typecheckers. .. From d61da81c687096d123d29541f9789354fb4b14fd Mon Sep 17 00:00:00 2001 From: Danny Yang Date: Tue, 10 Dec 2024 06:57:41 -0800 Subject: [PATCH 05/13] Update docs/guides/writing_stubs.rst Co-authored-by: Rebecca Chen --- docs/guides/writing_stubs.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/guides/writing_stubs.rst b/docs/guides/writing_stubs.rst index cb092bb9..60b938df 100644 --- a/docs/guides/writing_stubs.rst +++ b/docs/guides/writing_stubs.rst @@ -804,7 +804,7 @@ Type Checker Error Suppression formats -------------------------------------- * Use mypy error codes for mypy-specific `# type: ignore` annotations, e.g. `# type: ignore[override]` for Liskov Substitution Principle violations. -* Use pyright error codes for pyright-specific suppressions, e.g. `# pyright: ignore[reportGeneralTypeIssues]`. Pyright is configured to discard `# type: ignore` annotations. +* Use pyright error codes for pyright-specific suppressions, e.g. `# pyright: ignore[reportGeneralTypeIssues]`. * If you need both on the same line, mypy's annotation needs to go first, e.g. `# type: ignore[override] # pyright: ignore[reportGeneralTypeIssues]`. From f253587dad59c3f1462ceb20117ae6000d9259be Mon Sep 17 00:00:00 2001 From: yangdanny97 Date: Wed, 11 Dec 2024 08:35:23 -0800 Subject: [PATCH 06/13] add Incomplete vs Any, the Any trick sections; add reference to error suppression formats section --- docs/guides/writing_stubs.rst | 57 ++++++++++++++++++++++++++++++++--- 1 file changed, 53 insertions(+), 4 deletions(-) diff --git a/docs/guides/writing_stubs.rst b/docs/guides/writing_stubs.rst index 60b938df..fa20a76f 100644 --- a/docs/guides/writing_stubs.rst +++ b/docs/guides/writing_stubs.rst @@ -93,7 +93,7 @@ Liskov substitutability or detecting problematic overloads. It may be instructive to examine `typeshed `__'s `setup for testing stubs `__. -To suppress type errors in stubs, use `# type: ignore` comments. Refer to the style guide for +To suppress type errors in stubs, use `# type: ignore` comments. Refer to the :ref:`type-checker-error-suppression` section of the style guide for error suppression formats specific to individual typecheckers. .. @@ -213,6 +213,20 @@ to use them freely to describe simple structural types. Incomplete Stubs ---------------- +When writing new stubs, it is not necessary to fully annotate all arguments, +return types, and fields. Some items may be left unannotated or +annotated with `_typeshed.Incomplete` (`documentation `_).:: + + from _typeshed import Incomplete + + field: Incomplete # unannotated + + def foo(x): ... # unannotated argument and return type + +`Incomplete` can also be used for partially known types:: + + def foo(x: Incomplete | None = None) -> list[Incomplete]: ... + Partial stubs can be useful, especially for larger packages, but they should follow the following guidelines: @@ -220,8 +234,7 @@ follow the following guidelines: can be left unannotated. * Do not use ``Any`` to mark unannotated or partially annotated values. Leave function parameters and return values unannotated. In all other cases, use - ``_typeshed.Incomplete`` - (`documentation `_):: + ``_typeshed.Incomplete``:: from _typeshed import Incomplete @@ -254,6 +267,40 @@ annotated function ``bar()``:: def bar(x: str, y, *, z=...): ... +`Any` vs. `Incomplete` +---------------------- + +While `Incomplete` is a type alias of `Any`, they serve difference purposes: +`Incomplete` is a placeholder where a proper type might be substituted. +It's a "to do" item and should be replaced if possible. `Any` is used when +it's not possible to accurately type an item using the current type system. +It should be used sparingly. + +The `Any` trick +--------------- + +In cases where a function or method can return `None`, but where forcing the +user to explicitly check for `None` can be detrimental, use +`_typeshed.MaybeNone` (an alias to `Any`), instead of `None`. + +Consider the following (simplified) signature of `re.Match[str].group`:: + + class Match: + def group(self, group: str | int, /) -> str | MaybeNone: ... + +This avoid forcing the user to check for `None`:: + + match = re.fullmatch(r"\d+_(.*)", some_string) + assert match is not None + name_group = match.group(1) # The user knows that this will never be None + return name_group.uper() # This typo will be flagged by the type checker + +In this case, the user of `match.group()` must be prepared to handle a `str`, +but type checkers are happy with `if name_group is None` checks, because we're +saying it can also be something else than an `str`. + +This is sometimes called "the Any trick". + Attribute Access ---------------- @@ -800,7 +847,9 @@ into any of those categories, use your best judgement. * Use `HasX` for protocols that have readable and/or writable attributes or getter/setter methods (e.g. `HasItems`, `HasFileno`). -Type Checker Error Suppression formats +:: _type-checker-error-suppression: + +Type Checker Error Suppression Formats -------------------------------------- * Use mypy error codes for mypy-specific `# type: ignore` annotations, e.g. `# type: ignore[override]` for Liskov Substitution Principle violations. From 076d101dac14ef2b2973055472b8da5ec900abe8 Mon Sep 17 00:00:00 2001 From: yangdanny97 Date: Wed, 11 Dec 2024 08:37:50 -0800 Subject: [PATCH 07/13] fix ref --- docs/guides/writing_stubs.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/guides/writing_stubs.rst b/docs/guides/writing_stubs.rst index fa20a76f..4fdf7145 100644 --- a/docs/guides/writing_stubs.rst +++ b/docs/guides/writing_stubs.rst @@ -847,7 +847,7 @@ into any of those categories, use your best judgement. * Use `HasX` for protocols that have readable and/or writable attributes or getter/setter methods (e.g. `HasItems`, `HasFileno`). -:: _type-checker-error-suppression: +.. _type-checker-error-suppression: Type Checker Error Suppression Formats -------------------------------------- From 0cc549ecd7c022e3a06918e12f2e928f61166575 Mon Sep 17 00:00:00 2001 From: yangdanny97 Date: Thu, 12 Dec 2024 09:13:33 -0500 Subject: [PATCH 08/13] format and fix link --- docs/guides/writing_stubs.rst | 70 ++++++++++++++++++----------------- 1 file changed, 37 insertions(+), 33 deletions(-) diff --git a/docs/guides/writing_stubs.rst b/docs/guides/writing_stubs.rst index 4fdf7145..88fa801d 100644 --- a/docs/guides/writing_stubs.rst +++ b/docs/guides/writing_stubs.rst @@ -215,7 +215,7 @@ Incomplete Stubs When writing new stubs, it is not necessary to fully annotate all arguments, return types, and fields. Some items may be left unannotated or -annotated with `_typeshed.Incomplete` (`documentation `_).:: +annotated with ``_typeshed.Incomplete`` (`documentation `_):: from _typeshed import Incomplete @@ -223,7 +223,7 @@ annotated with `_typeshed.Incomplete` (`documentation list[Incomplete]: ... @@ -267,39 +267,16 @@ annotated function ``bar()``:: def bar(x: str, y, *, z=...): ... -`Any` vs. `Incomplete` ----------------------- +``Any`` vs. ``Incomplete`` +-------------------------- -While `Incomplete` is a type alias of `Any`, they serve difference purposes: -`Incomplete` is a placeholder where a proper type might be substituted. -It's a "to do" item and should be replaced if possible. `Any` is used when -it's not possible to accurately type an item using the current type system. -It should be used sparingly. +While ``Incomplete`` is a type alias of ``Any``, they serve difference purposes: +``Incomplete`` is a placeholder where a proper type might be substituted. +It's a "to do" item and should be replaced if possible. -The `Any` trick ---------------- - -In cases where a function or method can return `None`, but where forcing the -user to explicitly check for `None` can be detrimental, use -`_typeshed.MaybeNone` (an alias to `Any`), instead of `None`. - -Consider the following (simplified) signature of `re.Match[str].group`:: - - class Match: - def group(self, group: str | int, /) -> str | MaybeNone: ... - -This avoid forcing the user to check for `None`:: - - match = re.fullmatch(r"\d+_(.*)", some_string) - assert match is not None - name_group = match.group(1) # The user knows that this will never be None - return name_group.uper() # This typo will be flagged by the type checker - -In this case, the user of `match.group()` must be prepared to handle a `str`, -but type checkers are happy with `if name_group is None` checks, because we're -saying it can also be something else than an `str`. - -This is sometimes called "the Any trick". +``Any`` is used when it's not possible to accurately type an item using the current +type system. It should be used sparingly, as described in the :ref:`using-any` +section of the style guide. Attribute Access ---------------- @@ -799,6 +776,8 @@ all type checkers:: def foo(x: int | str) -> int | None: ... # recommended def foo(x: Union[int, str]) -> Optional[int]: ... # ok +.. _using-any: + Using `Any` and `object` ------------------------ @@ -814,6 +793,31 @@ that some function can accept literally anything: in those cases use When using `Any`, document the reason for using it in a comment. Ideally, document what types could be used. +The `Any` Trick +----------------- + +In cases where a function or method can return ``None``, but where forcing the +user to explicitly check for ``None`` can be detrimental, use +``_typeshed.MaybeNone`` (an alias to ``Any``), instead of ``None``. + +Consider the following (simplified) signature of ``re.Match[str].group``:: + + class Match: + def group(self, group: str | int, /) -> str | MaybeNone: ... + +This avoid forcing the user to check for ``None``:: + + match = re.fullmatch(r"\d+_(.*)", some_string) + assert match is not None + name_group = match.group(1) # The user knows that this will never be None + return name_group.uper() # This typo will be flagged by the type checker + +In this case, the user of ``match.group()`` must be prepared to handle a ``str``, +but type checkers are happy with ``if name_group is None`` checks, because we're +saying it can also be something else than an ``str``. + +This is sometimes called "the Any trick". + Context Managers ---------------- From d3a652adb206351052a914345e8441648cfdf7b5 Mon Sep 17 00:00:00 2001 From: yangdanny97 Date: Thu, 12 Dec 2024 15:50:23 -0500 Subject: [PATCH 09/13] fix backticks, add Docstrings section --- docs/guides/writing_stubs.rst | 79 +++++++++++++++++++---------------- 1 file changed, 44 insertions(+), 35 deletions(-) diff --git a/docs/guides/writing_stubs.rst b/docs/guides/writing_stubs.rst index 88fa801d..1a253665 100644 --- a/docs/guides/writing_stubs.rst +++ b/docs/guides/writing_stubs.rst @@ -503,7 +503,7 @@ more concise files. Syntax Example -------------- -The below is an excerpt from the types for the `datetime` module. +The below is an excerpt from the types for the ``datetime`` module. MAXYEAR: int MINYEAR: int @@ -680,13 +680,12 @@ No:: ... def to_int3(x: str) -> int: pass -Avoid invariant collection types (`list`, `dict`) for function parameters, -in favor of covariant types like `Mapping` or `Sequence`. +Avoid invariant collection types (``list``, ``dict``) for function parameters, +in favor of covariant types like ``Mapping`` or ``Sequence``. Avoid union return types. See https://github.com/python/mypy/issues/1693 -Use `float` instead of `int | float` for parameter annotations. -See [PEP 484](https://peps.python.org/pep-0484/#the-numeric-tower). +Use ``float`` instead of ``int | float`` for parameter annotations. See :pep:`484` for more details. Language Features ----------------- @@ -723,7 +722,7 @@ older versions of Python. Platform-dependent APIs ----------------------- -Use platform checks like `if sys.platform == 'win32'` to denote platform-dependent APIs. +Use :ref:`version-and-platform-checks` like ``if sys.platform == 'win32'`` to denote platform-dependent APIs. NamedTuple and TypedDict ------------------------ @@ -752,7 +751,7 @@ No:: Built-in Generics ----------------- -:pep:`585` built-in generics (such as `list`, `dict`, `tuple`, `set`) are supported and should be used instead +:pep:`585` built-in generics (such as ``list``, ``dict``, ``tuple``, ``set``) are supported and should be used instead of the corresponding types from ``typing``:: from collections import defaultdict @@ -770,7 +769,7 @@ generally possible and recommended:: Unions ------ -Declaring unions with the shorthand `|` syntax is recommended and supported by +Declaring unions with the shorthand ``|`` syntax is recommended and supported by all type checkers:: def foo(x: int | str) -> int | None: ... # recommended @@ -778,22 +777,22 @@ all type checkers:: .. _using-any: -Using `Any` and `object` ------------------------- +Using ``Any`` and ``object`` +---------------------------- -When adding type hints, avoid using the `Any` type when possible. Reserve -the use of `Any` for when: +When adding type hints, avoid using the ``Any`` type when possible. Reserve +the use of ``Any`` for when: * the correct type cannot be expressed in the current type system; and * to avoid union returns (see above). -Note that `Any` is not the correct type to use if you want to indicate +Note that ``Any`` is not the correct type to use if you want to indicate that some function can accept literally anything: in those cases use -`object` instead. +``object`` instead. -When using `Any`, document the reason for using it in a comment. Ideally, +When using ``Any``, document the reason for using it in a comment. Ideally, document what types could be used. -The `Any` Trick +The ``Any`` Trick ----------------- In cases where a function or method can return ``None``, but where forcing the @@ -822,17 +821,17 @@ Context Managers ---------------- When adding type annotations for context manager classes, annotate -the return type of `__exit__` as bool only if the context manager -sometimes suppresses exceptions -- if it sometimes returns `True` +the return type of ``__exit__`` as bool only if the context manager +sometimes suppresses exceptions -- if it sometimes returns ``True`` at runtime. If the context manager never suppresses exceptions, -have the return type be either `None` or `bool | None`. If you +have the return type be either ``None`` or ``bool | None``. If you are not sure whether exceptions are suppressed or not or if the -context manager is meant to be subclassed, pick `bool | None`. +context manager is meant to be subclassed, pick ``bool | None``. See https://github.com/python/mypy/issues/7214 for more details. -`__enter__` methods and other methods that return instances of the -current class should be annotated with `typing_extensions.Self` -([example](https://github.com/python/typeshed/blob/3581846/stdlib/contextlib.pyi#L151)). +``__enter__`` methods and other methods that return ``self`` or ``cls(...)`` +should be annotated with the `typing.Self` +(`example `_). Naming ------ @@ -845,29 +844,39 @@ A few guidelines for protocol names below. In cases that don't fall into any of those categories, use your best judgement. * Use plain names for protocols that represent a clear concept - (e.g. `Iterator`, `Container`). -* Use `SupportsX` for protocols that provide callable methods (e.g. - `SupportsInt`, `SupportsRead`, `SupportsReadSeek`). -* Use `HasX` for protocols that have readable and/or writable attributes - or getter/setter methods (e.g. `HasItems`, `HasFileno`). + (e.g. ``Iterator``, ``Container``). +* Use ``SupportsX`` for protocols that provide callable methods (e.g. + ``SupportsInt``, ``SupportsRead``, ``SupportsReadSeek``). +* Use ``HasX`` for protocols that have readable and/or writable attributes + or getter/setter methods (e.g. ``HasItems``, ``HasFileno``). .. _type-checker-error-suppression: Type Checker Error Suppression Formats -------------------------------------- -* Use mypy error codes for mypy-specific `# type: ignore` annotations, e.g. `# type: ignore[override]` for Liskov Substitution Principle violations. -* Use pyright error codes for pyright-specific suppressions, e.g. `# pyright: ignore[reportGeneralTypeIssues]`. -* If you need both on the same line, mypy's annotation needs to go first, e.g. `# type: ignore[override] # pyright: ignore[reportGeneralTypeIssues]`. +* Use mypy error codes for mypy-specific ``# type: ignore`` annotations, e.g. ``# type: ignore[override]`` for Liskov Substitution Principle violations. +* Use pyright error codes for pyright-specific suppressions, e.g. ``# pyright: ignore[reportGeneralTypeIssues]``. +* If you need both on the same line, mypy's annotation needs to go first, e.g. ``# type: ignore[override] # pyright: ignore[reportGeneralTypeIssues]``. -`@deprecated` -------------- +``@deprecated`` +--------------- -The `@typing_extensions.deprecated` decorator (`@warnings.deprecated` +The ``@typing_extensions.deprecated`` decorator (``@warnings.deprecated`` since Python 3.13) can be used to mark deprecated functionality; see -[PEP 702](https://peps.python.org/pep-0702/). +:pep:`702`. Keep the deprecation message concise, but try to mention the projected version when the functionality is to be removed, and a suggested replacement. + +Docstrings +---------- + +There are several tradeoffs around including docstrings in type stubs. Consider the intended purpose +of your stubs when deciding whether to include docstrings in your project's stubs. + +* They do not affect type checking results and will be ignored by type checkers. +* Docstrings can improve certain IDE functionality, such as hover info. +* Duplicating docstrings between source code and stubs requires extra work to keep them in sync. From 3e405f994459633ab8549c40105e8f5ecd840f89 Mon Sep 17 00:00:00 2001 From: yangdanny97 Date: Thu, 12 Dec 2024 15:51:29 -0500 Subject: [PATCH 10/13] fix one more backtick --- docs/guides/writing_stubs.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/guides/writing_stubs.rst b/docs/guides/writing_stubs.rst index 1a253665..ed22ee5b 100644 --- a/docs/guides/writing_stubs.rst +++ b/docs/guides/writing_stubs.rst @@ -93,7 +93,7 @@ Liskov substitutability or detecting problematic overloads. It may be instructive to examine `typeshed `__'s `setup for testing stubs `__. -To suppress type errors in stubs, use `# type: ignore` comments. Refer to the :ref:`type-checker-error-suppression` section of the style guide for +To suppress type errors in stubs, use ``# type: ignore`` comments. Refer to the :ref:`type-checker-error-suppression` section of the style guide for error suppression formats specific to individual typecheckers. .. From 830f2479f0a62917e67ee264278889f3a191f9f2 Mon Sep 17 00:00:00 2001 From: yangdanny97 Date: Thu, 12 Dec 2024 15:52:13 -0500 Subject: [PATCH 11/13] fix typo --- docs/guides/writing_stubs.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/guides/writing_stubs.rst b/docs/guides/writing_stubs.rst index ed22ee5b..b01f0191 100644 --- a/docs/guides/writing_stubs.rst +++ b/docs/guides/writing_stubs.rst @@ -270,7 +270,7 @@ annotated function ``bar()``:: ``Any`` vs. ``Incomplete`` -------------------------- -While ``Incomplete`` is a type alias of ``Any``, they serve difference purposes: +While ``Incomplete`` is a type alias of ``Any``, they serve different purposes: ``Incomplete`` is a placeholder where a proper type might be substituted. It's a "to do" item and should be replaced if possible. From fbd2031cd1864e3e33d47d39c6f8fec515bbb0e3 Mon Sep 17 00:00:00 2001 From: yangdanny97 Date: Thu, 12 Dec 2024 16:50:40 -0500 Subject: [PATCH 12/13] fix label --- docs/guides/writing_stubs.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/guides/writing_stubs.rst b/docs/guides/writing_stubs.rst index b01f0191..71b5d0eb 100644 --- a/docs/guides/writing_stubs.rst +++ b/docs/guides/writing_stubs.rst @@ -722,7 +722,7 @@ older versions of Python. Platform-dependent APIs ----------------------- -Use :ref:`version-and-platform-checks` like ``if sys.platform == 'win32'`` to denote platform-dependent APIs. +Use :ref:`platform checks` like ``if sys.platform == 'win32'`` to denote platform-dependent APIs. NamedTuple and TypedDict ------------------------ From cc9cdf01b44556db91739d3af70dd3d4a3827e43 Mon Sep 17 00:00:00 2001 From: Rebecca Chen Date: Mon, 16 Dec 2024 22:35:37 -0800 Subject: [PATCH 13/13] Minor style and formatting fixes Removed repeated example, fixed formatting, removed stray "the". --- docs/guides/writing_stubs.rst | 13 +++---------- 1 file changed, 3 insertions(+), 10 deletions(-) diff --git a/docs/guides/writing_stubs.rst b/docs/guides/writing_stubs.rst index 71b5d0eb..47f48b40 100644 --- a/docs/guides/writing_stubs.rst +++ b/docs/guides/writing_stubs.rst @@ -234,15 +234,7 @@ follow the following guidelines: can be left unannotated. * Do not use ``Any`` to mark unannotated or partially annotated values. Leave function parameters and return values unannotated. In all other cases, use - ``_typeshed.Incomplete``:: - - from _typeshed import Incomplete - - field1: Incomplete - field2: dict[str, Incomplete] - - def foo(x): ... - + ``_typeshed.Incomplete``. * Partial classes should include a ``__getattr__()`` method marked with ``_typeshed.Incomplete`` (see example below). * Partial modules (i.e. modules that are missing some or all classes, @@ -782,6 +774,7 @@ Using ``Any`` and ``object`` When adding type hints, avoid using the ``Any`` type when possible. Reserve the use of ``Any`` for when: + * the correct type cannot be expressed in the current type system; and * to avoid union returns (see above). @@ -830,7 +823,7 @@ context manager is meant to be subclassed, pick ``bool | None``. See https://github.com/python/mypy/issues/7214 for more details. ``__enter__`` methods and other methods that return ``self`` or ``cls(...)`` -should be annotated with the `typing.Self` +should be annotated with ``typing.Self`` (`example `_). Naming