Should we support type annotations in docstrings for Python 2 code?

[JukkaL]

We could support an alternative, docstring-based syntax for types in programs that need to be Python 2 compatible. The idea is from http://sphinxcontrib-napoleon.readthedocs.org/en/latest/example_google.html

Example:

def first_name(name):
    """Return the first name from a name with both a first and a last name.

    Args:
      name (str): The full name, such as 'John Smith'.

    Returns:
      str: The first name.
    """
    return name.split()[0]

Mypy would threat this as equivalent to this:

def first_name(name: str) -> str:
    return name.split()[0]

A bunch of details would have to be fleshed out before implementing this. I'm adding this issue to get feedback on whether people would consider this useful. I've started seeing code using the above convention (though with slightly ad-hoc types).

Note that we can support # type: annotations as an alternative. They could be pretty useful for small, internal functions that may not have a docstring at all. The docstring syntax would be pretty verbose if used for every function. Example:

def first_name(name):  # type: (str) -> str
    return name.split()[0]

Maybe we should extend the docstring syntax to have optional shorter forms:

def join_name(first_name, last_name):
    """Join first and last name into a full name.

    Args: first_name (str), last_name (str)
    Returns: str
    """
    return first_name + ' ' + last_name

Another shorter form:

def join_name(first_name, last_name):
    """Join first and last name into a full name.

    Args: 
      first_name (str)
      last_name (str)
    Returns: str
    """
    return first_name + ' ' + last_name

Another alternative for Python 2 code is to use the mypy codec that supports Python 3 type annotations in Python 2 code by automatically stripping them away. This has a few problems:

1. The codec generally needs to be activated before it can be used, which is a minor pain, especially for entry point / main module(s).
2. Tools such as IDEs and linters may get confused by it (but they could potentially be used in Python 3 mode, if the code is also Python 3 compatible).
3. It slows down bytecode compilation or program startup. However, if we'd implement it in C, it could be very fast for CPython. But what about PyPy, Jython etc.?

[refi64]

I don't really like this. It feels conflicting with the Python philosophy, "There should be one way to do it." (I might have slightly misquoted it...)

[JukkaL]

@kirbyfan64 Which syntax would you prefer for Python 2 code?

[refi64]

@JukkaL I thought mypy was going to use a decorator?

Maybe something like this could be used:

@typed(ret_type)(arg1_type)
def xyz(arg1):
    ...

[JukkaL]

The main use case for decorators was function overloading, as runtime dispatch needs access to the type objects, and decorators could easily support this. But PEP 484 only supports overloading in stubs, so this use case no longer exists. Thus we could have Python 2 annotations in comments without issues.

I don't have much enthusiasm for decorators mainly for aesthetic reasons: they make my code look "noisy" compared to normal Python code (including code with Python 3 function annotations). I think that the syntax alternatives are closer to the look of normal Python code.

However, using a comment together with a docstring looks pretty bad to me:

def join_name(first_name, last_name):
    # type: (str, str) -> str
    """Join first and last name into a full name.

    ... more discussion ...
    """
    return first_name + ' ' + last_name

This isn't much better, and the type annotation is easy to miss:

def join_name(first_name, last_name):
    """Join first and last name into a full name.

    ... more discussion ...
    """
    # type: (str, str) -> str
    return first_name + ' ' + last_name

[refi64]

@JukkaL Yeah...this is probably a lose-lose situation...

[presidento]

Another ideas from PyCharm IDE.

Using :type for reStructuredText:

def first_name(name):
    """Return the first name from a name with both a first and a last name.

    :type name: str
    :rtype: str
    """
    return name.split()[0]

Using @type annotation:

def first_name(name):
    """Return the first name from a name with both a first and a last name.

    @type name: str
    @rtype: str
    """
    return name.split()[0]

Using the param element:

def first_name(name):
    """Return the first name from a name with both a first and a last name.

    :param str name: The full name, such as 'John Smith'.
    :rtype: str
    """
    return name.split()[0]

The rtype is for return type.

[gvanrossum]

FWIW we decided against this, at least in the short term -- instead we're using a comment-based notation that is now added to PEP 484 (and also to PEP 8).

[JukkaL]

We experimented with it using some Dropbox code but weren't very successful and the main problems were these:

• People are used to putting free form text in the docstring. Enforcing a particular syntax would probably require a lot of existing docstrings to be converted manually, or we'd have multiple docstring conventions in the same codebase, some of which would be understood by mypy and others wouldn't.
• Even existing docstrings that try to follow a particular convention such as Google's generally don't follow the conventions very rigorously.
• Existing, informal signature type descriptions in docstrings are free-form and can be ambiguous, and can't be trivially translated to PEP 484 syntax. Understanding the code is needed for translating existing types in docstrings.
• If people use the wrong (even slightly wrong) convention for a docstring due to historical reasons, by accident or by habit, mypy would probably treat it as a regular docstring and wouldn't perform type checking. Comment-based signatures have a simpler syntax and it's less likely that people write something that looks like a comment-based annotation but mypy would ignore it. A 'strict type checking mode' could complain about functions without recognized annotations, though.
• The docstring based convention is very verbose for short utility functions. We'd probably want to support a shorter alternative syntax anyway, and having multiple syntax variants in a single file would be awkward.

[JukkaL]

I'm tagging this as postponed.

[JukkaL]

Should we just close this? We can always reopen this if we change our minds.

[gvanrossum]

Yeah, we're not going to do this.

• Types in docstrings are usually not checked so are often incorrect.
• The docstring syntax for types is unnecessarily verbose.
• We don't want to support multiple syntaxes forever.
• There doesn't even seem to be agreement on how to write types in docstrings.