TYP: annotate

[jbrockmendel]

cc @simonjayhawkins had to revert some annotations because mypy couldn't tell that ArrayLike had "copy" or "view". Is a fix for that depending on numpy making annotations/stubs?

[simonjayhawkins]

maybe no need to revert. this was a typo. pandas.typing -> pandas._typing

[simonjayhawkins]

according to the docstring the return type should be just ExtensionArray?

[jreback]

can you update either the doc-string or the type

[jbrockmendel]

updated

[jorisvandenbossche]

Can you indicate when it returns an ndarray?

(I am wondering if we actually should be strict here, and tighten this method to always return an EA)

[WillAyd]

lgtm outside of this comment

[jbrockmendel]

Can you indicate when it returns an ndarray?

It isn't obvious to me how; did you have something in mind?

(I am wondering if we actually should be strict here, and tighten this method to always return an EA)

xref #24877 same idea but for astype.

[simonjayhawkins]

Is a fix for that depending on numpy making annotations/stubs?

I think it's because ArrayLike is a TypeVar. without numpy stubs, numpy imports resolve to Any and shouldn't therefore cause any mypy errors.

To us a typevar as the return type for a method in a class, the typevar needs tobe bound to something. I think in this case typing.Generic is needed. The following diff on 8f7d8aa resolves the has no attribute errors

diff --git a/pandas/core/arrays/base.py b/pandas/core/arrays/base.py
index 1991c8e8c..8750265a1 100644
--- a/pandas/core/arrays/base.py
+++ b/pandas/core/arrays/base.py
@@ -931,7 +931,7 @@ class ExtensionArray:
         """
         raise AbstractMethodError(self)
 
-    def view(self, dtype=None) -> ArrayLike:
+    def view(self, dtype=None) -> "ExtensionArray":
         """
         Return a view on the array.
 
diff --git a/pandas/core/base.py b/pandas/core/base.py
index 764a6fdfd..eac7f86e8 100644
--- a/pandas/core/base.py
+++ b/pandas/core/base.py
@@ -4,11 +4,12 @@ Base and utility classes for pandas objects.
 
 import builtins
 import textwrap
-from typing import Dict, FrozenSet, List, Optional
+from typing import Dict, FrozenSet, Generic, List, Optional
 
 import numpy as np
 
 import pandas._libs.lib as lib
+from pandas._typing import ArrayLike
 from pandas.compat import PYPY
 from pandas.compat.numpy import function as nv
 from pandas.errors import AbstractMethodError
@@ -34,7 +35,6 @@ from pandas.core.algorithms import duplicated, unique1d, value_counts
 from pandas.core.arrays import ExtensionArray
 from pandas.core.construction import create_series_with_explicit_dtype
 import pandas.core.nanops as nanops
-from pandas.typing import ArrayLike
 
 _shared_docs: Dict[str, str] = dict()
 _indexops_doc_kwargs = dict(
@@ -587,7 +587,7 @@ class ShallowMixin:
         return self._constructor(obj, **kwargs)
 
 
-class IndexOpsMixin:
+class IndexOpsMixin(Generic[ArrayLike]):
     """
     Common ops mixin to support a unified interface / docs for Series / Index
     """
diff --git a/pandas/core/indexes/base.py b/pandas/core/indexes/base.py
index 319664894..48221c922 100644
--- a/pandas/core/indexes/base.py
+++ b/pandas/core/indexes/base.py
@@ -12,7 +12,7 @@ from pandas._libs.lib import is_datetime_array, no_default
 from pandas._libs.tslibs import OutOfBoundsDatetime, Timestamp
 from pandas._libs.tslibs.period import IncompatibleFrequency
 from pandas._libs.tslibs.timezones import tz_compare
-from pandas._typing import Label
+from pandas._typing import ArrayLike, Label
 from pandas.compat import set_function_name
 from pandas.compat.numpy import function as nv
 from pandas.util._decorators import Appender, Substitution, cache_readonly
@@ -181,7 +181,7 @@ def _new_Index(cls, d):
     return cls.__new__(cls, **d)
 
 
-class Index(IndexOpsMixin, PandasObject):
+class Index(IndexOpsMixin[ArrayLike], PandasObject):
     """
     Immutable ndarray implementing an ordered, sliceable set. The basic object
     storing axis labels for all pandas objects.
@@ -3840,7 +3840,7 @@ class Index(IndexOpsMixin, PandasObject):
         return array
 
     @property
-    def _values(self) -> Union[ExtensionArray, np.ndarray]:
+    def _values(self) -> ArrayLike:
         """
         The best array representation.
 

[simonjayhawkins]

as a follow-on could then do either class ExtensionIndex(Index[ExtensionArray]): which would make the subclass of the generic class non-generic or class ExtensionIndex(Index[ExtensionArrayT]): and define ExtensionArrayT as a typevar bound only by ExtensionArray and keep ExtensionIndex as generic and the make the subclasses non-geneirc. i.e class CategoricalIndex(ExtensionIndex[Categorical], accessor.PandasDelegate):

updated: typo class CategoricalIndex(Categorical[], accessor.PandasDelegate): -> class CategoricalIndex(ExtensionIndex[Categorical], accessor.PandasDelegate):

[simonjayhawkins]

Thanks @jbrockmendel lgtm. some comments/suggestions (for follow-up), not blockers.

[jbrockmendel]

are any of the remaining discussion points actionable?

[simonjayhawkins]

lgtm

[simonjayhawkins]

Thanks @jbrockmendel