Merge upstream changes

commit 63f5658 Author: Erik De Bonte <[email protected]> Date: Tue Jun 20 10:50:18 2023 -0700 Support PEP 712's new attribute assignment conversion (microsoft#5343) commit 9cbe8c0 Author: Eric Traut <[email protected]> Date: Tue Jun 20 09:17:40 2023 -0700 Fixed a bug that caused an incorrect false positive error for an `assert_type` call involving a `Callable[[], X]` type. Pyright was generating a signature with a positional-only separator in this case. This addresses python/typeshed#10325 (comment). (microsoft#5345) Co-authored-by: Eric Traut <[email protected]> commit d12ad6e Author: Eric Traut <[email protected]> Date: Mon Jun 19 23:59:08 2023 -0700 Updated typeshed stubs to the latest version. commit eff69f5 Author: Eric Traut <[email protected]> Date: Mon Jun 19 22:35:56 2023 -0700 Fixed a bug that caused incorrect type inference for parameters in unannotated methods within child classes who derive from a generic parent class. This addresses a bug reported on stack overflow: https://stackoverflow.com/questions/76466874/trouble-specifying-type-parameter-when-inheriting-from-generic-class. commit 43c5fae Author: Eric Traut <[email protected]> Date: Mon Jun 19 20:17:00 2023 -0700 Finished cleanup of test cases. commit b0d4080 Author: Eric Traut <[email protected]> Date: Mon Jun 19 19:15:53 2023 -0700 Next batch of test case cleanup. commit efea9b9 Author: Eric Traut <[email protected]> Date: Mon Jun 19 18:26:49 2023 -0700 Changed behavior of non-ClassVar variables within a protocol definition. Previously, an error was reported when such variables were accessed from the class (as opposed to an instance of the class). Mypy (which was the reference implementation for PEP 544) does not report an error here. This addresses microsoft/pylance-release#4389. commit 5568b4d Author: Eric Traut <[email protected]> Date: Mon Jun 19 18:00:31 2023 -0700 Enhanced command-line version of pyright to allow file or directory names to be passed via stdin if `-` option is used in the command line. This addresses microsoft#5342. commit cd257f7 Author: Eric Traut <[email protected]> Date: Mon Jun 19 15:38:13 2023 -0700 Next batch of test case cleanup. commit 0c5c7b6 Author: Eric Traut <[email protected]> Date: Mon Jun 19 14:23:50 2023 -0700 Next batch of text case cleanup. commit 899799d Author: Eric Traut <[email protected]> Date: Mon Jun 19 13:33:41 2023 -0700 Finished cleaning up "genericTypesX.py" test files. commit 1111f1b Author: Eric Traut <[email protected]> Date: Mon Jun 19 11:49:56 2023 -0700 Started to untangle the hairball related to the "genericTypesX.py" test cases. commit 67394ad Author: Eric Traut <[email protected]> Date: Mon Jun 19 10:21:36 2023 -0700 Continued cleanup of test cases. commit 96e03aa Author: Eric Traut <[email protected]> Date: Mon Jun 19 09:39:09 2023 -0700 Did a cleanup pass for the dataclass test cases. Renamed and reordered test cases for maintainability. commit b7df200 Author: Eric Traut <[email protected]> Date: Mon Jun 19 08:20:54 2023 -0700 Fixed bug that resulted in a false positive error when defining a new type alias using the `TypeAliasType` constructor that defines no new type parameters but references an outer-scoped type parameter in the type alias definition. This addresses microsoft#5341. commit d816d00 Author: Eric Traut <[email protected]> Date: Mon Jun 19 00:00:07 2023 -0700 Started to do a pass over the test cases to make them more consistent. commit 23bcbce Author: Eric Traut <[email protected]> Date: Sun Jun 18 17:42:25 2023 -0700 Fixed bug in code flow engine that led to incorrect type evaluation of a variable in a nested loop. This addresses https://github.com/microsoft/pylance-release/issues/4509. commit 056415f Author: Eric Traut <[email protected]> Date: Sun Jun 18 16:40:00 2023 -0700 Fixed a bug in the control flow debugging code that prints the control flow graph. It was not correctly handling one of the node types which led to incomplete graphs. commit 643bb1d Author: Eric Traut <[email protected]> Date: Sun Jun 18 13:20:49 2023 -0700 Improved type inference for lambdas in the case where a parameter includes a default value and the expected type doesn't include that parameter. This improvement was suggested in the [mypy issue tracker](python/mypy#15459). (microsoft#5337) Co-authored-by: Eric Traut <[email protected]> commit 8ce23eb Author: Eric Traut <[email protected]> Date: Sun Jun 18 12:55:22 2023 -0700 Improved `reportUnnecessaryCast` so it works with types other than cl… (microsoft#5336) Improved `reportUnnecessaryCast` so it works with types other than class instances. This addresses microsoft#5333. commit 52c8cac Author: Eric Traut <[email protected]> Date: Sun Jun 18 12:41:24 2023 -0700 Changed type printer (the component that renders types into text) to use the lowercase `type[x]` instead of `Type[x]`. It has now been four years since PEP 585 deprecated the use of the upper-case version, so most developers should be getting comfortable with the lowercase version at this point. commit e3080b1 Author: Eric Traut <[email protected]> Date: Sun Jun 18 10:15:58 2023 -0700 Fixed a bug that led to a false positive error under certain circumstances when a literal type argument was used in conjunction with a protocol that used a covariant type parameter and an implementation of that protocol that used an invariant type parameter. This addresses microsoft#5282. (microsoft#5332) Co-authored-by: Eric Traut <[email protected]> commit 2b0f8d2 Author: Eric Traut <[email protected]> Date: Sun Jun 18 09:46:18 2023 -0700 Improved hover text to display the calculated variance for a PEP 695-style class-scoped type variable when the user hovers over the type parameter in the type param list. commit 02b7769 Author: Eric Traut <[email protected]> Date: Sun Jun 18 00:17:53 2023 -0700 Fixed a false positive error arising from the use of a binary expression for a base class in a class declaration statement. This addresses microsoft#5326. (microsoft#5331) Co-authored-by: Eric Traut <[email protected]> commit 2de35e3 Author: Eric Traut <[email protected]> Date: Sun Jun 18 00:00:06 2023 -0700 Added documentation about higher-order functions. commit 42a37f4 Author: Eric Traut <[email protected]> Date: Sat Jun 17 23:38:50 2023 -0700 Re-enabled a test case that was previously disabled because it was broken. commit 7ea11a1 Author: Eric Traut <[email protected]> Date: Sat Jun 17 23:17:32 2023 -0700 Minor code cleanup — rename constant for clarity and refactor validation function. No functional change. commit 53cb3f9 Author: Eric Traut <[email protected]> Date: Sat Jun 17 23:00:49 2023 -0700 Improved consistency of parameter ordering internally to type evaluator. No functional change. commit 9bf8231 Author: Eric Traut <[email protected]> Date: Sat Jun 17 22:48:18 2023 -0700 Fixed a bug that led to incorrect type evaluation when passing a generic class (with a constructor that includes class-scoped TypeVars) as an argument for a callable parameter. The class was being specialized prematurely (with type arguments set to `Unknown`) before the constraint solver was able to solve the higher-order function's type variables. This addresses microsoft#5324. (microsoft#5328) Co-authored-by: Eric Traut <[email protected]> commit 47cd514 Author: Eric Traut <[email protected]> Date: Sat Jun 17 22:21:52 2023 -0700 Changed auto-variance algorithm to ignore `__new__` and `__init__` methods for purposes of calculating the variance of a TypeVar. This mirrors the behavior of mypy. (microsoft#5327) Co-authored-by: Eric Traut <[email protected]> commit 3021b9c Author: Eric Traut <[email protected]> Date: Sat Jun 17 21:52:57 2023 -0700 Minor code cleanup — removed dead code and converted lambda to function. No functional change. commit 327ce37 Author: Eric Traut <[email protected]> Date: Sat Jun 17 20:30:36 2023 -0700 Added test case for microsoft#5027. commit 0559382 Author: Eric Traut <[email protected]> Date: Sat Jun 17 14:19:12 2023 -0700 Fixed false negative when a literal and non-literal are assigned to the same TypeVar in an invariant context. This addresses microsoft#5321. (microsoft#5323) Co-authored-by: Eric Traut <[email protected]> commit 2829429 Author: Eric Traut <[email protected]> Date: Sat Jun 17 12:33:13 2023 -0700 Fixed bug that led to an incorrect type evaluation for nested call expressions where an inner call expression used a ParamSpec. This addresses microsoft#5281. (microsoft#5322) Co-authored-by: Eric Traut <[email protected]>
jbradaric · Jun 20, 2023 · 7599cbb · 7599cbb
1 parent 149c366
commit 7599cbb
Show file tree

Hide file tree

Showing 844 changed files with 9,722 additions and 6,815 deletions.
diff --git a/docs/command-line.md b/docs/command-line.md
@@ -25,6 +25,7 @@ Pyright can be run as either a VS Code extension or as a node-based command-line
 | --version                               | Print pyright version and exit                       |
 | --warnings                              | Use exit code of 1 if warnings are reported          |
 | -w, --watch                             | Continue to run and watch for changes (5)            |
+| -                                       | Read file or directory list from stdin               |
 
 (1) If specific files are specified on the command line, it overrides the files or directories specified in the pyrightconfig.json or pyproject.toml file.
 

diff --git a/docs/mypy-comparison.md b/docs/mypy-comparison.md
@@ -346,6 +346,29 @@ def func2(x: list[int], y: list[str] | int):
     reveal_type(v2) # pyright: "list[int | str]" ("list[list[str] | int]" is also a valid answer)
 ```
 
+#### Constraint Solver: Higher-order Functions
+
+If a generic higher-order function is passed another generic callable as an argument, pyright is able to solve the type variables for both the target function and the argument. Mypy isn’t able to handle higher-order functions.
+
+```python
+def identity(val: T) -> T:
+    return val
+
+
+def higher_order1(cb: Callable[[S], T], arg: S) -> T:
+    return cb(arg)
+
+v1 = higher_order1(identity, 1.0)  # mypy generates an error
+reveal_type(v1)  # mypy: T, pyright: float
+
+
+def higher_order2(cb: Callable[P, R], *args: P.args, **kwargs: P.kwargs) -> R:
+    return cb(*args, **kwargs)
+
+v2 = higher_order2(identity, "")  # mypy generates an error
+reveal_type(v2)  # mypy: T, pyright: str
+```
+
 
 ### Constrained Type Variables
 

diff --git a/packages/pyright-internal/src/analyzer/codeFlowEngine.ts b/packages/pyright-internal/src/analyzer/codeFlowEngine.ts
@@ -864,6 +864,7 @@ export function getCodeFlowEngine(
                         reference === undefined &&
                         cacheEntry.incompleteSubtypes?.some((subtype) => subtype.type !== undefined);
                     let firstAntecedentTypeIsIncomplete = false;
+                    let firstAntecedentTypeIsPending = false;
 
                     loopNode.antecedents.forEach((antecedent, index) => {
                         // If we've trying to determine reachability and we've already proven
@@ -872,6 +873,10 @@ export function getCodeFlowEngine(
                             return;
                         }
 
+                        if (firstAntecedentTypeIsPending && index > 0) {
+                            return;
+                        }
+
                         cacheEntry = getCacheEntry(loopNode)!;
 
                         // Is this entry marked "pending"? If so, we have recursed and there
@@ -883,9 +888,21 @@ export function getCodeFlowEngine(
                             index < cacheEntry.incompleteSubtypes.length &&
                             cacheEntry.incompleteSubtypes[index].isPending
                         ) {
-                            sawIncomplete = true;
-                            sawPending = true;
-                            return;
+                            // In rare circumstances, it's possible for a code flow graph with
+                            // nested loops to hit the case where the first antecedent is marked
+                            // as pending. In this case, we'll evaluate only the first antecedent
+                            // again even though it's pending. We're guaranteed to make forward
+                            // progress with the first antecedent, and that will allow us to establish
+                            // an initial type for this expression, but we don't want to evaluate
+                            // any other antecedents in this case because this could result in
+                            // infinite recursion.
+                            if (index === 0) {
+                                firstAntecedentTypeIsPending = true;
+                            } else {
+                                sawIncomplete = true;
+                                sawPending = true;
+                                return;
+                            }
                         }
 
                         // Have we already been here (i.e. does the entry exist and is
@@ -895,7 +912,11 @@ export function getCodeFlowEngine(
                             cacheEntry.incompleteSubtypes !== undefined && index < cacheEntry.incompleteSubtypes.length
                                 ? cacheEntry.incompleteSubtypes[index]
                                 : undefined;
-                        if (subtypeEntry === undefined || (!subtypeEntry?.isPending && subtypeEntry?.isIncomplete)) {
+                        if (
+                            subtypeEntry === undefined ||
+                            (!subtypeEntry?.isPending && subtypeEntry?.isIncomplete) ||
+                            index === 0
+                        ) {
                             const entryEvaluationCount = subtypeEntry === undefined ? 0 : subtypeEntry.evaluationCount;
 
                             // Set this entry to "pending" to prevent infinite recursion.

diff --git a/packages/pyright-internal/src/analyzer/codeFlowUtils.ts b/packages/pyright-internal/src/analyzer/codeFlowUtils.ts
@@ -104,6 +104,8 @@ export function formatControlFlowGraph(flowNode: FlowNode) {
                 FlowFlags.WildcardImport |
                 FlowFlags.TrueCondition |
                 FlowFlags.FalseCondition |
+                FlowFlags.TrueNeverCondition |
+                FlowFlags.FalseNeverCondition |
                 FlowFlags.NarrowForPattern |
                 FlowFlags.ExhaustedMatch |
                 FlowFlags.Call |
@@ -115,7 +117,6 @@ export function formatControlFlowGraph(flowNode: FlowNode) {
                 | FlowVariableAnnotation
                 | FlowWildcardImport
                 | FlowCondition
-                | FlowCondition
                 | FlowExhaustedMatch
                 | FlowCall
                 | FlowPreFinallyGate

diff --git a/packages/pyright-internal/src/analyzer/constructors.ts b/packages/pyright-internal/src/analyzer/constructors.ts
@@ -722,7 +722,7 @@ export function createFunctionFromConstructor(
         const initType = evaluator.getTypeOfMember(initInfo);
         const objectType = ClassType.cloneAsInstance(classType);
 
-        const convertInitToConstructor = (initSubtype: FunctionType) => {
+        function convertInitToConstructor(initSubtype: FunctionType) {
             let constructorFunction = evaluator.bindFunctionToClassOrObject(
                 objectType,
                 initSubtype,
@@ -747,7 +747,7 @@ export function createFunctionFromConstructor(
             }
 
             return constructorFunction;
-        };
+        }
 
         if (isFunction(initType)) {
             return convertInitToConstructor(initType);

diff --git a/packages/pyright-internal/src/analyzer/dataClasses.ts b/packages/pyright-internal/src/analyzer/dataClasses.ts
@@ -25,10 +25,11 @@ import {
     TypeAnnotationNode,
 } from '../parser/parseNodes';
 import * as AnalyzerNodeInfo from './analyzerNodeInfo';
+import { getFileInfo } from './analyzerNodeInfo';
 import { createFunctionFromConstructor } from './constructors';
 import { DeclarationType } from './declaration';
 import { updateNamedTupleBaseClass } from './namedTuples';
-import { getEnclosingClassOrFunction, getScopeIdForNode } from './parseTreeUtils';
+import { getClassFullName, getEnclosingClassOrFunction, getScopeIdForNode, getTypeSourceId } from './parseTreeUtils';
 import { evaluateStaticBoolExpression } from './staticExpressions';
 import { Symbol, SymbolFlags } from './symbol';
 import { isPrivateName } from './symbolNameUtils';
@@ -513,7 +514,19 @@ export function synthesizeDataClassMethods(
                     effectiveType = transformDescriptorType(evaluator, effectiveType);
 
                     if (entry.converter) {
+                        const fieldType = effectiveType;
                         effectiveType = getConverterInputType(evaluator, entry.converter, effectiveType, entry.name);
+                        symbolTable.set(
+                            entry.name,
+                            getDescriptorForConverterField(
+                                evaluator,
+                                node,
+                                entry.converter,
+                                entry.name,
+                                fieldType,
+                                effectiveType
+                            )
+                        );
                     }
 
                     const effectiveName = entry.alias || entry.name;
@@ -799,6 +812,88 @@ function getConverterAsFunction(
     return undefined;
 }
 
+// Synthesizes an asymmetric descriptor class to be used in place of the
+// annotated type of a field with a converter. The descriptor's __get__ method
+// returns the declared type of the field and its __set__ method accepts the
+// converter's input type. Returns the symbol for an instance of this descriptor
+// type.
+function getDescriptorForConverterField(
+    evaluator: TypeEvaluator,
+    dataclassNode: ParseNode,
+    converterNode: ParseNode,
+    fieldName: string,
+    getType: Type,
+    setType: Type
+): Symbol {
+    const fileInfo = getFileInfo(dataclassNode);
+    const typeMetaclass = evaluator.getBuiltInType(dataclassNode, 'type');
+    const descriptorName = `__converterDescriptor_${fieldName}`;
+
+    const descriptorClass = ClassType.createInstantiable(
+        descriptorName,
+        getClassFullName(converterNode, fileInfo.moduleName, descriptorName),
+        fileInfo.moduleName,
+        fileInfo.filePath,
+        ClassTypeFlags.None,
+        getTypeSourceId(converterNode),
+        /* declaredMetaclass */ undefined,
+        isInstantiableClass(typeMetaclass) ? typeMetaclass : UnknownType.create()
+    );
+    descriptorClass.details.baseClasses.push(evaluator.getBuiltInType(dataclassNode, 'object'));
+    computeMroLinearization(descriptorClass);
+
+    const fields = descriptorClass.details.fields;
+    const selfType = synthesizeTypeVarForSelfCls(descriptorClass, /* isClsParam */ false);
+
+    const setFunction = FunctionType.createSynthesizedInstance('__set__');
+    FunctionType.addParameter(setFunction, {
+        category: ParameterCategory.Simple,
+        name: 'self',
+        type: selfType,
+        hasDeclaredType: true,
+    });
+    FunctionType.addParameter(setFunction, {
+        category: ParameterCategory.Simple,
+        name: 'obj',
+        type: AnyType.create(),
+        hasDeclaredType: true,
+    });
+    FunctionType.addParameter(setFunction, {
+        category: ParameterCategory.Simple,
+        name: 'value',
+        type: setType,
+        hasDeclaredType: true,
+    });
+    setFunction.details.declaredReturnType = NoneType.createInstance();
+    const setSymbol = Symbol.createWithType(SymbolFlags.ClassMember, setFunction);
+    fields.set('__set__', setSymbol);
+
+    const getFunction = FunctionType.createSynthesizedInstance('__get__');
+    FunctionType.addParameter(getFunction, {
+        category: ParameterCategory.Simple,
+        name: 'self',
+        type: selfType,
+        hasDeclaredType: true,
+    });
+    FunctionType.addParameter(getFunction, {
+        category: ParameterCategory.Simple,
+        name: 'obj',
+        type: AnyType.create(),
+        hasDeclaredType: true,
+    });
+    FunctionType.addParameter(getFunction, {
+        category: ParameterCategory.Simple,
+        name: 'objtype',
+        type: AnyType.create(),
+        hasDeclaredType: true,
+    });
+    getFunction.details.declaredReturnType = getType;
+    const getSymbol = Symbol.createWithType(SymbolFlags.ClassMember, getFunction);
+    fields.set('__get__', getSymbol);
+
+    return Symbol.createWithType(SymbolFlags.ClassMember, ClassType.cloneAsInstance(descriptorClass));
+}
+
 // If the specified type is a descriptor — in particular, if it implements a
 // __set__ method, this method transforms the type into the input parameter
 // for the set method.

diff --git a/packages/pyright-internal/src/analyzer/operations.ts b/packages/pyright-internal/src/analyzer/operations.ts
@@ -487,8 +487,8 @@ export function validateBinaryOperation(
 export function getTypeOfBinaryOperation(
     evaluator: TypeEvaluator,
     node: BinaryOperationNode,
-    inferenceContext: InferenceContext | undefined,
-    flags: EvaluatorFlags
+    flags: EvaluatorFlags,
+    inferenceContext: InferenceContext | undefined
 ): TypeResult {
     const leftExpression = node.leftExpression;
     let rightExpression = node.rightExpression;
@@ -505,7 +505,7 @@ export function getTypeOfBinaryOperation(
             operatorSupportsChaining(rightExpression.operator)
         ) {
             // Evaluate the right expression so it is type checked.
-            getTypeOfBinaryOperation(evaluator, rightExpression, inferenceContext, flags);
+            getTypeOfBinaryOperation(evaluator, rightExpression, flags, inferenceContext);
 
             // Use the left side of the right expression for comparison purposes.
             rightExpression = rightExpression.leftExpression;
@@ -622,12 +622,12 @@ export function getTypeOfBinaryOperation(
             adjustedLeftType = evaluator.reportMissingTypeArguments(
                 node.leftExpression,
                 adjustedLeftType,
-                flags | EvaluatorFlags.ExpectingType
+                flags | EvaluatorFlags.ExpectingInstantiableType
             );
             adjustedRightType = evaluator.reportMissingTypeArguments(
                 node.rightExpression,
                 adjustedRightType,
-                flags | EvaluatorFlags.ExpectingType
+                flags | EvaluatorFlags.ExpectingInstantiableType
             );
 
             const newUnion = combineTypes([adjustedLeftType, adjustedRightType]);