[WIP] Variable collector #224

dantleech · 2016-11-27T13:59:48Z

This is an initial attempt at a variable collector (#222), which can be used by the ReflectionMethod to determine the variables that are used. The bulk of this code is concerned with resolving types so that:

/**
 * @var Foobar
 */
public function something()
{
     $bar = $this->foobar->getSomeClass()->getFoo();
}

If the getSomeClass method has the return type SomeClass class and SomeClass a method getFoo that returns type int, then $bar will be resolved to int.

$reflection = $reflector->reflect(SomeClass::class);
$vars = $reflection->getMethod('something')->getVariables();

echo $vars[0]->getName(); // bar
echo $vars[0]->getType(); // int
echo $vars[0]->getDeclaredAt(); // 6

The variable collector visitor can return locally assigned and parameter variables available in the AST it is traversing (regardless of scope). In the case of getting the variables in a method it would suffice to traverse only the method AST.

This is really rough WIP atm and contains an unknown number of bugs, so before reviewing do you agree with the general approach?

TODO:

Types from static methods/properties.
...

Ocramius · 2016-11-27T15:38:29Z

src/Reflection/ReflectionVariable.php

+
+    /**
+     * @param string $name
+     * @param int $declaredAt


Unsure about just having the line number.

Well, ideally there would be the range of (characters?) that the variable is available for, but the PHP parser just gives us line numbers ..

Ocramius · 2016-11-27T15:39:29Z

src/Reflection/ReflectionVariable.php

+    /**
+     * @param string $name
+     * @param int $declaredAt
+     * @param string $type


Doesn't say much - need some sort of more restrictive hint here

Its actually a ReflectionType ...

Ocramius · 2016-11-27T15:40:51Z

src/Reflection/ReflectionVariable.php

+    }
+
+    /**
+     * Get a PhpDocumentor type object for this type


What if the type mutated during execution? Two variables with same name, but different type and declaration line, then?

We return all declarations, so you could get two with the same name, but covering different parts of the code.

Ocramius · 2016-11-27T15:41:12Z

src/Reflection/ReflectionVariable.php

+     * @see https://github.com/php/php-src/blob/master/ext/reflection/php_reflection.c#L2993
+     * @return string
+     */
+    public function __toString()


Wouldn't add this

I found it useful for testing, but mainly I added it to be consistent with the other Reflection* classes.

Unless it actually is required API, I wouldn't add it. __toString() is often abused, so let's not give users more rope to hang themselves :)

ok, will remove it. I can only guess that the __toString functionality in the other classes is useful for dumping a representation of a class as a whole, and that Variables would not be part of the representation /cc @asgrim

Ocramius · 2016-11-27T15:41:52Z

src/Util/Visitor/VariableCollectionVisitor.php

+        }
+    }
+
+    private function processAssignation(Expr\Assign $node)


Just processAssign

Ocramius · 2016-11-27T15:42:50Z

Approach looking sound to me. Loads of testing and refactoring to be done though

dantleech · 2016-11-28T10:20:09Z

.travis.yml

@@ -1,7 +1,7 @@
 language: php

 php:
-  - 5.6
+    #  - 5.6


Have switched to using PHP7 at least for development - guessing that you are going to switch to PHP7 in the near-future anyway?

Yes - that's the plan :) #221

dantleech · 2016-11-28T10:22:18Z

src/Util/Visitor/VariableCollectionVisitor.php

+        //       actually have a return type??
+        if ($expr instanceof Expr\FuncCall) {
+            $func = (string) $expr->name;
+            $reflection = new \ReflectionFunction($func);


This is pretty hacky - ~~wonder if we should simply not support functions~~ I guess we should at least try and use BR\ReflecitonFunction.

You can stop at the return type declared on the function btw - no need to introspect.

hmm, this is a function_call(), how else would I determine the return type? But anyway, not tested yet, so I have no idea what would happen here.

dantleech · 2016-11-28T10:23:09Z

src/Util/Visitor/VariableCollectionVisitor.php

+
+    private function createUnknownReflectionType()
+    {
+        return $this->createReflectionTypeFromString('mixed', false);


What should we do when the type cannot be determined? Currently returning mixed - should we return the type as null?

Should be null, IMO. Adding mixed is a very specific thing, plus nothing prevents a class mixed {} to exist.

dantleech · 2016-12-04T20:24:27Z

At the moment this PR scope is about collecting "available" variables (#222), currently including the parameters given in a method call. If taken further this means we should also return $this as a variable, and further still, globals, which we cannot do.

For my specifc use case, I need the "available" variables (albeit without the globals). And I am already using this PR as a base for further introspection.

Do you think this functionality be limited to variables assigned within a scope? Or should we return the func/method param variables also?

MarkRedeman · 2016-12-22T19:38:20Z

@dantleech I think there are still a couple of variable definitions that you're missing in this PR.

Variable definition through a foreach statement. (felixfbecker/php-language-server/ReflectionTypes API additions for Enums/Union types #200)
Pass by reference (Count pass-by-reference calls as variable definition felixfbecker/php-language-server#205)
Assignment trough lists (Support for variable assignments with list() felixfbecker/php-language-server#170)
Array destructuring (PHP 7.1+, https://wiki.php.net/rfc/short_list_syntax)

dantleech · 2016-12-22T21:12:19Z

I think there are still a couple of variable definitions that you're missing in this PR.

indeed, it is far from being complete.

Variable definition through a foreach statement.

How? PHPStorm would rely on an annotation above the foreach loop here I think (?)

MarkRedeman · 2016-12-23T09:11:43Z

indeed, it is far from being complete.

I understand, I mostly wanted to make you aware of those use cases :).

How? PHPStorm would rely on an annotation above the foreach loop here I think (?)

That's probably going to be a tricky one.. You might be able to check whether the type of the iterable. So that if you're iterating over an Iterator then you can check the return type of current.
If the iterable is an IteratorAggregate then this would require some recursive checks.
For arrays I guess the best use case would be to guess their types from the docblocks.

Btw the use cases I mentioned were mostly about finding the definitions of a variable, not necessarily including their types (though it would be very useful).

asgrim · 2016-12-24T12:37:27Z

@dantleech note: the core namespace is now \Roave\BetterReflection so this PR will need updating. No need to provide BC shim class entries for this PR as it's a new feature anyway :)

TomasVotruba · 2017-10-24T13:48:20Z

@dantleech Hey, thanks for this awesome PR. I need exactly this feature, just without reflection and based on nikic\php-parser, so I might take a lot from it.

Do you expect to finish this now on BR 2.0? I could help if you need

dantleech · 2017-10-24T14:06:48Z

@TomasVotruba I don't anticipate working on this, I already started another project which does this. The problem is that it is a huge amount of work with lots of edge cases. My implementation works "OK" for the purpose (auto completion) / reference finding.

TomasVotruba · 2017-10-24T14:11:29Z

@dantleech Oh, I found out your package WorseReflection where you've probably implemented this.

Where could I find there this feature? I looked to ReflectionMethod but could not find it.

dantleech · 2017-10-24T14:18:37Z

The entry point is Reflector::reflectOffset(): ReflectionOffset ($offset->frame() returns the frame).

Most of the code is here and close by (but I think ReflectionMethod also has a ->frame() method which returns all the variables).

TomasVotruba · 2017-10-26T08:36:28Z

@dantleech Not sure why, but I didn't see your comments on mobile, that explains my comments :).
Neither I got notified about your last comment. Thanks a lot for that, I'll look on it.

How are you satisfied with the "microsoft/tolerant-php-parser"?

Any useful code parts that you think might be useful for me?

dantleech · 2017-10-26T08:41:33Z

In general I am happy with tolerant parser. It's designed for IDE usage, the API is richer than nikic/php-parser. For example $node->getFirstAncestor(FunctionLike::class, ClassLike::class), $node->getDescendantAtOffset(1234). It also will resolve the fully qualified names of things, so $node->getFirstAncestor(ClassLike::class)->getFullyQualifiedName() etc.

The tree is read-only however, so you can't build code (but it offers a TextEdit class which can apply changes to the raw source code).

It's not stable, and obviously not ubiquitous as nikic/php-parser, so some potential risk there I think

TomasVotruba · 2017-10-26T09:20:00Z

@dantleech Thanks, I'll look on these features.
I managed to implement most of them via custom NodeVisitors already.

Just off-topic: It looks like you have found another solution, so maybe this PR can be closed. I thouth it's WIP and expected it might be finished, but now I see there is complete another way you took. What do you think?

dantleech · 2017-10-26T09:50:09Z

The approach is basically the same - the only reason I created WorseReflection was because it's the core domain of Phpactor and I had lots of requirements. I think the WR implementation could be used as a reference for implementing this feature in the future (I also think the PHP Language Server would be worth a look in this respect).

TomasVotruba · 2017-10-27T16:01:03Z

In case anyone would be looking for getFirstAncestor for nikic/php-parser, here it is.

Variable collector

d9ff53e

Ocramius reviewed Nov 27, 2016

View reviewed changes

Refactoring

448b9ba

dantleech commented Nov 28, 2016

View reviewed changes

asgrim added enhancement feature request labels Nov 28, 2016

dantleech commented Nov 28, 2016

View reviewed changes

Use CompilerContext

8c599d8

asgrim added the WIP label Nov 30, 2016

dantleech added 2 commits December 4, 2016 18:26

Use isBuiltin and FunctionLike, custom Lexer

3a73b19

Added ReflectionVariable test

21a81c3

dantleech added 2 commits December 6, 2016 08:27

Segregating logic

3ba940b

Added scope binding

eeffd3b

dantleech force-pushed the variable_collector branch from a860170 to eeffd3b Compare December 6, 2016 09:11

dantleech added 2 commits December 7, 2016 20:49

Allow null returns and reassigned var handling

ddd165c

Added some 'temporary' hacks

bf38f08

asgrim assigned dantleech Dec 8, 2016

MarkRedeman mentioned this pull request Dec 22, 2016

Possible useful features from Roave/BetterReflection felixfbecker/php-language-server#223

Closed

TomasVotruba mentioned this pull request Oct 26, 2017

Examine Tolerant-parser inspiration rectorphp/rector#111

Closed

dantleech closed this Oct 26, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Variable collector #224

[WIP] Variable collector #224

dantleech commented Nov 27, 2016 •

edited

Loading

Ocramius Nov 27, 2016

dantleech Nov 27, 2016

Ocramius Nov 27, 2016

dantleech Nov 27, 2016

Ocramius Nov 27, 2016

dantleech Nov 27, 2016

Ocramius Nov 27, 2016

dantleech Nov 27, 2016

Ocramius Nov 27, 2016

dantleech Dec 4, 2016 •

edited

Loading

Ocramius Nov 27, 2016

Ocramius commented Nov 27, 2016

dantleech Nov 28, 2016

asgrim Nov 28, 2016

dantleech Nov 28, 2016 •

edited

Loading

Ocramius Dec 4, 2016

dantleech Dec 4, 2016

dantleech Nov 28, 2016

Ocramius Dec 4, 2016

dantleech commented Dec 4, 2016 •

edited

Loading

MarkRedeman commented Dec 22, 2016

dantleech commented Dec 22, 2016

MarkRedeman commented Dec 23, 2016

asgrim commented Dec 24, 2016

TomasVotruba commented Oct 24, 2017

dantleech commented Oct 24, 2017 •

edited

Loading

TomasVotruba commented Oct 24, 2017

dantleech commented Oct 24, 2017 •

edited

Loading

TomasVotruba commented Oct 26, 2017

dantleech commented Oct 26, 2017 •

edited

Loading

TomasVotruba commented Oct 26, 2017

dantleech commented Oct 26, 2017

TomasVotruba commented Oct 27, 2017 •

edited

Loading

[WIP] Variable collector #224

[WIP] Variable collector #224

Conversation

dantleech commented Nov 27, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dantleech Dec 4, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Ocramius commented Nov 27, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dantleech Nov 28, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dantleech commented Dec 4, 2016 • edited Loading

MarkRedeman commented Dec 22, 2016

dantleech commented Dec 22, 2016

MarkRedeman commented Dec 23, 2016

asgrim commented Dec 24, 2016

TomasVotruba commented Oct 24, 2017

dantleech commented Oct 24, 2017 • edited Loading

TomasVotruba commented Oct 24, 2017

dantleech commented Oct 24, 2017 • edited Loading

TomasVotruba commented Oct 26, 2017

dantleech commented Oct 26, 2017 • edited Loading

TomasVotruba commented Oct 26, 2017

dantleech commented Oct 26, 2017

TomasVotruba commented Oct 27, 2017 • edited Loading

dantleech commented Nov 27, 2016 •

edited

Loading

dantleech Dec 4, 2016 •

edited

Loading

dantleech Nov 28, 2016 •

edited

Loading

dantleech commented Dec 4, 2016 •

edited

Loading

dantleech commented Oct 24, 2017 •

edited

Loading

dantleech commented Oct 24, 2017 •

edited

Loading

dantleech commented Oct 26, 2017 •

edited

Loading

TomasVotruba commented Oct 27, 2017 •

edited

Loading