-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Format-preserving AST transformations TODO #344
Comments
Hi, preserving original code formatting when performing AST transformations will be very useful for code refactoring. However, even without it php-parser has still been useful to me as a validating tool for automatic refactoring done with other mechanisms which do preserve the formatting. Here is an example using Feel free to take the php code as an example for php-parse, it implements a visitor which changes the second argument of a particular function to be null under certain conditions. |
As a small update here: While there has been no movement on the above TODOs, all the necessary parser changes are now in 4.0, without the need to enable special options. As such, the boilerplate code to perform format-preserving transformations is now: $lexer = new Lexer\Emulative([
'usedAttributes' => [
'comments',
'startLine', 'endLine',
'startTokenPos', 'endTokenPos',
],
]);
$parser = new Parser\Php7($lexer);
$traverser = new NodeTraverser();
$traverser->addVisitor(new NodeVisitor\CloningVisitor());
$printer = new PrettyPrinter\Standard();
$oldStmts = $parser->parse($code);
$oldTokens = $lexer->getTokens();
$newStmts = $traverser->traverse($oldStmts);
// MODIFY $newStmts HERE
$newCode = $printer->printFormatPreserving($newStmts, $oldStmts, $oldTokens); |
Hello A few months ago, I wrote an AST merger: https://github.com/kiboko-labs/akeneo-product-values-management/blob/master/src/Builder/ClassMerger.php It is very specialized to the needs I had: appending properties and methods to existing classes. Maybe this solution could be some way to answer some of these needs? |
Thanks for creating this issue. I wonder about another approach to this, let me share:
If you already use coding standard, just use it after PHP-Parser run and code is both refactored and according your style needs. No need to integrate various coding standard rules to parser itself. Parser should parse and focus on printing, if there are maintained tools for that. Maybe there are other features of Parser that would be happy for attention and that cannot be handled by coding standards. I'm working on this approach in EasyCodingStandard combined with Rector - CLI tool to refactor legacy code to modern and clean code. Rector will do refactoring + coding standard fixes in one command. What do you think? |
Thanks for the update and sharing specific code. Is there any related PR to check? I'm curious what have you changed. |
I don't have concrete plans for a 4.0 release yet. I did not have time to work on this further and would like to at least resolve some of the TODO items (especially the first one is a large limitation). |
I see. I'm adding a method to specific position like this. |
@nikic any ETA on releasing a stable version of format preservation? We really need it :) |
Yes please! |
+1 |
Hi all, I'm trying to modify some existing code - just pushing an item onto the end of an array, but the formatting is coming out like so:
I cloned an existing node in the hope that it would copy the starting position. Here is the relevant code: public function addToArray($code)
{
$visitor = new class extends NodeVisitorAbstract {
public function leaveNode(Node $node) {
if ($node instanceof Array_) {
$newItem = clone $node->items[0];
$node->items[] = $newItem;
return $node;
}
}
};
return $this->modify($code, $visitor);
}
public function modify($code, $visitor)
{
$oldStmts = $this->parser->parse($code);
$oldTokens = $this->lexer->getTokens();
$newStmts = $this->traverser->traverse($oldStmts);
$this->traverser->addVisitor($visitor);
$newStmts = $this->traverser->traverse($newStmts);
return $this->printer->printFormatPreserving($newStmts, $oldStmts, $oldTokens);
} Could someone please advise me if this is a bug or not yet supported? |
@Juddling It's half-supported. The formatting of the array is preserved, but the new element is added on the same line. There is currently no detection that the array is formatted in multi-line style and the new element should be added in multi-line style as well. |
I've released a beta version for PHP-Parser 4.0 (https://github.com/nikic/PHP-Parser/releases/tag/v4.0.0beta1) and plan to create a stable release soonishly. I'd say that at this point, this feature is working pretty well, and it just comes down to fixing formatting issues as they come up. If I'm going to wait until this functionality is "finished", I'm afraid we'll never see a release... |
@nikic it is definitely good point not to delay release (you will be able to release next major if needed) Anyway this feature is very important for many developers like me :) Current PHP-Parser implementation is cool, but if some code changes needed then developers should recombine AST with tokens in order to keep formatting like this. It's also impossible to generate custom code with required formatting from AST. So will it be possible to work on this feature later? But this will require a lot of changes to switch from lexical parsing to raw nodes, because of skipping whitespace nodes :( PS. Therefore in Java (IDEA engine) AST implementation provides an access to whitespace nodes. But you need to use special methods |
What about preserving redundant parentheses? When I parse:
I get:
Which makes it almost impossible to restore the parentheses without some very complicated walking. |
Hi, I may meet some "half-supported". Here is my code namespace app;
//use PDO;
class Foo
{
} I want add some footnodes into it namespace app;
//use PDO;
class Foo
{
}
require_once __CACHE__."header.php" While, if this file includes "use xxx;", everything is fine, the formatting is the expected. namespace app;
//use PDO; <---------------------here -----------------
class Foo
{
}
require_once __CACHE__."header.php"
|
@eeliu I can't reproduce this issue. The following test passes for me:
Adding a |
@nikic Add some annotations in it.
become
|
Unfortunately for me this simple code:
after deletion of global stmt become this:
New code doesn't preserve tabulation and open bracket in functions. |
Closing this tracking issue as done. If there are issues with formatting preservation, they should get reported as separate issues. |
With #41 (comment) the master branch now supports preserving original code formatting when performing AST transformations. At this point formatting-preservation works well in practice, and remaining issues are addressed on an as-needed basis. Please open an issue or comment if you encounter a problem.
$key =>
)."${foo[0]}"
works correctly. I didn't test, but suspect that changing thefoo
here might result in incorrect output.getLexer()
method to parser. Otherwise it would be necessary to pass around the lexer with the parser.The text was updated successfully, but these errors were encountered: