PHP 8.0 | Tokenizer/PHP: support PHP8 dereferencing of text strings w…

…ith interpolated variables

As of PHP 8, interpolated text strings can be dereferenced, however, the square brackets are incorrectly tokenized as _short array_ brackets instead of as "normal" square brackets.

Fixed by adding the `T_DOUBLE_QUOTED_STRING` token to the allowed tokens for leaving the brackets alone in the PHP Tokenizer class.

Includes unit test.

Ref: https://wiki.php.net/rfc/variable_syntax_tweaks#interpolated_and_non-interpolated_strings

src/Tokenizers/PHP.php

@@ -2147,6 +2147,7 @@ protected function processAdditional()
                     T_NULLSAFE_OBJECT_OPERATOR => T_NULLSAFE_OBJECT_OPERATOR,
                     T_STRING                   => T_STRING,
                     T_CONSTANT_ENCAPSED_STRING => T_CONSTANT_ENCAPSED_STRING,
+                    T_DOUBLE_QUOTED_STRING     => T_DOUBLE_QUOTED_STRING,
                 ];
                 $allowed     += Util\Tokens::$magicConstants;
 

tests/Core/Tokenizer/ShortArrayTest.inc

@@ -65,6 +65,9 @@ echo (clone $iterable)[20];
 /* testNullsafeMethodCallDereferencing */
 $var = $obj?->function_call()[$x];
 
+/* testInterpolatedStringDereferencing */
+$var = "PHP{$rocks}"[1];
+
 /*
  * Short array brackets.
  */

tests/Core/Tokenizer/ShortArrayTest.php

@@ -73,6 +73,7 @@ public function dataSquareBrackets()
             ['/* testClassMemberDereferencingOnInstantiation2 */'],
             ['/* testClassMemberDereferencingOnClone */'],
             ['/* testNullsafeMethodCallDereferencing */'],
+            ['/* testInterpolatedStringDereferencing */'],
             ['/* testLiveCoding */'],
         ];