Port babel-parser changes from 2022-01-10 to 2022-02-26 #716

alangpierce · 2022-07-06T22:59:13Z

Instructions: https://github.com/alangpierce/sucrase/wiki/Porting-changes-from-Babel's-parser

69246b6212 [babel 8] fix properties name for function-like TS nodes (#13709)
🚫 AST only.

bc0d1ef860 v7.16.8
🚫 Release only.

478a9709ab Improve errors location tracking (#14130)
🚫 Change to error messaging details that were removed in Sucrase.

4a737547e5 v7.16.10
🚫 Release only.

e5d29f6e40 fix: incorrect conciseBody lookahead (#14194)
✅ Bug doesn't appear in Sucrase, but I added a test anyway.

5266605528 v7.16.12
🚫 Release only.

5861002b33 Reinterpret << when parsing TS type arguments (#14145)
✅ Ported following a very similar implementation.

6b427ced22 Fuzz test location-related parser options (#14201)
🚫 Babel-internal change testing location logic not present in Sucrase.

96a8251def Add parser support for the "regexp unicode sets" proposal (#14086)
✅ No implementation necessary, but I added a quick test.

f893b333a8 Add the decoratorsAutoAccessors parser plugin (#13681)
✅ Ported basic parsing for accessor properties.

de5c7b1bce Parse destructuring private fields (#13931)
✅ Seems to be already working, and I added a test.

🚫 d50c18dbc2 fix: usePrivateName usage
Only affects Babel scope code, not relevant to Sucrase.

🚫 b092bd0cb5 remove invalid test output
Only affects Babel tests.

df27d542ef proposal-pipe: Add support for ^^ and @@ topics (#13973)
🚫 Sucrase is holding off on pipeline topic parsing until the details are finalized. Tracked in #674.

97a8bcb9cc Expose .index on Position to internally track nodes location (#14174)
🚫 Only affects class not used in Sucrase.

38c23cded4 v7.17.0
🚫 Release only.

19ede090eb parser: Update babel-parser.d.ts for 7v.17.0 features (#14266)
🚫 Not relevant to Sucrase.

f52c70c357 v7.17.3
🚫 Release only.

a53c2fa4a2 fix(ts): skip func-type param start on parsing (#14293)
🚫 Bug not present in Sucrase.

5749c16dc2 [tsx] raise error on single arrow type argument without comma (#14135)
🚫 Only affects error handling.

Instructions: https://github.com/alangpierce/sucrase/wiki/Porting-changes-from-Babel's-parser 69246b6212 [babel 8] fix properties name for function-like TS nodes (#13709) 🚫 AST only. bc0d1ef860 v7.16.8 🚫 Release only. 478a9709ab Improve errors location tracking (#14130) 🚫 Change to error messaging details that were removed in Sucrase. 4a737547e5 v7.16.10 🚫 Release only. e5d29f6e40 fix: incorrect conciseBody lookahead (#14194) ✅ Bug doesn't appear in Sucrase, but I added a test anyway. 5266605528 v7.16.12 🚫 Release only. 5861002b33 Reinterpret << when parsing TS type arguments (#14145) ✅ Ported following a very similar implementation. 6b427ced22 Fuzz test location-related parser options (#14201) 🚫 Babel-internal change testing location logic not present in Sucrase. 96a8251def Add parser support for the "regexp unicode sets" proposal (#14086) ✅ No implementation necessary, but I added a quick test. f893b333a8 Add the `decoratorsAutoAccessors` parser plugin (#13681) ✅ Ported basic parsing for accessor properties. de5c7b1bce Parse destructuring private fields (#13931) ✅ Seems to be already working, and I added a test. 🚫 d50c18dbc2 fix: usePrivateName usage Only affects Babel scope code, not relevant to Sucrase. 🚫 b092bd0cb5 remove invalid test output Only affects Babel tests. df27d542ef proposal-pipe: Add support for `^^` and `@@` topics (#13973) 🚫 Sucrase is holding off on pipeline topic parsing until the details are finalized. Tracked in #674. 97a8bcb9cc Expose `.index` on Position to internally track nodes location (#14174) 🚫 Only affects class not used in Sucrase. 38c23cded4 v7.17.0 🚫 Release only. 19ede090eb parser: Update `babel-parser.d.ts` for 7v.17.0 features (#14266) 🚫 Not relevant to Sucrase. f52c70c357 v7.17.3 🚫 Release only. a53c2fa4a2 fix(ts): skip func-type param start on parsing (#14293) 🚫 Bug not present in Sucrase. 5749c16dc2 [tsx] raise error on single arrow type argument without comma (#14135) 🚫 Only affects error handling.

codecov · 2022-07-06T23:03:01Z

Codecov Report

Merging #716 (268be2a) into main (00a86a2) will increase coverage by 1.33%.
The diff coverage is 96.87%.

@@            Coverage Diff             @@
##             main     #716      +/-   ##
==========================================
+ Coverage   85.53%   86.86%   +1.33%     
==========================================
  Files          55       54       -1     
  Lines        5883     5665     -218     
  Branches     1332     1333       +1     
==========================================
- Hits         5032     4921     -111     
+ Misses        574      472     -102     
+ Partials      277      272       -5

Impacted Files	Coverage Δ
src/parser/tokenizer/readWordTree.ts	`100.00% <ø> (ø)`
src/transformers/JSXTransformer.ts	`96.53% <ø> (ø)`
src/parser/traverser/statement.ts	`83.27% <66.66%> (+0.42%)`	⬆️
src/parser/plugins/typescript.ts	`86.40% <100.00%> (+0.27%)`	⬆️
src/parser/tokenizer/index.ts	`85.96% <100.00%> (+1.22%)`	⬆️
src/parser/tokenizer/keywords.ts	`100.00% <100.00%> (ø)`
src/parser/tokenizer/types.ts

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

github-actions · 2022-07-06T23:05:52Z

Benchmark results

Before this PR: 347.3 thousand lines per second
After this PR: 350.7 thousand lines per second

Measured change: 0.97% faster (0.54% slower to 1.55% faster)
Summary: Likely no significant difference

) Instructions: https://github.com/alangpierce/sucrase/wiki/Porting-changes-from-Babel's-parser 69246b6212 [babel 8] fix properties name for function-like TS nodes (#13709) 🚫 AST only. bc0d1ef860 v7.16.8 🚫 Release only. 478a9709ab Improve errors location tracking (#14130) 🚫 Change to error messaging details that were removed in Sucrase. 4a737547e5 v7.16.10 🚫 Release only. e5d29f6e40 fix: incorrect conciseBody lookahead (#14194) ✅ Bug doesn't appear in Sucrase, but I added a test anyway. 5266605528 v7.16.12 🚫 Release only. 5861002b33 Reinterpret << when parsing TS type arguments (#14145) ✅ Ported following a very similar implementation. 6b427ced22 Fuzz test location-related parser options (#14201) 🚫 Babel-internal change testing location logic not present in Sucrase. 96a8251def Add parser support for the "regexp unicode sets" proposal (#14086) ✅ No implementation necessary, but I added a quick test. f893b333a8 Add the `decoratorsAutoAccessors` parser plugin (#13681) ✅ Ported basic parsing for accessor properties. de5c7b1bce Parse destructuring private fields (#13931) ✅ Seems to be already working, and I added a test. 🚫 d50c18dbc2 fix: usePrivateName usage Only affects Babel scope code, not relevant to Sucrase. 🚫 b092bd0cb5 remove invalid test output Only affects Babel tests. df27d542ef proposal-pipe: Add support for `^^` and `@@` topics (#13973) 🚫 Sucrase is holding off on pipeline topic parsing until the details are finalized. Tracked in alangpierce#674. 97a8bcb9cc Expose `.index` on Position to internally track nodes location (#14174) 🚫 Only affects class not used in Sucrase. 38c23cded4 v7.17.0 🚫 Release only. 19ede090eb parser: Update `babel-parser.d.ts` for 7v.17.0 features (#14266) 🚫 Not relevant to Sucrase. f52c70c357 v7.17.3 🚫 Release only. a53c2fa4a2 fix(ts): skip func-type param start on parsing (#14293) 🚫 Bug not present in Sucrase. 5749c16dc2 [tsx] raise error on single arrow type argument without comma (#14135) 🚫 Only affects error handling.

Fixes #758 There was already some logic from #716 to handle cases like `f<<T>() => void>`, where we're transitioning from a non-type context to a type context and need to detect if `<<` is actually two open-type-parameter/argument tokens. There was a simpler missing case, though, which is that `<<` is never allowed within a type, so we should exit early with a simple `<` in a type context. There was already a similar case for `>`, so this just uses the same logic for `<`. This fixes the same issue in TS and Flow.

Fixes #758 There was already some logic from #716 to handle cases like `f<<T>() => void>`, where we're transitioning from a non-type context to a type context and need to detect if `<<` is actually two open-type-parameter/argument tokens. There was a simpler missing case, though, which is that `<<` is never allowed within a type, so we should tokenize as a simple `<` in a type context. To handle the various cases correctly, this extra logic needs to only happen for `<<`, and this PR also adds some additional comments explaining the nuances.

alangpierce added 2 commits July 6, 2022 18:09

Add tests for uncovered lines and fix coverage issue in generated file

74e11d8

Update comment to note that TS doesn't even fully handle <<

268be2a

alangpierce merged commit 173e4fe into main Jul 7, 2022

alangpierce deleted the port-babel-parser-2022-01-10-to-2022-02-26 branch July 7, 2022 20:06

alangpierce mentioned this pull request Nov 27, 2022

Fix parsing of << within a type #769

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Port babel-parser changes from 2022-01-10 to 2022-02-26 #716

Port babel-parser changes from 2022-01-10 to 2022-02-26 #716

alangpierce commented Jul 6, 2022

codecov bot commented Jul 6, 2022 •

edited

Loading

github-actions bot commented Jul 6, 2022 •

edited

Loading

Port babel-parser changes from 2022-01-10 to 2022-02-26 #716

Port babel-parser changes from 2022-01-10 to 2022-02-26 #716

Conversation

alangpierce commented Jul 6, 2022

codecov bot commented Jul 6, 2022 • edited Loading

Codecov Report

github-actions bot commented Jul 6, 2022 • edited Loading

Benchmark results

codecov bot commented Jul 6, 2022 •

edited

Loading

github-actions bot commented Jul 6, 2022 •

edited

Loading