Skip to content

Commit

Permalink
Port all babel-parser changes from 2019-01-28 to 2019-03-26 (#440)
Browse files Browse the repository at this point in the history 
Fixes #437

Details:
828169e61 Fix line continuation with Unicode line terminators (#9403)
🚫 We don't do escape parsing anyway.

4c4c22a31 Run prettier
🚫 Tools only.

00c3e3c8e Fixed link to @babel/parser's issues in README (#9427)
🚫 Docs only.

9eb010da5 Unify reserved word checking and update error messages (#9402)
🚫 Error handling only.

344d35bbe Simplify await and yield tracking in params (#9405)
🚫 Error handling only.

d896ce2b5 v7.3.2
🚫 Release only.

e03e5ba01 Add TypeScript definitions for parser plugin options. (#9457)
🚫 Types only.

07b0f22a3 Fix range for TypeScript optional parameter in arrow function (#9463)
🚫 AST only.

d1514f57b Typescript function destructured params (#9431)
🚫 Already fixed in my code (with a fix that I think is better).

2817844e8 Fix regression with let (#9477)
🚫 This doesn't seem to affect Sucrase.

d349b74a4 Better error output in parser tests (#9491)
🚫 Test only.

4ba998c5d Add importKind to spec
🚫 Types only.

d1fe2d05f v7.3.3
🚫 Release only.

058f05742 Also check AssignmentPatterns for export name (#9521)
🚫 Error handling only.

a1ea765b9 Make tests spec compliant and avoid duplicate declarations in input files (#9522)
🚫 Test only.

dd8b700a2 Parenthesized expressions (#8025)
🚫 New feature that won't go into Sucrase.

9f3457797 Fix TypeScript parsers missing token check (#9571) (#9572)
✅ Added new check with test.

fc1ea7f49 Revert "Parenthesized expressions (#8025)"
🚫 New feature that won't go into Sucrase.

1f6454cc9 v7.3.4
🚫 Release only.

a7391144b Introduce scope tracking in the parser (#9493)
🚫 Looks like this is just for error handling.

e6c1065d1 Fix strict mode prescanning with EmptyStatement (#9585)
🚫 We don't detect strict mode.

d0e196d21 Treat for loop body as part of loop scope (#9586)
🚫 Scopes not tracked in Sucrase in the same way.

0345c1bc1 Use `for..of Object.keys` instead of `for..in` (#9518)
🚫 Not relevant for Sucrase.

244e4580e Remove always false param allowExpressionBody (#9591)
✅ Removed, plus removed some other unnecessary params.

a029071b8 [TS] Correctly forget `await`s after parsing async arrows with type args (#9593)
🚫 Doesn't come up in Sucrase.

e883ff295 Merge pull request #9597 from danez/Update-charcodes
🚫 Mostly just changes in types.

43eed1ac9 Check exported bindings are defined (#9589)
🚫 Error handling only.

5cb280f98 Fix scope check for 2nd+ lexical bindings (#9600)
🚫 Error handling only.

208195f42 Disallow duplicate params in methods (#9599)
🚫 Error handling only.

98ab1b642 Refactor parsing object members (#9607)
🚫 I won't try to port this refactor for now.

f13f4adcb [TS] Disallow type casts in arrow parameters (#9612)
🚫 Error handling only.

17f4195bc Allow any reserved word in `export {} from` specifiers (#9616)
🚫 Already works in Sucrase.

c60c4dd37 Partial Application Syntax: Stage 1 (#9343)
✅ Added basic parsing for ? expression.

d832c0f43 Add parser support for placeholders (#9364)
🚫 I won't include this feature in Sucrase.

54ba6d80c Update identifier parsing per Unicode v12 (#9637)
🚫 Sucrase doesn't need this validation.

29999007f Disallow escape sequences in contextual keywords (#9618)
🚫 We don't support identifier escape sequences in the first place.

fba5655a4 Parenthesized expressions (#8025)
🚫 I won't include this feature in Sucrase.

e53be4b38 [TS] Allow context type annotation on getters/setters (#9641)
🚫 Bug in validation that Sucrase doesn't do.

d8a532983 Reorganize token types and use a map for them (#9645)
🚫 Not really relevant for current Sucrase code.

cf4bd8bb8 Remove input and length from state (#9646)
🚫 Not relevant for Sucrase.

29cd27b54 Partial application plugin (#9474)
🚫 Tests only.

25a3825a1 TypeScript Constant contexts (#9534)
✅ Added test, already works from previous change to make it an identifier.

cc4560842 Add `readonly` to TypeScript type modifier (#9529)
✅ Added new case with test.

48d66eb64 Correctly parse TS TypeAssertions around arrow functions (#9699)
🚫 AST only, I think.

f1328fb91 v7.4.0
🚫 Release only.

ab41cb2cd Fix scope checks with enabled flow plugin (#9719)
🚫 Scope code doesn't exist in Sucrase.

2201fd839 Modules might be in loose mode when checking for undecl exports (#9725)
🚫 Sucrase always uses strict mode.

7dea0f23d v7.4.2
🚫 Release only.

ef0722b4b Fix compatibility between estree and TS plugin (#9700)
🚫 Sucrase doesn't support estree mode.

aaefc83a6 Allow HTML comments on first line (#9760)
🚫 Unclear what this is needed for, but doesn't seem important in Sucrase.

d720c6cff Explicit labels for tokenTypes (#9761)
🚫 Internal change.

444daf922 Optimize parseBindingAtom code to get better error messages (#9762)
🚫 Error checking only.

2867bbf19 [typescript] parsing template literal as type (#9748)
✅ Added new case with test.

7f4427432 Parse right-hand-side of for/of as an assignment expression (#9767)
🚫 Error checking only.

6bc9e7ebd Correctly check for-in and for-of loop for invalid left-hand side (#9768)
🚫 Error checking only.

60d7e940e Fix merge error
🚫 Not relevant to Sucrase.
  • Loading branch information
@alangpierce
alangpierce authored Mar 31, 2019
1 parent e5e9003 commit fd0f689
Show file tree
Hide file tree
Showing 7 changed files with 130 additions and 91 deletions.
17 changes: 6 additions & 11 deletions src/parser/plugins/flow.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -709,18 +709,13 @@ export function flowParseVariance(): void {
// Overrides
// ==================================

export function flowParseFunctionBodyAndFinish(
functionStart: number,
isGenerator: boolean,
allowExpressionBody: boolean = false,
funcContextId: number,
): void {
export function flowParseFunctionBodyAndFinish(funcContextId: number): void {
// For arrow functions, `parseArrow` handles the return type itself.
if (!allowExpressionBody && match(tt.colon)) {
if (match(tt.colon)) {
flowParseTypeAndPredicateInitialiser();
}

parseFunctionBody(functionStart, isGenerator, allowExpressionBody, funcContextId);
parseFunctionBody(false, funcContextId);
}

export function flowParseSubscript(startPos: number, noCalls: boolean, stopState: StopState): void {
Expand Down Expand Up @@ -1029,7 +1024,7 @@ export function flowParseSubscripts(startPos: number, noCalls: boolean = false):
match(tt.lessThan)
) {
const snapshot = state.snapshot();
const wasArrow = parseAsyncArrowWithTypeParameters(startPos);
const wasArrow = parseAsyncArrowWithTypeParameters();
if (wasArrow && !state.error) {
return;
}
Expand All @@ -1040,13 +1035,13 @@ export function flowParseSubscripts(startPos: number, noCalls: boolean = false):
}

// Returns true if there was an arrow function here.
function parseAsyncArrowWithTypeParameters(startPos: number): boolean {
function parseAsyncArrowWithTypeParameters(): boolean {
state.scopeDepth++;
const startTokenIndex = state.tokens.length;
parseFunctionParams();
if (!parseArrow()) {
return false;
}
parseArrowExpression(startPos, startTokenIndex);
parseArrowExpression(startTokenIndex);
return true;
}
30 changes: 15 additions & 15 deletions src/parser/plugins/typescript.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,8 @@ function tsNextTokenCanFollowModifier(): boolean {
!match(tt.parenR) &&
!match(tt.colon) &&
!match(tt.eq) &&
!match(tt.question);
!match(tt.question) &&
!match(tt.bang);

if (canFollowModifier) {
return true;
Expand Down Expand Up @@ -448,6 +449,9 @@ function tsParseNonArrayType(): void {
case tt.parenL:
tsParseParenthesizedType();
return;
case tt.backQuote:
parseTemplate();
return;
default:
if (state.type & TokenType.IS_KEYWORD) {
next();
Expand Down Expand Up @@ -477,7 +481,11 @@ function tsParseInferType(): void {
}

function tsParseTypeOperatorOrHigher(): void {
if (isContextual(ContextualKeyword._keyof) || isContextual(ContextualKeyword._unique)) {
if (
isContextual(ContextualKeyword._keyof) ||
isContextual(ContextualKeyword._unique) ||
isContextual(ContextualKeyword._readonly)
) {
next();
tsParseTypeOperatorOrHigher();
} else if (isContextual(ContextualKeyword._infer)) {
Expand Down Expand Up @@ -969,10 +977,7 @@ function tsTryParseGenericAsyncArrowFunction(): boolean {
return false;
}

// We don't need to be precise about the function start since it's only used if this is a
// bodiless function, which isn't valid here.
const functionStart = state.start;
parseFunctionBody(functionStart, false /* isGenerator */, true);
parseFunctionBody(true);
return true;
}

Expand Down Expand Up @@ -1009,21 +1014,16 @@ export function tsIsDeclarationStart(): boolean {
// OVERRIDES
// ======================================================

export function tsParseFunctionBodyAndFinish(
functionStart: number,
isGenerator: boolean,
allowExpressionBody: boolean = false,
funcContextId: number,
): void {
export function tsParseFunctionBodyAndFinish(functionStart: number, funcContextId: number): void {
// For arrow functions, `parseArrow` handles the return type itself.
if (!allowExpressionBody && match(tt.colon)) {
if (match(tt.colon)) {
tsParseTypeOrTypePredicateAnnotation(tt.colon);
}

// The original code checked the node type to make sure this function type allows a missing
// body, but we skip that to avoid sending around the node type. We instead just use the
// allowExpressionBody boolean to make sure it's not an arrow function.
if (!allowExpressionBody && !match(tt.braceL) && isLineTerminator()) {
if (!match(tt.braceL) && isLineTerminator()) {
// Retroactively mark the function declaration as a type.
let i = state.tokens.length - 1;
while (
Expand All @@ -1038,7 +1038,7 @@ export function tsParseFunctionBodyAndFinish(
return;
}

parseFunctionBody(functionStart, isGenerator, allowExpressionBody, funcContextId);
parseFunctionBody(false, funcContextId);
}

export function tsParseSubscript(startPos: number, noCalls: boolean, stopState: StopState): void {
Expand Down
68 changes: 22 additions & 46 deletions src/parser/traverser/expression.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -382,7 +382,7 @@ function parseAsyncArrowFromCallExpression(functionStart: number, startTokenInde
flowStartParseAsyncArrowFromCallExpression();
}
expect(tt.arrow);
parseArrowExpression(functionStart, startTokenIndex);
parseArrowExpression(startTokenIndex);
}

// Parse a no-call expression (like argument of `new` or `::` operators).
Expand Down Expand Up @@ -450,7 +450,7 @@ export function parseExprAtom(): boolean {
!canInsertSemicolon()
) {
next();
parseFunction(functionStart, false, false);
parseFunction(functionStart, false);
return false;
} else if (
canBeArrow &&
Expand All @@ -462,15 +462,15 @@ export function parseExprAtom(): boolean {
parseBindingIdentifier(false);
expect(tt.arrow);
// let foo = async bar => {};
parseArrowExpression(functionStart, startTokenIndex);
parseArrowExpression(startTokenIndex);
return true;
}

if (canBeArrow && !canInsertSemicolon() && match(tt.arrow)) {
state.scopeDepth++;
markPriorBindingIdentifier(false);
expect(tt.arrow);
parseArrowExpression(functionStart, startTokenIndex);
parseArrowExpression(startTokenIndex);
return true;
}

Expand Down Expand Up @@ -611,12 +611,10 @@ function parseParenAndDistinguishExpression(canBeArrow: boolean): boolean {
// get proper token annotations.
state.restoreFromSnapshot(snapshot);
state.scopeDepth++;
// We don't need to worry about functionStart for arrow functions, so just use something.
const functionStart = state.start;
// Don't specify a context ID because arrow functions don't need a context ID.
parseFunctionParams();
parseArrow();
parseArrowExpression(functionStart, startTokenIndex);
parseArrowExpression(startTokenIndex);
return true;
}
}
Expand Down Expand Up @@ -751,7 +749,7 @@ export function parseObj(isPattern: boolean, isBlockScope: boolean): void {
parsePropertyName(contextId);
}

parseObjPropValue(isGenerator, isPattern, isBlockScope, contextId);
parseObjPropValue(isPattern, isBlockScope, contextId);
}

state.tokens[state.tokens.length - 1].contextId = contextId;
Expand All @@ -771,23 +769,19 @@ function isGetterOrSetterMethod(isPattern: boolean): boolean {
}

// Returns true if this was a method.
function parseObjectMethod(
isGenerator: boolean,
isPattern: boolean,
objectContextId: number,
): boolean {
function parseObjectMethod(isPattern: boolean, objectContextId: number): boolean {
// We don't need to worry about modifiers because object methods can't have optional bodies, so
// the start will never be used.
const functionStart = state.start;
if (match(tt.parenL)) {
if (isPattern) unexpected();
parseMethod(functionStart, isGenerator, /* isConstructor */ false);
parseMethod(functionStart, /* isConstructor */ false);
return true;
}

if (isGetterOrSetterMethod(isPattern)) {
parsePropertyName(objectContextId);
parseMethod(functionStart, /* isGenerator */ false, /* isConstructor */ false);
parseMethod(functionStart, /* isConstructor */ false);
return true;
}
return false;
Expand Down Expand Up @@ -822,7 +816,6 @@ function parseObjectProperty(isPattern: boolean, isBlockScope: boolean): void {
}

function parseObjPropValue(
isGenerator: boolean,
isPattern: boolean,
isBlockScope: boolean,
objectContextId: number,
Expand All @@ -832,7 +825,7 @@ function parseObjPropValue(
} else if (isFlowEnabled) {
flowStartParseObjPropValue();
}
const wasMethod = parseObjectMethod(isGenerator, isPattern, objectContextId);
const wasMethod = parseObjectMethod(isPattern, objectContextId);
if (!wasMethod) {
parseObjectProperty(isPattern, isBlockScope);
}
Expand Down Expand Up @@ -860,23 +853,14 @@ export function parsePropertyName(objectContextId: number): void {
}

// Parse object or class method.
export function parseMethod(
functionStart: number,
isGenerator: boolean,
isConstructor: boolean,
): void {
export function parseMethod(functionStart: number, isConstructor: boolean): void {
const funcContextId = getNextContextId();

state.scopeDepth++;
const startTokenIndex = state.tokens.length;
const allowModifiers = isConstructor; // For TypeScript parameter properties
parseFunctionParams(allowModifiers, funcContextId);
parseFunctionBodyAndFinish(
functionStart,
isGenerator,
false /* allowExpressionBody */,
funcContextId,
);
parseFunctionBodyAndFinish(functionStart, funcContextId);
const endTokenIndex = state.tokens.length;
state.scopes.push(new Scope(startTokenIndex, endTokenIndex, true));
state.scopeDepth--;
Expand All @@ -885,35 +869,24 @@ export function parseMethod(
// Parse arrow function expression.
// If the parameters are provided, they will be converted to an
// assignable list.
export function parseArrowExpression(functionStart: number, startTokenIndex: number): void {
parseFunctionBody(functionStart, false /* isGenerator */, true);
export function parseArrowExpression(startTokenIndex: number): void {
parseFunctionBody(true);
const endTokenIndex = state.tokens.length;
state.scopes.push(new Scope(startTokenIndex, endTokenIndex, true));
state.scopeDepth--;
}

export function parseFunctionBodyAndFinish(
functionStart: number,
isGenerator: boolean,
allowExpressionBody: boolean = false,
funcContextId: number = 0,
): void {
export function parseFunctionBodyAndFinish(functionStart: number, funcContextId: number = 0): void {
if (isTypeScriptEnabled) {
tsParseFunctionBodyAndFinish(functionStart, isGenerator, allowExpressionBody, funcContextId);
tsParseFunctionBodyAndFinish(functionStart, funcContextId);
} else if (isFlowEnabled) {
flowParseFunctionBodyAndFinish(functionStart, isGenerator, allowExpressionBody, funcContextId);
flowParseFunctionBodyAndFinish(funcContextId);
} else {
parseFunctionBody(functionStart, isGenerator, allowExpressionBody, funcContextId);
parseFunctionBody(false, funcContextId);
}
}

// Parse function body and check parameters.
export function parseFunctionBody(
functionStart: number,
isGenerator: boolean,
allowExpression: boolean,
funcContextId: number = 0,
): void {
export function parseFunctionBody(allowExpression: boolean, funcContextId: number = 0): void {
const isExpression = allowExpression && !match(tt.braceL);

if (isExpression) {
Expand Down Expand Up @@ -948,6 +921,9 @@ function parseExprListItem(allowEmpty: boolean): void {
} else if (match(tt.ellipsis)) {
parseSpread();
parseParenItem();
} else if (match(tt.question)) {
// Partial function application proposal.
next();
} else {
parseMaybeAssign(false, true);
}
Expand Down
Loading

0 comments on commit fd0f689

Please sign in to comment.