Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Block Serialization Default Parser: Include TypeScript type declarations #43722

Merged
merged 1 commit into from
Aug 31, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions packages/block-serialization-default-parser/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@

## Unreleased

### New Feature

- Include TypeScript type declarations ([#43722](https://github.com/WordPress/gutenberg/pull/43722)).

## 4.16.0 (2022-08-24)

## 4.15.0 (2022-08-10)
Expand Down
2 changes: 1 addition & 1 deletion packages/block-serialization-default-parser/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ _Parameters_

_Returns_

- `Array`: A block-based representation of the input HTML.
- `ParsedBlock[]`: A block-based representation of the input HTML.

<!-- END TOKEN(Autogenerated API docs) -->

Expand Down
1 change: 1 addition & 0 deletions packages/block-serialization-default-parser/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
"main": "build/index.js",
"module": "build-module/index.js",
"react-native": "src/index",
"types": "build-types",
"sideEffects": false,
"dependencies": {
"@babel/runtime": "^7.16.0"
Expand Down
161 changes: 129 additions & 32 deletions packages/block-serialization-default-parser/src/index.js
Original file line number Diff line number Diff line change
@@ -1,8 +1,50 @@
/**
* @type {string}
*/
let document;
/**
* @type {number}
*/
let offset;
/**
* @type {ParsedBlock[]}
*/
let output;
/**
* @type {ParsedFrame[]}
*/
let stack;

/**
* @typedef {Object|null} Attributes
*/

/**
* @typedef {Object} ParsedBlock
* @property {string|null} blockName Block name.
* @property {Attributes} attrs Block attributes.
* @property {ParsedBlock[]} innerBlocks Inner blocks.
* @property {string} innerHTML Inner HTML.
* @property {Array<string|null>} innerContent Inner content.
*/

/**
* @typedef {Object} ParsedFrame
* @property {ParsedBlock} block Block.
* @property {number} tokenStart Token start.
* @property {number} tokenLength Token length.
* @property {number} prevOffset Previous offset.
* @property {number|null} leadingHtmlStart Leading HTML start.
*/

/**
* @typedef {'void-block'|'block-opener'|'block-closer'} TokenType
*/

/**
* @typedef {[TokenType, string, Attributes, number, number]} Token
*/

/**
* Matches block comment delimiters
*
Expand Down Expand Up @@ -47,6 +89,16 @@ let stack;
const tokenizer =
/<!--\s+(\/)?wp:([a-z][a-z0-9_-]*\/)?([a-z][a-z0-9_-]*)\s+({(?:(?=([^}]+|}+(?=})|(?!}\s+\/?-->)[^])*)\5|[^]*?)}\s+)?(\/)?-->/g;

/**
* Constructs a block object.
*
* @param {string|null} blockName
* @param {Attributes} attrs
* @param {ParsedBlock[]} innerBlocks
* @param {string} innerHTML
* @param {string[]} innerContent
* @return {ParsedBlock} The block object.
*/
function Block( blockName, attrs, innerBlocks, innerHTML, innerContent ) {
return {
blockName,
Expand All @@ -57,10 +109,26 @@ function Block( blockName, attrs, innerBlocks, innerHTML, innerContent ) {
};
}

/**
* Constructs a freeform block object.
*
* @param {string} innerHTML
* @return {ParsedBlock} The freeform block object.
*/
function Freeform( innerHTML ) {
return Block( null, {}, [], innerHTML, [ innerHTML ] );
}

/**
* Constructs a frame object.
*
* @param {ParsedBlock} block
* @param {number} tokenStart
* @param {number} tokenLength
* @param {number} prevOffset
* @param {number|null} leadingHtmlStart
* @return {ParsedFrame} The frame object.
*/
function Frame( block, tokenStart, tokenLength, prevOffset, leadingHtmlStart ) {
return {
block,
Expand Down Expand Up @@ -146,7 +214,7 @@ function Frame( block, tokenStart, tokenLength, prevOffset, leadingHtmlStart ) {
* }
* ];
* ```
* @return {Array} A block-based representation of the input HTML.
* @return {ParsedBlock[]} A block-based representation of the input HTML.
*/
export const parse = ( doc ) => {
document = doc;
Expand All @@ -162,42 +230,47 @@ export const parse = ( doc ) => {
return output;
};

/**
* Parses the next token in the input document.
*
* @return {boolean} Returns true when there is more tokens to parse.
*/
function proceed() {
const stackDepth = stack.length;
const next = nextToken();
if ( next === null ) {
Copy link
Member Author

@gziolo gziolo Aug 31, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This refactoring was driven by TypeScript that wasn't quite happy about the token being an array with 1 item or 5 items. It turns out that when using null instead the array with 1 string item, the behavior remains the same, and we can bail out earlier to avoid other operations that aren't necessary for this case anyway.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did you look at using [TokenType, string?, Attributes?, number?, number?]?

the refactor looks fine but I'd rather we return 'no-more-tokens' than null because of the self-documenting nature of the string value.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that was my initial try, but TS was complaining about the number of items in the array returned: 1 instead of 5. Maybe return [ 'no-more-tokens', , , , ]; would work then?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I opened #44459 to address it.

// If not in a block then flush output.
if ( 0 === stackDepth ) {
addFreeform();
return false;
}

// Otherwise we have a problem
// This is an error
// we have options
// - treat it all as freeform text
// - assume an implicit closer (easiest when not nesting)

// For the easy case we'll assume an implicit closer.
if ( 1 === stackDepth ) {
addBlockFromStack();
return false;
}

// For the nested case where it's more difficult we'll
// have to assume that multiple closers are missing
// and so we'll collapse the whole stack piecewise.
while ( 0 < stack.length ) {
addBlockFromStack();
}
return false;
}
const [ tokenType, blockName, attrs, startOffset, tokenLength ] = next;
const stackDepth = stack.length;

// We may have some HTML soup before the next block.
const leadingHtmlStart = startOffset > offset ? offset : null;

switch ( tokenType ) {
case 'no-more-tokens':
// If not in a block then flush output.
if ( 0 === stackDepth ) {
addFreeform();
return false;
}

// Otherwise we have a problem
// This is an error
// we have options
// - treat it all as freeform text
// - assume an implicit closer (easiest when not nesting)

// For the easy case we'll assume an implicit closer.
if ( 1 === stackDepth ) {
addBlockFromStack();
return false;
}

// For the nested case where it's more difficult we'll
// have to assume that multiple closers are missing
// and so we'll collapse the whole stack piecewise.
while ( 0 < stack.length ) {
addBlockFromStack();
}
return false;

case 'void-block':
// easy case is if we stumbled upon a void block
// in the top-level of the document.
Expand Down Expand Up @@ -261,7 +334,7 @@ function proceed() {

// Otherwise we're nested and we have to close out the current
// block and add it as a innerBlock to the parent.
const stackTop = stack.pop();
const stackTop = /** @type {ParsedFrame} */ ( stack.pop() );
const html = document.substr(
stackTop.prevOffset,
startOffset - stackTop.prevOffset
Expand Down Expand Up @@ -304,6 +377,11 @@ function parseJSON( input ) {
}
}

/**
* Finds the next token in the document.
*
* @return {Token|null} The next matched token.
*/
function nextToken() {
// Aye the magic
// we're using a single RegExp to tokenize the block comment delimiters
Expand All @@ -315,7 +393,7 @@ function nextToken() {

// We have no more tokens.
if ( null === matches ) {
return [ 'no-more-tokens' ];
return null;
}

const startedAt = matches.index;
Expand Down Expand Up @@ -355,6 +433,11 @@ function nextToken() {
return [ 'block-opener', name, attrs, startedAt, length ];
}

/**
* Adds a freeform block to the output.
*
* @param {number} [rawLength]
*/
function addFreeform( rawLength ) {
const length = rawLength ? rawLength : document.length - offset;

Expand All @@ -365,6 +448,14 @@ function addFreeform( rawLength ) {
output.push( Freeform( document.substr( offset, length ) ) );
}

/**
* Adds inner block to the parent block.
*
* @param {ParsedBlock} block
* @param {number} tokenStart
* @param {number} tokenLength
* @param {number} [lastOffset]
*/
function addInnerBlock( block, tokenStart, tokenLength, lastOffset ) {
const parent = stack[ stack.length - 1 ];
parent.block.innerBlocks.push( block );
Expand All @@ -382,8 +473,14 @@ function addInnerBlock( block, tokenStart, tokenLength, lastOffset ) {
parent.prevOffset = lastOffset ? lastOffset : tokenStart + tokenLength;
}

/**
* Adds block from the stack to the output.
*
* @param {number} [endOffset]
*/
function addBlockFromStack( endOffset ) {
const { block, leadingHtmlStart, prevOffset, tokenStart } = stack.pop();
const { block, leadingHtmlStart, prevOffset, tokenStart } =
/** @type {ParsedFrame} */ ( stack.pop() );

const html = endOffset
? document.substr( prevOffset, endOffset - prevOffset )
Expand Down
8 changes: 8 additions & 0 deletions packages/block-serialization-default-parser/tsconfig.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"extends": "../../tsconfig.base.json",
"compilerOptions": {
"rootDir": "src",
"declarationDir": "build-types"
},
"include": [ "src/**/*" ]
}
1 change: 1 addition & 0 deletions tsconfig.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
{ "path": "packages/autop" },
{ "path": "packages/blob" },
{ "path": "packages/block-editor" },
{ "path": "packages/block-serialization-default-parser" },
{ "path": "packages/components" },
{ "path": "packages/compose" },
{ "path": "packages/core-data" },
Expand Down