Skip to content

Commit

Permalink
refactor(tokenizer): Introduce sequences & fast forwarding (#1007)
Browse files Browse the repository at this point in the history
Sequences allow us to remove a lot of repetitive states. Fast forwarding allows us to skip ahead in the buffer, if we are only looking for one specific character.

Also fixes a bug where immediately closed comments (eg. `<!-->`) wouldn't be recognised.
  • Loading branch information
fb55 authored Nov 11, 2021
1 parent 3adc0e7 commit 6a1b54d
Showing 1 changed file with 294 additions and 415 deletions.
Loading

0 comments on commit 6a1b54d

Please sign in to comment.