Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
refactor(tokenizer): Introduce sequences & fast forwarding (#1007)
Sequences allow us to remove a lot of repetitive states. Fast forwarding allows us to skip ahead in the buffer, if we are only looking for one specific character. Also fixes a bug where immediately closed comments (eg. `<!-->`) wouldn't be recognised.
- Loading branch information