Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scanner token parsing refactor #353

Merged
merged 7 commits into from
Oct 11, 2021
Merged

Scanner token parsing refactor #353

merged 7 commits into from
Oct 11, 2021

Commits on Oct 11, 2021

  1. Refactor scanner to emit more granular tokens

    The DOMAIN token (among others) has been removed in favour of WORD, UWORD and other tokens. SCHEME (formerly PROTOCOL) tokens now come in several flavours. Includes additional facilities for token groups, which will be useful for future plugins down the line
    
    Also fixes file URL behaviour
    nfrasser committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    53c77fd View commit details
    Browse the repository at this point in the history
  2. Update plugins to use new scanner tokens

    Also improves hashtag and mention plugin accuracy
    nfrasser committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    be55ebd View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    f464ec5 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    3f7a070 View commit details
    Browse the repository at this point in the history
  5. Update benchmark require

    nfrasser committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    856ea99 View commit details
    Browse the repository at this point in the history
  6. Remove resolved FIXME

    nfrasser committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    78fab92 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    6a0a7fa View commit details
    Browse the repository at this point in the history