Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: handle interpolations without spaces and $$ in interpolated strings #324

Merged
merged 7 commits into from
Jul 23, 2023

Conversation

johannescoetzee
Copy link
Contributor

@johannescoetzee johannescoetzee commented Jul 20, 2023

The Scala syntax summary doesn't describe the $identifier-type interpolations in interpolated/processed strings correctly. While a$b_<operator_chars> is a valid identifier name, in the interpolated string s"$a$b_<operator_chars>, this is actually treated as two identifier interpolations $a and $b_, followed by the string part <operator_chars>.

To fix this, I've added a new rule, _interpolation_identifier to describe identifiers in simple interpolations that matches the regex /[A-Za-z_][A-Z_a-z0-9]/; instead of the full regex in _alpha_identifier. This does mean that queries relating to interpolated strings will need to be updated, but the changes to the queries in this repo are minimal. I've aliased the rule to $.identifier instead, so this will work without query changes.

This also includes a smaller change to handle $$ patterns in interpolated strings (which evaluate to just $) without errors. These don't appear in the tree, similar to regular escape sequences.

@eed3si9n
Copy link
Collaborator

Thanks for the contribution @johannescoetzee!

Could you provide some highlighting tests under here plz?

val hello2 = c"some $meth"
// ^method

@johannescoetzee
Copy link
Contributor Author

@eed3si9n Done! I think the added test covers everything, but please let me know if I should add more.

Copy link
Collaborator

@eed3si9n eed3si9n left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@eed3si9n
Copy link
Collaborator

eed3si9n commented Jul 20, 2023

Note Different editors maintain their own queries for tree-sitter-scala (#283), so:

  1. after this lands, we need to propagate the new highlight queries to different repos to take effect
  2. however, since this adds a new token interpolation_identifier, if the end user somehow gets the updated queries but not the updated tree-sitter-scala, it may crash their editor?

@johannescoetzee
Copy link
Contributor Author

however, since this adds a new token interpolation_identifier, if the end user somehow gets the updated queries but not the updated tree-sitter-scala, it may crash their editor?

There are 2 potential crashes when updating tree-sitter-scala and the queries:

  1. If the queries are updated first, tree-sitter will crash on the unknown token.
  2. If tree-sitter-scala is updated first, then at least nvim-treesitter crashes since (interpolation (identifier)) is no longer a possible pattern.

@eed3si9n
Copy link
Collaborator

If tree-sitter-scala is updated first, then at least nvim-treesitter crashes since (interpolation (identifier)) is no longer a possible pattern.

Could we somehow prevent this by creating a dummy token that can return (interpolation (identifier))? By nature of the tree-sitter, the parser becomes available immediately when the *.c is checked into the master branch.

@johannescoetzee
Copy link
Contributor Author

It turns out it's possible to alias rules, so I've pushed a change to alias $._interpolation_identifier to $.identifier. This means queries won't have to be updated at all.

@johannescoetzee
Copy link
Contributor Author

@eed3si9n I realised this morning that this doesn't handle the $" dollar escape, so I've updated the dollar_escape rule to include that case. To confirm that it's valid, I tested both scala 2 and 3 (results only shown for scala 3)

❯ scala
Welcome to Scala 3.3.0 (19.0.2, Java OpenJDK 64-Bit Server VM).
Type in expressions for evaluation. Or try :help.
                                                                                                                                                               
scala> val a = 2
val a: Int = 2
                                                                                                                                                               
scala> s"$"$a"
val res0: String = "2

@eed3si9n eed3si9n merged commit 8cdd8bd into tree-sitter:master Jul 23, 2023
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants