Python Syntax Highlighting is incomplete compared to VSCode?

[Zeddicus414]

Hi all,

Thanks for making Helix! It's really cool and I've been trying to set it up as my daily driver for python. One thing I've been trying to "fix" and I don't know where to look. Helix seems to be missing some distinctions in the syntax highlighting.

Helix	VSCode

In particular, notice the greenish types definitions on Error(Exception) and self.msg: str. Also, the purple control words, if, else, return

Is there some upstream library (maybe treesitter?) causing the lack of more detailed syntax highlighting found in VSCode?

I know the colors are configured in the dark_plus.toml theme file, so I don't think it's that:

Thanks!

[sudormrfbin]

(I edited the images in the question to be side by side so that it's easier to see the distinction if that's okay with you.)

The problem is that the highlight captures for python aren't as detailed as it could be, for example control keywords like if, else, are grouped under the keyword highlight group instead of the more specific keyword.control highlight group (which is assigned a different color in the dark plus theme):

helix/runtime/queries/python/highlights.scm

Lines 91 to 121 in ee889aa

	[
	"as"
	"assert"
	"async"
	"await"
	"break"
	"class"
	"continue"
	"def"
	"del"
	"elif"
	"else"
	"except"
	"exec"
	"finally"
	"for"
	"from"
	"global"
	"if"
	"import"
	"lambda"
	"nonlocal"
	"pass"
	"print"
	"raise"
	"return"
	"try"
	"while"
	"with"
	"yield"
	] @keyword

As for the type names of Error and Exception they actually are different shade of blue than the highlighting of class, so I think that is more of a deviation with the theme definition of dark plus in helix.

For highlighting the str in self.msg: str properly we would need better tree sitter queries in the highlights file for python.

[Zeddicus414]

Ahh, just changing the highlights.scm to have a @keyword.control section makes a big difference. This is now from helix:

[Zeddicus414]

I am definitely in a place to start understanding this much better with a path forward for getting to where I want to be. Thank you!

[Zeddicus414]

Alright, this is where we are so far with help from @sudormrfbin. See the attached highlights.scm file.

highlights.txt

Helix	VSCode

Some improvements:

• Types are green (line 6 & 25). This was done by placing the type definition more toward the top of the SCM file
• keyword.control allows most of the purple words that really increases readability, IMHO.

A few interesting things:

• I figured out that the class name (line 21) is being classified as a @constant, but I haven't figured out how to separate the class definition from an actual CONSTANT definition at the top of the code. In this case, I just made constants green too. I think it looks better. Notice the CONSTANT on line 3.
• VSCode makes imported modules the green color. I haven't found a way to duplicate that. See lines 1 & 19.
• VSCode is smart enough to tell when "in" is an operator vs part of the "for x in". See lines 8 & 9.

This is definitely close enough for my purposes. If I want to commit these changes to the projects, is that okay? I'm not sure if this can mess up any of the other themes. I think it's a clear improvement for this one. It's certainly good enough that I feel comfortable using it.

[archseer]

Feel free to open a PR!

[valpackett]

tell when "in" is an operator vs part of the "for x in"

Quite easy to do here as well!

(for_statement "in" @ui.linenr)
(for_in_clause "in" @ui.cursor.primary)

(place these earlier than the general "in" query)

(obviously use more sensible scopes, this is just for demonstration)

[Zeddicus414]

Ahh, that makes sense. Thank you!

I'm not particularly familiar with the treesitter scm syntax, but I'm getting there.

[qbedard]

Neovim's tree-sitter queries also seem better:
https://github.com/nvim-treesitter/nvim-treesitter/blob/master/queries/python/highlights.scm

Is there a reason why we wouldn't want to use some unified set of queries between different editors? Are tree-sitter queries necessarily editor-specific?

[David-Else]

See the lightbulb: https://docs.helix-editor.com/master/guides/adding_languages.html?highlight=neovim#queries