Weird Python highlighting

[ubitux]

Summary

The following Python code has very weird highlights:

K = 0
X = 0
XY = 0
k = 0
XYZ = 0
ZF00D = 0
zf00d = 0
BAR = 0
KV = 0
A0 = 0
_A0 = 0

For some reason, helix seems to have its own set of custom highlights in runtime/queries/python/highlights.scm (I'm assuming ((identifier) @constant (#match? @constant "^[A-Z_]{2,}$")) is at least partly responsible for this.

Why does helix need such a file though? Shouldn't tree-sitter be enough?

Reproduction Steps

I tried this:

1. hx /tmp/x.py
2. enter the above snippet

I expected this to happen: same color for all identifiers (at least the same for all capitalized ones)

Instead, this happened:

Helix log

~/.cache/helix/helix.log

2023-04-12T23:25:56.536 helix_vcs [ERROR] Error {
    context: "failed to open git repo",
    source: Discover(
        NoGitRepositoryWithinFs {
            path: "/tmp",
            limit: "/",
        },
    ),
}
2023-04-12T23:25:56.536 helix_vcs [ERROR] failed to open diff base for /tmp/x.py
2023-04-12T23:25:56.537 helix_vcs [ERROR] Error {
    context: "failed to open git repo",
    source: Discover(
        NoGitRepositoryWithinFs {
            path: "/tmp",
            limit: "/",
        },
    ),
}
2023-04-12T23:25:56.537 helix_vcs [ERROR] failed to obtain current head name for /tmp/x.py
2023-04-12T23:25:56.676 helix_term::application [ERROR] Language Server: Received malformed method call workspace/configuration in request 0: protocol error: InvalidParams: Invalid params: invalid value: string "", expected relative URL without a base.

Platform

Linux

Terminal Emulator

alacritty 0.12.0 (5a728195)

Helix Version

helix 23.03

[the-mikedavis]

It looks like that regex for highlighting constants is too strict:

helix/runtime/queries/python/highlights.scm

Lines 96 to 97 in 161fef2

	((identifier) @constant
	(#match? @constant "^[A-Z_]{2,}$"))

The python style guide (here) isn't explicit about it but I think constant names can include digits and could be single-character. So that regex could change to something more permissive like ^[A-Z][A-Z\\d_]*$ which we use for a few other languages IIRC.

[pascalkuthe]

A bit if a meta comment but it seems that githubs own syntax highlighting also doesn't highlight _A0, zf00d and k so it seems like that lowercase identifiers and identifiers starting with _ are generally not highkighted as constamts (which matches my expections regarding python naming conventions}

[ubitux]

It looks like the highlights.scm is severely outdated compared to the one distributed by tree-sitter-python at the revision indicated in languages.toml. Is the synchronization supposed to be done manually regularly? 🤔

[the-mikedavis]

Feel free to open a PR here or upstream to copy over any patterns you think would be an improvement. The highlights.scm in a grammar repository are usually just a reference and we do a lot of customization in the highlights.scm we have in this repository under runtime/queries/<lang>/highlights.scm

[Boscop]

I get some weird highlighting in Rust, notice both occurrences of the in the comment and then unwrap: