Python: ../array.h file not found

[dougthor42]

When trying to update rules_python's Gazelle plugin to support python 3.12, I found that (I think) we need to update go-tree-sitter.

We bump:

-- a/gazelle/go.mod
+++ b/gazelle/go.mod
@@ -9,7 +9,7 @@ require (
        github.com/bmatcuk/doublestar/v4 v4.6.1
        github.com/emirpasic/gods v1.18.1
        github.com/ghodss/yaml v1.0.0
-       github.com/smacker/go-tree-sitter v0.0.0-20240422154435-0628b34cbf9c
+       github.com/smacker/go-tree-sitter v0.0.0-20240827094217-dd81d9e9be82
        github.com/stretchr/testify v1.9.0
        golang.org/x/sync v0.2.0
        gopkg.in/yaml.v2 v2.4.0

However, in the source code for gazelle, I see this comment:

// at github.com/smacker/go-tree-sitter@latest (after v0.0.0-20240422154435-0628b34cbf9c we used)
// "__main__" is the second child of b. But now, it isn't.
// we cannot use the latest go-tree-sitter because of the top level reference in scanner.c.
// https://github.com/smacker/go-tree-sitter/blob/04d6b33fe138a98075210f5b770482ded024dc0f/python/scanner.c#L1

I find this same thing when trying to run with the newer version of go-tree-sitter:

external/gazelle~~go_deps~com_github_smacker_go_tree_sitter/python/scanner.c:1:10: fatal error: '../array.h' file not found
    1 | #include "../array.h"
      |          ^~~~~~~~~~~~
1 error generated.

Full Bazel logs

$ bazel run //:gazelle                                                                                     
INFO: Invocation ID: 52faab94-019d-4e81-a99a-8b4f6499ac2a                                               
WARNING: For repository 'bazel_skylib', the root module requires module version [email protected], but got [email protected] in the resolved dependency graph.
INFO: Analyzed target //:gazelle (6 packages loaded, 72 targets configured).
ERROR: /usr/local/google/home/dthor/.cache/bazel/_bazel_dthor/dbe74c4144b5c9a438d84a119652bef9/external/gazelle~~go_deps~com_github_smacker_go_tree_sitter/python/BUILD.bazel:3:11: GoCompilePkg external/gazelle~~go_deps~com_github_smacker_go_tree_sitter/python/python.a [for tool] failed: (Exit 1): builder failed: error executing GoCompilePkg command (from target @@gazelle~~go_deps~com_github_smacker_go_tree_sitter//python:python) bazel-out/k8-opt-exec-ST-7ac76b60419a/bin/external/rules_go~~go_sdk~rules_helm__download_0/builder_reset/builder compilepkg -sdk external/rules_go~~go_sdk~rules_helm__download_0 -installsuffix ... (remaining 43 arguments skipped)
external/gazelle~~go_deps~com_github_smacker_go_tree_sitter/python/scanner.c:1:10: fatal error: '../array.h' file not found
    1 | #include "../array.h"
      |          ^~~~~~~~~~~~
1 error generated.
compilepkg: error running subcommand PATH=/usr/bin:/usr/lib/llvm-18/bin:/bin \
HOSTNAME=localhost \
HOST_INPUT_ROOT=/b/r/w \
GOPATH= \
CC=/usr/lib/llvm-18/bin/clang \
GOROOT_FINAL=GOROOT \
ZERO_AR_DATE=1 \
HOST_NETWORK_NAME=network-name-dont-hardcode-use-HOST_NETWORK_NAME-env-var \
CGO_ENABLED=1 \
HOST_CONTAINER_NAME=rbe-container-fc05ec71-47d6-4b90-94af-e052acdec4a8 \
GOARCH=amd64 \
GOEXPERIMENT= \
GOOS=linux \
GOROOT=external/rules_go~~go_sdk~rules_helm__download_0 \
GOTOOLCHAIN=local \
HOME=/home/nobody \
CGO_LDFLAGS="-fuse-ld=/usr/bin/ld.gold -B/usr/lib/llvm-18/bin -Wl,-no-as-needed -Wl,-z,relro,-z,now -Wl,--push-state,-as-needed -Wl,--pop-state -Wl,--push-state,-as-needed -lm -Wl,--pop-state -pthread" \
/usr/lib/llvm-18/bin/clang -I external/gazelle~~go_deps~com_github_smacker_go_tree_sitter/python -iquote . -iquote /b/f/w/external/gazelle~~go_deps~com_github_smacker_go_tree_sitter/python -iquote /tmp/rules_go_work-2733723449/cgo/github.com/smacker/go-tree-sitter/python -U_FORTIFY_SOURCE -fstack-protector -Wthread-safety -Wself-assign -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -O2 -D_FORTIFY_SOURCE=1 -DNDEBUG -ffunction-sections -fdata-sections -no-canonical-prefixes -Wno-builtin-macro-redefined -D__DATE__=redacted -D__TIMESTAMP__=redacted -D__TIME__=redacted -fPIC -fdebug-prefix-map=/b/f/w=. -fdebug-prefix-map=/tmp/rules_go_work-2733723449/cgo/github.com/smacker/go-tree-sitter/python=. -pthread -c /b/f/w/external/gazelle~~go_deps~com_github_smacker_go_tree_sitter/python/scanner.c -o /tmp/rules_go_work-2733723449/cgo/github.com/smacker/go-tree-sitter/python/_x3.o: exit status 1
Target //:gazelle failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 33.140s, Critical Path: 28.43s
INFO: 3 processes: 2 internal, 1 remote.
ERROR: Build did NOT complete successfully
ERROR: Build failed. Not running target