Gazelle is >5x slower after #1022 merged

[wolfd]

What version of gazelle are you using?

current master (.bazelignore change #1022): d1bb564
previous commit (#1024 ): 876c759

What version of rules_go are you using?

v0.25.1

What version of Bazel are you using?

4.0.0

What operating system and processor architecture are you using?

Ubuntu Linux x86_64

What did you do?

Swapped out gazelle version in workspace and ran our gazelle tooling under both versions:

current master (.bazelignore change #1022): d1bb564

bazel run --run_under="time" //tools/gazelle

real    0m15.114s
user    0m21.834s
sys     0m0.578s

previous commit (#1024): 876c759

bazel run --run_under="time" //tools/gazelle

real    0m1.978s
user    0m2.938s
sys     0m0.385s

For what it's worth, our .bazelignore has ~50 items.

For now, I've chosen to run 876c759, but the performance regression in d1bb564 is giving me pause to updating to latest.

[wolfd]

Looking a bit into this, I think d1bb564#r49128622 is causing this issue. Is there a good place to cache that information?

[danny-skydio]

Dug a bit more into this, and it seems that the issue is more to do with how the # gazelle:exclude xyz feature works. Caching the parsed version of .bazelignore only marginally improved wall-clock time in my latest testing.

Some more info:

• .bazelignore is all relative paths from the repo root (at least currently, No way to tell .bazelignore to ignore all folders of the given name bazelbuild/bazel#8106 would change this, but who knows when that might happen).
• excludes added globally always check, for every single file. I assume this is to support ** globbing.

I suspect that a fix here is to have a more complicated implementation of excludes.

For .bazelignore, we could probably generate a mapping, and check if these are files or folders once upon loading. Walk could potentially look up items in that static map to exclude them, without looping over each item in the .bazelignore for every file in the source tree.

I also suspect that a similar change could be made to regular # gazelle:exclude xyz behavior, where the expression is checked to see if it has ** globs or / separators, and otherwise just restricting the exclude to the same directory. I think this would be equivalent behavior, but I'm not certain. Making this change would also fix the .bazelignore performance hit.

[jayconrod]

Marking this as a bug. Thanks for investigating this. It sounds like the best thing to do is to improve the exclude implementation rather than roll back bazelignore support.