ESQL analyzer slow on many fields

[dnhatn]

ESQL Analyzer seems to be slow on analyzing many fields. It took more than 30 seconds to analyze 2000 fields in the testHugeManyConcat. In our CI this sometimes this sometimes takes more than 5 minutes.

[FROM manylongs | EVAL str = CONCAT(TO_STRING(a), "  ", TO_STRING(b), "  ", TO_STRING(c), "  ", TO_STRING(d), "  ", TO_STRING(e))
| EVAL 
str0=CONCAT(str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str),
str1=CONCAT(str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str),
str2=CONCAT(str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str),
str3=CONCAT(str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str),

...

str1996=CONCAT(str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str),
str1997=CONCAT(str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str),
str1998=CONCAT(str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str),
str1999=CONCAT(str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str, str)

| KEEP str0, str1, str2, str3, ... str1996, str1997, str1998, str1999

query.txt

[elasticsearchmachine]

Pinging @elastic/es-analytics-geo (Team:Analytics)

[dnhatn]

I have opened a quick fix in #104246. Feel free to take over this issue after that PR.

[dnhatn]

I've made this O(N^2) from O(N^3) in #104246. I think we should make this linear (at least for such cases).

[astefan]

@dnhatn thank you for spotting this one.
Regarding the PR's description

Resolving 2000 fields can take 35 seconds on a fast machine and more than 5 minutes on a slow machine. Although I think we should try to make this linear if possible, this quick fix only changes the resolution to O(N^2). This reduces the resolution time from 35s to 170ms (200 times faster) for 2000 fields. This is good enough to re-enable the HeapAttack tests.

and the recorded times, can we do a bit more digging into this? For one, I am curious on what was the previous performance (before PR #103316) for the same amount of fields. Going from 35 seconds to 170 ms is amazing, but there was a third non-ideal solution (before #103316 which is a bug fix, but I don't remember the details of the bug) that we don't have any execution times for.