Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ESQL: Ensure data types in field attributes are always widened #112691

Open
alex-spies opened this issue Sep 10, 2024 · 1 comment
Open

ESQL: Ensure data types in field attributes are always widened #112691

alex-spies opened this issue Sep 10, 2024 · 1 comment
Assignees
Labels
:Analytics/ES|QL AKA ESQL Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) >tech debt

Comments

@alex-spies
Copy link
Contributor

#112610 needs to address multiple places where "raw" Elasticsearch data types somehow proliferate into the planner/optimizer; this led to bugs because ESQL only cares about a subset of data types that actually occur within the compute engine. Small numeric types like short and float are widened into compute engine's types, like integer resp. double.

For the longest time, this was okay because there was only 1 place where we dealt with this difference: while processing an index' mapping, we decide if a type is supported, and if it needs widening, and stuff it into a FieldAttribute. This means, we had an implicit invariant: data types inside FieldAttributes are always widened, resp. FieldAttributes' types are always types supported by the compute engine.

Let's enforce this invariant, at least via asserts, and let's make sure that type widening happens in as few places as possible. (We could also separate compute engine types from ES types and have FieldAttributes use the former, but that probably has a much wider blast radius that's not worth it.)

@elasticsearchmachine elasticsearchmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Sep 10, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/ES|QL AKA ESQL Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) >tech debt
Projects
None yet
Development

No branches or pull requests

2 participants