Geo: Proposal for refactoring out ShapeBuilder hierarchy #40908
Assignees
Labels
:Analytics/Geo
Indexing, search aggregations of geo points and shapes
Meta
Team:Analytics
Meta label for analytical engine team (ESQL/Aggs/Geo)
Comments
imotov
added
:Analytics/Geo
|
Pinging @elastic/es-analytics-geo |
imotov
added a commit
to imotov/elasticsearch
that referenced
this issue
Apr 11, 2019
Adds 3 new geosql functions. The ST_Z functions as well as ST_X and ST_Y functions for polygons are not supported because of the issues described in elastic#40908. The suppor will be added as soon as the ShapeBuilder issues are resolved. Relates to elastic#29872
|
+1 I ran into the same feeling when debugging shapes. a lot of wkt/geojson functionality is wrapped up together and depends on s4j. This constant translation of geometries always felt awkward in tests to me. Maybe these helper methods exist in other util classes that I am not aware of |
imotov
added a commit
to imotov/elasticsearch
that referenced
this issue
Apr 26, 2019
Adds GeoJson parser for Geometry classes defined in libs/geo. This should allow us to bypass ShapeBuilders in geosql and is the first step in ShapeBuilder refactoring. The provided functionality should be sufficient for geosql parsing, but additional functionality will be required for complete parity with ShapeBuilder parser - we need to add handling of mapper parameters in WKT parser as well. That will be added in a follow up PR. Relates elastic#40908
imotov
added a commit
that referenced
this issue
Apr 29, 2019
imotov
added a commit
to imotov/elasticsearch
that referenced
this issue
Apr 29, 2019
Adds GeoJson parser for Geometry classes defined in libs/geo. Relates elastic#40908 and elastic#29872
imotov
added a commit
that referenced
this issue
Apr 29, 2019
akhil10x5
pushed a commit
to akhil10x5/elasticsearch
that referenced
this issue
May 2, 2019
Adds GeoJson parser for Geometry classes defined in libs/geo. Relates elastic#40908 and elastic#29872
gurkankaymak
pushed a commit
to gurkankaymak/elasticsearch
that referenced
this issue
May 27, 2019
Adds GeoJson parser for Geometry classes defined in libs/geo. Relates elastic#40908 and elastic#29872
imotov
added a commit
to imotov/elasticsearch
that referenced
this issue
Jun 26, 2019
Moves coordinate validation from Geometry constructors into parser. Relates elastic#40908
This was referenced Jul 5, 2019
imotov
added a commit
that referenced
this issue
Jul 16, 2019
Extracts dateline decomposition logic from ShapeBuilder into a separate utility class that is used on the indexing side. The search side will be handled as part of another PR at this time we will remove the decomposition logic from ShapeBuilders as well. This PR also doesn't change any existing logic including bugs. Relates to #40908
imotov
added a commit
that referenced
this issue
Jul 17, 2019
Extracts dateline decomposition logic from ShapeBuilder into a separate utility class that is used on the indexing side. The search side will be handled as part of another PR at this time we will remove the decomposition logic from ShapeBuilders as well. This PR also doesn't change any existing logic including bugs. Relates to #40908
imotov
added a commit
to imotov/elasticsearch
that referenced
this issue
Jul 22, 2019
Removes unnecessary now timeline decompositions from shape builders and deprecates ShapeBuilders in QueryBuilder in favor of libs/geo shapes. Relates to elastic#40908
imotov
added a commit
that referenced
this issue
Jul 24, 2019
Removes unnecessary now timeline decompositions from shape builders and deprecates ShapeBuilders in QueryBuilder in favor of libs/geo shapes. Relates to #40908
imotov
added a commit
that referenced
this issue
Jul 24, 2019
Removes unnecessary now timeline decompositions from shape builders and deprecates ShapeBuilders in QueryBuilder in favor of libs/geo shapes. Relates to #40908
rjernst
added
the
Team:Analytics
|
Where do we track removal of this module? |
|
Shall we open a new issue just for that with more specific objective?. I found the title of this issue misleading as the refactoring is done. |
|
Yeah, I am ok with closing this one in favor of a more specific issue |
|
Pinging @elastic/es-analytics-geo (Team:Analytics) |
|
closing in favour of #96097 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
At the moment shapeBuilders perform much more functionality than just shape building. We have serialization, parsing logic plus logic necessary for handling dateline transformation and conversion of shapes for lucene indexing in both libs/geo and S4J formats. As we are removing support for the S4J shapes, I would like to propose refactoring ShapeBuilder set of classes.
In #36477 we introduced a hierarchy of geo classes that can replace the hierarchy build with ShapeBuilder as a root. This hierarchy has pretty much all functionality except GeoJson parser and logic that is needed to convert shapes into the format consumable by Lucene, if we could add the GeoJson parser to libs/geo, and extract the logic we can remove the entire ShapeBuilder hierarchy and significantly simplify the geo processing pipeline structure. I think it will also simplify switching to different reference system, since it will allows us to have the reference system-specific logic separate from the geo hierarchy.
It will also remove the issue that we are facing in geosql at the moment. The only way to retrieve a shape in geo at the moment is through the
ShapeParser.parse(....).buildGeometry()path, which converts some shapes into somewhat equivalent set of shapes for lucene consumption, among other things it changes the order of coordinates of polygons, splits polygons on dateline, simplifies collections, etc. This can be really confusing for geosql users, since they will get shapes different from what they put into elasticsearch.To summarize the following actions will be required:
The text was updated successfully, but these errors were encountered: