Polygon2D doesn't render when there are crossed lines

[altered-dev]

Godot version: 3.2.2 (Steam)

OS/device including version: Ubuntu 20.04, Intel HD 3000, GLES2

Issue description: I was doing a project where a player draws shapes as some sort of platforms, and I tried to make a shape that has crossed lines, and it silently fails to render. Some of the shapes do draw though, but I don't really know how to categorize that.

Steps to reproduce:

1. Create a Polygon2D object
2. Make a shape which contains crossed lines

Here is the video demonstration (in the game, it draws Line2D's first, then when a shape is finished, it replaces with Polygon2D):
https://drive.google.com/file/d/1G5EqbVSwqZp6JxG7KvOMvfNVOelYnbht/view?usp=sharing

[Xrayez]

Yeah the triangulator used to render those polygons in Godot doesn't handle self-intersecting polygons at the moment.

See #16423 which can help this perhaps.
See my proposal at godotengine/godot-proposals#913 which could help this too.

But currently you can try to pass those polygons vertices through Geometry.merge_polygons_2d(polygon, PoolVector2Array([])) so that the boolean operation can turn this into several simple polygons (using the EVEN_ODD fill rule by default) which can be drawn individually. It may not always work because ensuring polygons to be strictly simple is computationally expensive, so this is disabled by default.

[Av3r3tt]

@Xrayez : Do I understand you correctly that, as a workaround to find a self-intersecting polygon, one might have to see whether
Geometry.merge_polygons2d(myPolygon, myPolygon)
returns more than 1?

[Xrayez]

Not really, the suggested workaround is that you can just do the merge operation, which does perform polygon simplification under the hood for you, so there's no need to check whether polygon is strictly-simple in the first place, as most of the time (95% of cases), these polygons will be converted to strictly-simple.

But yeah, the simplification alone could result in multiple polygons to be returned. I wouldn't rely on this information because there might be other reasons as to why merge operation would result in multiple polygons (say, merging two halves of a donut, now you have both outer and inner polygons).

[Av3r3tt]

@Xrayez

@alt3r3dgd not really, the suggested workaround is that you can just do the merge operation, which does perform polygon simplification under the hood for you, so there's no need to check whether polygon is strictly-simple in the first place, as most of the time (95% of cases), these polygons will be converted to strictly-simple.

But yeah, the simplification alone could result in multiple polygons to be returned. I wouldn't rely on this information because there might be other reasons as to why merge operation would result in multiple polygons (say, merging two halves of a donut, now you have both outer and inner polygons).

Merge... with what? When I have one polygon, and it is self-intersecting...? Do I do
var myCleanedUpPolygon = Geometry.merge_polygons2d(myDirtyPolygon, null) ?

[Xrayez]

@Av3r3tt yes, see my first post:

var polygons = Geometry.merge_polygons_2d(polygon, PoolVector2Array([]))

It may work with null, not sure.

Technically it wouldn't be a merge, but this is how Clipper library does simplification internally as well. I agree it's not intuitive at the moment, this is why I suggested to add simplify_polygon() to Geometry in godotengine/godot-proposals#913, which would call into merge_polygons() with strictly_simple parameter.

[Giwayume]

so that the boolean operation can turn this into several simple polygons (using the EVEN_ODD fill rule by default) which can be drawn individually.

When you say EVEN_ODD is used by default, you're talking about the underlying "Clipper" library, right? I don't see a way to specify "NON_ZERO" from the gdscript interface side.

[RobertBColton]

I would like to point out that Unity3D will remove the offending self-intersection, just the one face/affected edge and still render the rest of the polygon. Unity will also display a warning to the user in the editor.

Going further, I think a bigger issue is that transitively the logic is actually broke.
#99745 (comment)

merge_polygons will actually report success (size() = 1) even if both of its inputs could be triangulated and rendered instead of simply returning the failure case (should return the zero-area holes).