-
Notifications
You must be signed in to change notification settings - Fork 361
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix numerical errors #3520
Fix numerical errors #3520
Conversation
values like 4.9999999999 were actually floored to 4, whereas it should be 5, caused pixel shifts in output
Found a simple test that exposes the issue. It's all about gridextents that are in fact aligned, but due to very small numerical differences, things go wrong. Still have to figure out where and how to integrate this as part of geotrellis unit tests.
|
@jdries a nice find! Could you add this unit test? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔥 Thx for a nice PR, but tests don't compile! Once the CI is green, I'll merge it.
targetBounds.height should be resultingBounds.height | ||
targetBounds.width should be resultingBounds.width |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tests don't compile!
targetBounds.height should be resultingBounds.height | |
targetBounds.width should be resultingBounds.width | |
targetBounds.height shouldBe resultingBounds.height | |
targetBounds.width shouldBe resultingBounds.width |
Overview
When working in epsg:4326 coordinates, I recently observed separate bugs caused by the same problem. Basically transformations between geographical coordinates and grid coordinates where wrong due to floating point imprecision.
In both cases, grid coordinates were calculated as doubles and resulted in e.g. 4.9999999 but were done floored (toLong) resulting in 4.
This resulted in an actual pixel shift in one case, and in another case a missing row of pixels at the bottom of a saved geotiff.
Checklist
Notes
Optional. Ancillary topics, caveats, alternative strategies that didn't work out, anything else.
Closes #XXX