Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Removed inefficient LayerFilter check #2324

Merged
merged 2 commits into from
Aug 9, 2017

Conversation

lossyrob
Copy link
Member

@lossyrob lossyrob commented Aug 9, 2017

The non-overlapping check took upwards of 2 minutes in a case where I was querying a zoom 12 512x512 GlobalLayout for the state of Nevada. This is a nice thing to ensure, but not at that price. Make a note in the scaladocs for the implementor's responsibility to ensure non-overlapping KeyBounds.

@lossyrob
Copy link
Member Author

lossyrob commented Aug 9, 2017

@echeipesh need advice here. That check really slows things down unacceptably in the workshop. But the failing test reminds me why it’s there (hint, it’s for the or that I doubt has been used in anger once). Is there some clever way we can get around having to do the costly check without allowing users to potentially do invalid queries (which I believe pulled back duplicate tiles)?

@echeipesh
Copy link
Contributor

Actually this should be fine now. I think this check existed before we started using MergeQueue on the RDDReaders: https://github.com/locationtech/geotrellis/blob/master/s3/src/main/scala/geotrellis/spark/io/s3/S3RDDReader.scala#L58

Any intersecting ranges should be taken care of there.

@lossyrob lossyrob merged commit ac4a6c2 into locationtech:master Aug 9, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants