Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor window intersection and union #2164

Merged
merged 5 commits into from
Oct 13, 2021

Conversation

groutr
Copy link
Contributor

@groutr groutr commented Apr 27, 2021

Window intersections and unions used suboptimal implementations.

wins = [windows.Window(i, i, 100-i, 100-i) for i in range(10)]
Intersect:
    
Old: %timeit windows.intersect(wins) 
141 µs ± 3.16 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)

New: %timeit intersect(wins)
26.4 µs ± 2.13 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)

Intersection

Old: %timeit windows.intersection(wins)
280 µs ± 5.07 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

New: %timeit intersection(wins)
24.9 µs ± 453 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)

Union:
    
Old: %timeit windows.union(wins)
114 µs ± 6.2 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)

New: %timeit union(wins)
24.4 µs ± 574 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)

@groutr groutr changed the title Refactor Intersection and Union Refactor window intersection and union Apr 27, 2021
@groutr
Copy link
Contributor Author

groutr commented May 4, 2021

@sgillies Could I get a review on this?

@sgillies
Copy link
Member

sgillies commented May 6, 2021

@groutr I'll take a look first thing next week. I'm on vacation now. How is this one related to #2162? Are they separate or dependent on each other?

@groutr
Copy link
Contributor Author

groutr commented May 7, 2021

@sgillies I didn't mean to interrupt your vacation. This is independent of #2162.

Two main things happen in this PR:

  1. Removes the use of numpy when computing intersections and unions. It isn't really helping as much as it is slowing things down.
  2. Remove the use of combinations from intersect. When we have 10 windows, it makes 90 (n!/(n-2)!) intersect computations instead of the (n-1) computations that are needed.

return Window.from_slices(
(stacked[0, 0].max(), stacked[0, 1].min()),
(stacked[1, 0].max(), stacked[1, 1]. min()))
return functools.reduce(_intersection, windows)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

if coeffs[2] > 0 and coeffs[3] > 0:
return Window(*coeffs)
else:
raise WindowError(f"Intersection is empty {w1} {w2}")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@groutr combine these as the first is only called from the second?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you're referring to why I split the functions up the way I did.

The reasoning is as follows:
_compute_intersection: Compute the intersection directly. A low-level function that is useful in the case where it is not desired to create a Window object. This is useful in the case where the calling method is creating its own Window object or manipulating the values. Reviewing the function, it should probably be modified to also work on the tuple form of a Window.
_intersection: Compute an intersection and check if the intersection is empty. It is a higher-level function that is primarily designed to receive as input Windows and return a Window. This is what makes the reduce call possible.

I can combine them if you feel the current split is unnecessary.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@groutr we deprecated the old tuple form at 1.0: https://github.com/mapbox/rasterio/blob/master/CHANGES.txt#L1082. No need to support it.

intersection(*windows)
return True
except WindowError:
return False
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes!

@sgillies
Copy link
Member

@groutr what would you think about rebasing this on the master branch, for 1.3? That would protect us a little more from regressions.

@groutr groutr force-pushed the window_refactor branch from f03bc6d to 8ff414b Compare June 2, 2021 20:45
@groutr
Copy link
Contributor Author

groutr commented Jun 2, 2021

@sgillies I rebased onto master. I'm fine with holding these changes for 1.3 as they don't directly fix any known bug.

@groutr groutr changed the base branch from maint-1.2 to master June 2, 2021 20:50
@groutr
Copy link
Contributor Author

groutr commented Jul 28, 2021

@sgillies Is this PR still destined for 1.3.0?

@sgillies
Copy link
Member

@groutr yes, thanks for the query!

@sgillies sgillies added this to the 1.3.0 milestone Jul 28, 2021
Copy link
Member

@sgillies sgillies left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's do this! Thanks @groutr.

@sgillies sgillies merged commit 9c79da6 into rasterio:master Oct 13, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants