-
-
Notifications
You must be signed in to change notification settings - Fork 18.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PERF: Index._shallow_copy shares _cache with copies of self #36840
Conversation
The issue @jreback raised means sharing ids is a bit more complicated than thought. I've repurposed this PR to make indexes share caches, as that was planned in a follow-up, and the code for that is not that different. Sharing caches gives some performance improvements also (in different circumstances than sharing _ids, but still nice). |
ping. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
looks good @topper-123 |
thanks @topper-123 |
…ev#36840) * PERF: Index.equals when comparing to copies of self * refactor _shallow_copy, add GH number * PERF: share _cache, don't share _id * rename tests * fix memory usage test Co-authored-by: Jeff Reback <[email protected]>
…ev#36840) * PERF: Index.equals when comparing to copies of self * refactor _shallow_copy, add GH number * PERF: share _cache, don't share _id * rename tests * fix memory usage test Co-authored-by: Jeff Reback <[email protected]>
#32640 copied the cache when copying indexes. Indexes are immutable, so some refactoring allows us to share the cache instead. This means potentially fewer expensive index ops.
Performance example: