Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve flight mode tracing performance for large tracings #3880

Merged
merged 3 commits into from
Mar 14, 2019

Conversation

daniel-wer
Copy link
Member

@daniel-wer daniel-wer commented Mar 12, 2019

This PR includes a couple of performance optimizations:

  • Decrease UNDO stack size to 20 (from 100). For a ~80.000 node tracing the undo stack grew to ~1GB which may decrease tracing performance. If users want to restore earlier versions they can always use the "Version Restore View".
  • Optimize adding a single edge to the edge_collection.
  • Tweak the diffable_map size. Decreased from 10.000 to 1.000, which improved diffing/cloning time when adding a single edge (which is the most frequent case).
  • Removed the existsCache from the diffable_map. Cloning this map was a major performance bottleneck. When setting we now no longer know whether the key already exists and need to search through all chunks, but that is much faster than the cloning. Also, much less garbage collection during the performance measurement (from up to 20% down to 5%). This change had the biggest positive performance impact :)

I tested this with an 80.000 node tracing. FPS during flight mode tracing increased from ~3-7FPS (and hefty lag) to ~17-22FPS, where smooth tracing is very possible :)
I also tested whether tree split performance was affected by the changes, which it was not (but it is and was rather slow).

Not sure whether this will also fix the crash that was reported, probably not :/

URL of deployed dev instance (used for testing):

Steps to test:

Issues:


Copy link
Member

@philippotto philippotto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent PR 👍 Thanks for investigating and improving this so thoroughly 🎉

@daniel-wer daniel-wer merged commit f70a6bf into master Mar 14, 2019
@daniel-wer daniel-wer deleted the improve-flight-performance branch March 14, 2019 10:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants