-
Notifications
You must be signed in to change notification settings - Fork 304
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve memory footprint and performance of graph creation #3542
Improve memory footprint and performance of graph creation #3542
Conversation
…awa/cugraph into cugraph_scaling_benchmark
python/cugraph/cugraph/structure/graph_implementation/simpleDistributedGraph.py
Outdated
Show resolved
Hide resolved
I think this PR may have some bugs, you may want to run the MG tests before finalizing it |
I ran into issues trying to use this with the latest changes to sampling and edge properties |
…thub.com/VibhuJawa/cugraph into improve_memory_footprint_symmetrsization
…thub.com/VibhuJawa/cugraph into improve_memory_footprint_symmetrsization
@alexbarghi-nv , Can you try again. i think i have resolved all the bugs from my side and testing on multiple GPU configurations. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
/merge |
This PR improves the memory footprint in graph creation and resolves story: #3439
The metric we are concerned about is input_to_peak_ratio which is currently 12.1x. Ideally this should be in 2x range.
Metric Meaning: How much memory expands for an input graph of size x. So if you want to analyze a graph of size x, we need 12x GPU ram currently.
Memory overhead ratio reduction:
12x
to6.14x
for undirected (Reverse edges added)5.9x
to3.07x
for directed graphs (No reverse edges)Time Reduction:
Comparison Table
Graph Creation:
CC: @rlratzel