Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add unbatch functionality #4628

Merged
merged 26 commits into from
May 16, 2022
Merged

Add unbatch functionality #4628

merged 26 commits into from
May 16, 2022

Conversation

tzuhanchang
Copy link
Contributor

Unbatching data from DataLoader batch to a list. This is useful for GNN edge classifier, in which case, graphs are required to be reconstructed with edge predictions.

tzuhanchang and others added 4 commits May 12, 2022 13:36
Unbatching data from DataLoader batch to a list.
Unbatching data from DataLoader batch to a list.
Unbatching data from DataLoader batch to a list. This is useful for GNN edge classifier, in which case, graphs are required to be reconstructed with edge predictions.
@codecov
Copy link

codecov bot commented May 12, 2022

Codecov Report

Merging #4628 (7365414) into master (da78713) will increase coverage by 0.00%.
The diff coverage is 100.00%.

❗ Current head 7365414 differs from pull request most recent head 3992f87. Consider uploading reports for the commit 3992f87 to get more accurate results

@@           Coverage Diff           @@
##           master    #4628   +/-   ##
=======================================
  Coverage   82.97%   82.98%           
=======================================
  Files         316      317    +1     
  Lines       16784    16792    +8     
=======================================
+ Hits        13927    13935    +8     
  Misses       2857     2857           
Impacted Files Coverage Δ
torch_geometric/utils/__init__.py 100.00% <100.00%> (ø)
torch_geometric/utils/unbatch.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update da78713...3992f87. Read the comment docs.

torch_geometric/loader/utils.py Outdated Show resolved Hide resolved
torch_geometric/loader/utils.py Outdated Show resolved Hide resolved
torch_geometric/loader/utils.py Outdated Show resolved Hide resolved
Copy link
Member

@rusty1s rusty1s left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think in order to get this merged, we would need to have a pure PyTorch function without for-loops. IMO, this is achievable by

src.split(degree(index).tolist())

In addition, we would need to add this function to utils rather than loader/utils.py, add doccumentation, and write tests for it. WDYT?

tzuhanchang and others added 2 commits May 15, 2022 18:30
For unbatching node features, for example, use ```src=data.x``` and ```index =data.x_batch``` (assume ```follow_batch``` is set to ```x```).
@tzuhanchang
Copy link
Contributor Author

@rusty1s Thank you very much! src.split(degree(index).tolist()) works perfectly, it can replace two for loops completely. I also add it to utils.

@tzuhanchang tzuhanchang requested a review from rusty1s May 16, 2022 11:00
Copy link
Member

@rusty1s rusty1s left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks better. Thank you! Can we also add some basic tests in test/utils/test_unbatch.py and add this operator to torch_geometric/utils/__init__.py?

torch_geometric/utils/unbatch.py Outdated Show resolved Hide resolved
torch_geometric/utils/unbatch.py Outdated Show resolved Hide resolved
@rusty1s rusty1s changed the title Add unbatching tool to loader utils. Add unbatch functionality May 16, 2022
Copy link
Member

@rusty1s rusty1s left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates!

@rusty1s rusty1s merged commit 3b503fa into pyg-team:master May 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants