Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[IMP] Sample with Offsets in the Bulk Sampler #3524

Merged
merged 64 commits into from
May 10, 2023

Conversation

alexbarghi-nv
Copy link
Member

@alexbarghi-nv alexbarghi-nv commented Apr 28, 2023

Resolves #3318
Resolves #3385

Updates the bulk sampler to use the offsets returned by the uniform_neighbor_sample call, and take advantage of the pre-partitioned data returned from C++ to avoid repartitioning and scanning, and also to reduce the number of files written. This PR is expected to significantly improve memory and compute time for bulk sampling.

Merge after #3517 - Merged
Merge after #3495 - Merged

alexbarghi-nv and others added 23 commits April 17, 2023 20:21
@alexbarghi-nv alexbarghi-nv self-assigned this Apr 28, 2023
@alexbarghi-nv alexbarghi-nv added improvement Improvement / enhancement to an existing function breaking Breaking change labels Apr 28, 2023
@alexbarghi-nv alexbarghi-nv added this to the 23.06 milestone Apr 28, 2023
@alexbarghi-nv alexbarghi-nv added the Blocked Cannot progress due to external reasons label Apr 28, 2023
Copy link
Contributor

@rlratzel rlratzel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks

@alexbarghi-nv alexbarghi-nv requested a review from a team as a code owner May 9, 2023 17:19
@rlratzel rlratzel removed the Blocked Cannot progress due to external reasons label May 9, 2023
@rlratzel
Copy link
Contributor

rlratzel commented May 9, 2023

/merge

@raydouglass raydouglass removed the request for review from a team May 9, 2023 20:00
@alexbarghi-nv alexbarghi-nv removed the request for review from a team May 9, 2023 20:36
@alexbarghi-nv
Copy link
Member Author

/merge

@rapids-bot rapids-bot bot merged commit 3d99613 into rapidsai:branch-23.06 May 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking Breaking change improvement Improvement / enhancement to an existing function
Projects
None yet
4 participants