Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Resultset and Dataset Refactors #3957

Merged
merged 49 commits into from
Nov 19, 2023
Merged

Resultset and Dataset Refactors #3957

merged 49 commits into from
Nov 19, 2023

Conversation

nv-rliu
Copy link
Contributor

@nv-rliu nv-rliu commented Oct 26, 2023 •
edited by rlratzel
Loading

This PR replaces and is a continuation of #3857 (by @betochimas)

This PR primarily adds testing for the Resultset class, introduced earlier in 23.10. The tests take a similar approach to test_dataset, creating a temporary directory to test downloading all result files. To align Resultset and Dataset, the setter and getter for each download directory is moved into DefaultDownloadDir, so that each class shares an instance of DefaultDownloadDir and can be configured independently, although their default locations are still both dependent on the RAPIDS_DATASET_ROOT_DIR_PATH environment variable. The old patterns are present but commented-out, so this change would be breaking.

This PR also removes the deprecated experimental.datasets package due to it being promoted to stable for >=1 release.

root and others added 30 commits August 30, 2023 21:58
Added tests to verify load_resultsets by refactoring
db99094
@betochimas
Added benchmark datasets yaml files for use with Dataset
5c37396
@betochimas
Introduce way to download larger datasets
93fb3e7
@betochimas
Merge branch 'rapidsai:branch-23.10' into branch-23.10-large-datasets
b236d85
@betochimas
Testing gpg verification
e94baa3
@betochimas
Use create_using pattern to give user choice for dataframe lib
aafd5d0
@betochimas
Merge branch 'branch-23.10' into branch-23.10-large-datasets
23edacf
@betochimas
Insert benchmarking Dataset instances
a4ed8ef
@betochimas
Merge branch 'branch-23.10' into branch-23.10-large-datasets
ddb8c35
@betochimas
Merge branch 'rapidsai:branch-23.10' into resultset-tests
87eea53
@betochimas
Refactored getters and setters for download_dir
95bf161
@betochimas
Update generate_resultsets.py to align with DefaultDownloadDir API ch…
be5ff11 
…ange
@betochimas
Merge branch 'branch-23.10' into resultset-tests
795ab55
@betochimas
Changes to testing and fixing urls
31ca5ea
@betochimas
Testing changes to pass CI, added and removed comments
8e0cfac
@betochimas
Merge branch 'branch-23.10' into branch-23.10-large-datasets
d15f601
@betochimas
Merge branch 'branch-23.10' into branch-23.10-large-datasets
045597a
@betochimas
Merge branch 'branch-23.10' into resultset-tests
6546034
@betochimas
Merge branch 'branch-23.10' into branch-23.10-large-datasets
a85cb7e
@betochimas
Add FIXMEs for CI failure points
f816f78
@nv-rliu
Merge branch 'branch-23.10-large-datasets' of https://github.com/beto…
12c3d66 
…chimas/cugraph into branch-23.12-large-datasets
@nv-rliu
Merge branch 'rapidsai:branch-23.12' into branch-23.12-large-datasets
7c0db5d
@nv-rliu
Merge branch 'rapidsai:branch-23.12' into branch-23.12-large-datasets
abda461
@nv-rliu
update large dataset work. primarily unit tests
3fb602b
@nv-rliu
Merge branch 'branch-23.12-large-datasets' of github.com:nv-rliu/cugr…
858e155 
…aph into branch-23.12-large-datasets
@nv-rliu
remove twitter
875011d
@nv-rliu
fix bug in test fixture for unweighted graphs
424bc51
@nv-rliu
Merge branch 'branch-23.12' into resultset-tests
56fad47
@nv-rliu
Merge branch 'branch-23.12-rs-ds-refactor' into resultset-tests
d8fe7b6
@nv-rliu
Merge pull request #14 from betochimas/resultset-tests
0d555b1 
Merge resultset-tests into continuation branch
@rlratzel rlratzel requested a review from a team as a code owner November 16, 2023 16:30
@rlratzel

This comment was marked as duplicate.

Sign in to view
@rlratzel

This comment was marked as duplicate.

Sign in to view
@nv-rliu
Copy link
Contributor Author

nv-rliu commented Nov 16, 2023

/ok to test

@nv-rliu
Copy link
Contributor Author

nv-rliu commented Nov 17, 2023

/ok to test

@nv-rliu
Copy link
Contributor Author

nv-rliu commented Nov 17, 2023

/ok to test

@raydouglass raydouglass removed the request for review from a team November 17, 2023 19:05
@raydouglass
Copy link
Member

Removing ops-codeowners from the required reviews since it doesn't seem there are any file changes that we're responsible for. Feel free to add us back if necessary.

@BradReesWork
Copy link
Member

/ok to test

@rlratzel
Copy link
Contributor

/ok to test

@rlratzel rlratzel mentioned this pull request Nov 18, 2023
Closed
@rlratzel

This comment was marked as duplicate.

Sign in to view
@rlratzel

This comment was marked as duplicate.

Sign in to view
@rlratzel
Copy link
Contributor

/merge

@rlratzel

This comment was marked as duplicate.

Sign in to view
@rapids-bot rapids-bot bot merged commit 0684f9d into rapidsai:branch-23.12 Nov 19, 2023
73 checks passed
@nv-rliu nv-rliu deleted the branch-23.12-rs-ds-refactor branch November 19, 2023 13:52
@nv-rliu
Copy link
Contributor Author

nv-rliu commented Aug 1, 2024

Closes https://github.com/rapidsai/graph_dl/issues/297

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rlratzel rlratzel rlratzel approved these changes

Assignees

@nv-rliu nv-rliu

Labels
breaking Breaking change improvement Improvement / enhancement to an existing function
Projects
None yet
Milestone
23.12
Development

Successfully merging this pull request may close these issues.
5 participants
@nv-rliu @rlratzel @raydouglass @BradReesWork @betochimas