-
Notifications
You must be signed in to change notification settings - Fork 304
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Temporarily disable the deletion of the dask dataframe #3814
Merged
rapids-bot
merged 39 commits into
rapidsai:branch-23.10
from
jnke2016:branch-23.10_copy-ddf
Sep 28, 2023
Merged
Changes from 31 commits
Commits
Show all changes
39 commits
Select commit
Hold shift + click to select a range
929edb5
add temporary workaround to copy the dask dataframe
49c1d55
update docstrings
7f392ea
temporarily avoid deleting the copy of the dataframe
6de48ca
Merge remote-tracking branch 'upstream/branch-23.10' into branch-23.1…
1673c8d
update docstrings with accurate column names
8cb3022
Merge remote-tracking branch 'upstream/branch-23.10' into branch-23.1…
e6a1d30
undo changes
89f3a0b
Enable temporarily disabled MG tests
ab59c9b
Enable temporarily disabled MG tests
208b1cc
Enable temporarily disabled MG tests
d31b296
Enable temporarily disabled MG tests
95e5249
Enable temporarily disabled MG tests
1016b77
Enable temporarily disabled MG tests
naimnv 2aa7f25
Enable temporarily disabled MG tests
naimnv 4d193da
Enable temporarily disabled MG tests
naimnv f016e8e
Enable temporarily disabled MG tests
naimnv c04aad8
Enable temporarily disabled MG tests
naimnv da917d6
Enable temporarily disabled MG tests
naimnv 49f9d0e
Merge branch 'branch-23.10' of github.com:rapidsai/cugraph into enabl…
naimnv fc72f31
Use token in the map_partitions to possibly get around error in graph…
naimnv 7b99d79
Use token in the map_partitions to possibly get around error in graph…
naimnv 12c0675
Merge branch 'branch-23.10' of github.com:rapidsai/cugraph into enabl…
naimnv bd85881
Skip deleting copied dask dataframe to avoid crash
naimnv 0d8ff97
Temporarily skip deleting copied dask dataframe to avoid crash
naimnv 9bb51fe
Skip deleting copied dask dataframe to avoid crash while creating fro…
naimnv a407617
Merge remote-tracking branch 'upstream/branch-23.10' into branch-23.1…
jnke2016 9ff7124
fix merge conflict
jnke2016 e5f45e7
Merge remote-tracking branch 'upstream/branch-23.10' into branch-23.1…
jnke2016 67fd5c6
undo change
jnke2016 c777f87
undo changes
jnke2016 1485c4c
Merge remote-tracking branch 'upstream/branch-23.10' into branch-23.1…
jnke2016 0262d34
update branch with the latest changes
jnke2016 384cf39
fix style
jnke2016 9b5635d
Merge remote-tracking branch 'upstream/branch-23.10' into branch-23.1…
jnke2016 dde98b3
create unique token and delete ddf after compute call
jnke2016 4f588de
create unique token and delete ddf after compute call
jnke2016 d7f3e59
fix style
jnke2016 8221b16
Merge remote-tracking branch 'upstream/branch-23.10' into branch-23.1…
jnke2016 557b5b3
Merge remote-tracking branch 'upstream/branch-23.10' into branch-23.1…
jnke2016 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't this deletion come after the computation below is done anyway? If you are building graph from
ddf
, then you cannot release the keys ofddf
untildelayed_tasks_d
is finished computing/persisting.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not sure why the keys were deleted before
delayed_tasks_d
computes. My PR essentially removed that deletion and it worked. I am not sure if doing it afterdelayed_tasks_d
computes will make a difference but let me try.