Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REVIEW] Fix usecols parameter handling in dask_cudf.read_csv #9618

Merged
merged 2 commits into from
Nov 5, 2021

Conversation

galipremsagar
Copy link
Contributor

Fixes: #9387

This PR fixes usecols parameter usage in dask_cudf.read_csv. When the csv read using byterange's the csv reader has to be passed complete column names in names param but should pass usecols to return the exact columns that are needed only.

@galipremsagar galipremsagar added bug Something isn't working 3 - Ready for Review Ready for review by team dask Dask issue 4 - Needs Dask Reviewer non-breaking Non-breaking change labels Nov 5, 2021
@galipremsagar galipremsagar requested a review from a team as a code owner November 5, 2021 14:24
@galipremsagar galipremsagar self-assigned this Nov 5, 2021
@github-actions github-actions bot added the Python Affects Python cuDF API. label Nov 5, 2021
@codecov
Copy link

codecov bot commented Nov 5, 2021

Codecov Report

Merging #9618 (8f18e72) into branch-21.12 (ab4bfaa) will decrease coverage by 0.09%.
The diff coverage is n/a.

❗ Current head 8f18e72 differs from pull request most recent head 18a9b3e. Consider uploading reports for the commit 18a9b3e to get more accurate results
Impacted file tree graph

@@               Coverage Diff                @@
##           branch-21.12    #9618      +/-   ##
================================================
- Coverage         10.79%   10.69%   -0.10%     
================================================
  Files               116      117       +1     
  Lines             18869    19353     +484     
================================================
+ Hits               2036     2070      +34     
- Misses            16833    17283     +450     
Impacted Files Coverage Δ
python/dask_cudf/dask_cudf/sorting.py 92.90% <0.00%> (-1.21%) ⬇️
python/cudf/cudf/io/csv.py 0.00% <0.00%> (ø)
python/cudf/cudf/io/orc.py 0.00% <0.00%> (ø)
python/cudf/cudf/__init__.py 0.00% <0.00%> (ø)
python/cudf/cudf/core/frame.py 0.00% <0.00%> (ø)
python/cudf/cudf/core/index.py 0.00% <0.00%> (ø)
python/cudf/cudf/io/parquet.py 0.00% <0.00%> (ø)
python/cudf/cudf/core/series.py 0.00% <0.00%> (ø)
python/cudf/cudf/core/reshape.py 0.00% <0.00%> (ø)
python/cudf/cudf/utils/dtypes.py 0.00% <0.00%> (ø)
... and 44 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9fa6ec0...18a9b3e. Read the comment docs.

@galipremsagar
Copy link
Contributor Author

rerun tests

@galipremsagar galipremsagar added 5 - Ready to Merge Testing and reviews complete, ready to merge and removed 3 - Ready for Review Ready for review by team 4 - Needs Dask Reviewer labels Nov 5, 2021
@galipremsagar
Copy link
Contributor Author

@gpucibot merge

@rapids-bot rapids-bot bot merged commit 4cba672 into rapidsai:branch-21.12 Nov 5, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
5 - Ready to Merge Testing and reviews complete, ready to merge bug Something isn't working dask Dask issue non-breaking Non-breaking change Python Affects Python cuDF API.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] Dask cuDF csv reader can incorrectly read rows when usecols is passed
2 participants