Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix memcheck error in groupby-tdigest get_scalar_minmax #9339

Merged

Conversation

davidwendt
Copy link
Contributor

The groupby tdigest logic in the get_scalar_minmax would access incorrect device memory when encountering an empty group. The index value calculated for a column element in an empty group would be off by -1. This was found running a cuda-memcheck (compute-sanitizer) with the AllNulls gtest. Here the first group is empty resulting an out-of-bounds read at col.element<T>(-1).

This PR adds a check for the empty group (valid_count==0) to prevent reading the incorrect column row.

@davidwendt davidwendt added bug Something isn't working 3 - Ready for Review Ready for review by team libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change labels Sep 29, 2021
@davidwendt davidwendt self-assigned this Sep 29, 2021
@davidwendt davidwendt requested a review from a team as a code owner September 29, 2021 17:57
Copy link
Contributor

@mythrocks mythrocks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, this.

@mythrocks
Copy link
Contributor

How slim are the odds of this fix making it back into 21.10?

@codecov
Copy link

codecov bot commented Sep 29, 2021

Codecov Report

Merging #9339 (0fac04c) into branch-21.12 (ab4bfaa) will decrease coverage by 0.01%.
The diff coverage is 0.00%.

❗ Current head 0fac04c differs from pull request most recent head 5d15628. Consider uploading reports for the commit 5d15628 to get more accurate results
Impacted file tree graph

@@               Coverage Diff                @@
##           branch-21.12    #9339      +/-   ##
================================================
- Coverage         10.79%   10.77%   -0.02%     
================================================
  Files               116      116              
  Lines             18869    19360     +491     
================================================
+ Hits               2036     2087      +51     
- Misses            16833    17273     +440     
Impacted Files Coverage Δ
python/cudf/cudf/core/_base_index.py 0.00% <0.00%> (ø)
python/cudf/cudf/core/dataframe.py 0.00% <0.00%> (ø)
python/cudf/cudf/core/frame.py 0.00% <0.00%> (ø)
python/cudf/cudf/core/index.py 0.00% <0.00%> (ø)
python/cudf/cudf/core/join/_join_helpers.py 0.00% <ø> (ø)
python/cudf/cudf/core/multiindex.py 0.00% <0.00%> (ø)
python/cudf/cudf/core/series.py 0.00% <0.00%> (ø)
python/cudf/cudf/core/udf/api.py 0.00% <0.00%> (ø)
python/cudf/cudf/core/udf/lowering.py 0.00% <0.00%> (ø)
python/cudf/cudf/core/udf/pipeline.py 0.00% <0.00%> (ø)
... and 60 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e63f455...5d15628. Read the comment docs.

Copy link
Contributor

@nvdbaranec nvdbaranec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We may have uncovered another bounds check that could get rolled into here.

@nvdbaranec nvdbaranec self-requested a review September 29, 2021 20:46
Copy link
Contributor

@nvdbaranec nvdbaranec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. I'll open a seperate PR for other issues encountered.

@nvdbaranec nvdbaranec self-requested a review September 30, 2021 16:08
@davidwendt
Copy link
Contributor Author

@gpucibot merge

@rapids-bot rapids-bot bot merged commit 4090c45 into rapidsai:branch-21.12 Oct 1, 2021
@davidwendt davidwendt deleted the memcheck-groupby-tdigest branch October 1, 2021 12:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3 - Ready for Review Ready for review by team bug Something isn't working libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants