Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Implement DecimalColumn + Scalar and add cudf.Scalars of Decimal64Dtype #7732

Conversation

brandon-b-miller
Copy link
Contributor

Closes #7680

@brandon-b-miller brandon-b-miller added 2 - In Progress Currently a work in progress Python Affects Python cuDF API. Cython labels Mar 25, 2021
@brandon-b-miller brandon-b-miller self-assigned this Mar 25, 2021
@isVoid isVoid self-assigned this Mar 25, 2021
@brandon-b-miller brandon-b-miller added 3 - Ready for Review Ready for review by team and removed 2 - In Progress Currently a work in progress labels Mar 29, 2021
@brandon-b-miller brandon-b-miller marked this pull request as ready for review March 29, 2021 21:06
@brandon-b-miller brandon-b-miller requested a review from a team as a code owner March 29, 2021 21:06
@isVoid isVoid added feature request New feature or request non-breaking Non-breaking change labels Mar 29, 2021
@isVoid
Copy link
Contributor

isVoid commented Mar 31, 2021

rerun tests

1 similar comment
@brandon-b-miller
Copy link
Contributor Author

rerun tests

@codecov
Copy link

codecov bot commented Mar 31, 2021

Codecov Report

Merging #7732 (d0a03a2) into branch-0.19 (7871e7a) will increase coverage by 0.43%.
The diff coverage is n/a.

Impacted file tree graph

@@               Coverage Diff               @@
##           branch-0.19    #7732      +/-   ##
===============================================
+ Coverage        81.86%   82.30%   +0.43%     
===============================================
  Files              101      101              
  Lines            16884    17093     +209     
===============================================
+ Hits             13822    14068     +246     
+ Misses            3062     3025      -37     
Impacted Files Coverage Δ
python/cudf/cudf/core/column/lists.py 86.86% <0.00%> (-4.54%) ⬇️
python/dask_cudf/dask_cudf/backends.py 87.16% <0.00%> (-2.47%) ⬇️
python/cudf/cudf/core/column/decimal.py 94.36% <0.00%> (-0.51%) ⬇️
python/cudf/cudf/utils/utils.py 85.06% <0.00%> (-0.38%) ⬇️
python/cudf/cudf/core/column/column.py 87.43% <0.00%> (-0.33%) ⬇️
python/cudf/cudf/core/scalar.py 86.93% <0.00%> (-0.17%) ⬇️
python/cudf/cudf/utils/ioutils.py 78.71% <0.00%> (ø)
python/cudf/cudf/utils/cudautils.py 50.38% <0.00%> (ø)
python/cudf/cudf/core/tools/datetimes.py 84.44% <0.00%> (ø)
python/cudf/cudf/core/column/numerical.py 95.02% <0.00%> (ø)
... and 18 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 42c3bf9...d0a03a2. Read the comment docs.

@brandon-b-miller
Copy link
Contributor Author

@shwina @isVoid @kkraus14 @rgsl888prabhu @galipremsagar I think this should be just about ready to go.

Comment on lines +56 to +62
cdef cppclass fixed_point_scalar[T](scalar):
fixed_point_scalar() except +
fixed_point_scalar(int64_t value,
scale_type scale,
bool is_valid) except +
int64_t value() except +
# TODO: Figure out how to add an int32 overload of value()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we be using the template type T here instead of int64_t?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The template type in practice ends up being decimal64, see decimals.pxd for the wrapper. But we don't actually want to return that - value returns an int64_t or an int32_t for decimal64 and decimal32 respectively. What we need in the long run is an overload of value that returns an int32_t, but we didn't really see an easy way of doing that in cython.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The hacky workaround I think I've seen in the past here is to do something like int32_t value2 "value"() except +, maybe that would work?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I see how that would work. I don't mind adding this, but only regret with it is that without plumbing at least some of the rest of the pieces of the decimal32 pipeline, it might be kind of hard to tell if it works or not.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's punt on it for now then, not a blocker for this PR.

Copy link
Collaborator

@kkraus14 kkraus14 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One small exception message improvement and this should be good to go!

@kkraus14
Copy link
Collaborator

@gpucibot merge

@kkraus14 kkraus14 added 5 - Ready to Merge Testing and reviews complete, ready to merge and removed 3 - Ready for Review Ready for review by team labels Mar 31, 2021
@rapids-bot rapids-bot bot merged commit e379ab1 into rapidsai:branch-0.19 Mar 31, 2021
rapids-bot bot pushed a commit that referenced this pull request Apr 1, 2021
Closes #7498 

This PR adds binary comparison `eq`, `lt`, `gt`, `le`, `ge` to 

- [x] decimal column v. decimal column
- [x] decimal column v. decimal scalar (`decimal.Decimal` and decimal `cudf.Scalar`)
- [x] decimal column v. integer column (`cudf.utils.dtypes.INTEGER_TYPES`)
- [x] decimal column v. integer scalar    (Python ints)

Other minor adds:
- Supports binary ops between `cudf.DecimalColumn` and `cudf.Scalar`, where `Scalar.dtype` is `cudf.Decimal64Dtype` (follow up for #7732 )
- Short comment noting use of decimal64 in `decimals.pxd`
- Adding decimal data type in `basics.rst`

Authors:
  - Michael Wang (https://github.com/isVoid)

Approvers:
  - Keith Kraus (https://github.com/kkraus14)
  - GALI PREM SAGAR (https://github.com/galipremsagar)

URL: #7716
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
5 - Ready to Merge Testing and reviews complete, ready to merge feature request New feature or request non-breaking Non-breaking change Python Affects Python cuDF API.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FEA] Mixed precision Decimal math support in cudf Python
5 participants