[C++] sum doesn't check for overflow

[pitrou]

Describe the bug, including details regarding any error messages, version, and platform.

>>> pc.sum(pa.array([2**63, 2**63], type=pa.uint64()))
<pyarrow.UInt64Scalar: 0>
>>> pc.sum(pa.array([2**62, 2**62], type=pa.int64()))
<pyarrow.Int64Scalar: -9223372036854775808>

Component(s)

C++

[pitrou]

@js8544 @felipecrv

[js8544]

I can add a sum_checked function that returns errors on overflow like cumulative_sum_checked. What do you think?

[pitrou]

Yes, I think that would be worthwhile.

[pitrou]

Of course, we have the same issue with product:

>>> pc.product([2**32,2**32])
<pyarrow.Int64Scalar: 0>

[js8544]

Of course, we have the same issue with product:

>>> pc.product([2**32,2**32])
<pyarrow.Int64Scalar: 0>

Sure, I'll add both.

[felipecrv]

@js8544 ping me for help with the reviews.

[js8544]

@felipecrv @pitrou I noticed that mean reuses the sum kernels and also suffer from overflow. It first sums all inputs (where overflow can happen) and then divide by count. However, since the result of mean is double, it should add the values as doubles, instead of reusing the sum kernels. Do we want:

1. Refactor the mean kernel to add the values as doubles.
2. Also add a mean_checked function.

I'm leaning towards option 1.

[assignUser]

We have an issue for the mean overflow: #34909

[js8544]

OK, I'll first make mean use double for intermediate result and then add sum_checked in two different PRs. There are overlaps in these two changes and splitting them into two PRs would make code review easier.