-
-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Uncaught overflow for gcd(typemin(T), zero(T)) where T<:Signed #33781
Comments
if we return 128.it not a type of Int8 . |
For sure. It shouldn't return julia> gcd(typemin(Int8), typemin(Int8))
ERROR: OverflowError: gcd(-128, -128) overflows |
In general,we don’t check for overflow in integer arithmetic for performance.Such as: julia> x = Int8(-128)
julia> abs(x) == -x == x
julia> true
julia> gcd(Int8(-128),Int8(0)) == abs(Int8(-128))
julia> true This is a compromise that has to be done for efficiency. |
I thought about this, but I think there are two conflicting points. First, consistency. The documentation says the result is positive and the function already checks for overflow in a very similar case: julia> gcd(typemin(Int8), typemin(Int8))
ERROR: OverflowError: gcd(-128, -128) overflows
Stacktrace:
[1] __throw_gcd_overflow(::Int8, ::Int8) at ./intfuncs.jl:50
[2] gcd(::Int8, ::Int8) at ./intfuncs.jl:47
[3] top-level scope at REPL[4]:1 Second, I don't think it's actually a performance hit, or at least I can't consistently get a measure of one. Here's the definition I was using: checked_gcd definitionusing Base: checked_abs
function checked_gcd(a::T, b::T) where T<:Union{Int8,UInt8,Int16,UInt16,Int32,UInt32,Int64,UInt64,Int128,UInt128}
if a == 0
return checked_abs(b)
end
if b == 0
return checked_abs(a)
end
# a == 0 && return abs(b)
# b == 0 && return abs(a)
za = trailing_zeros(a)
zb = trailing_zeros(b)
k = min(za, zb)
u = unsigned(abs(a >> za))
v = unsigned(abs(b >> zb))
while u != v
if u > v
u, v = v, u
end
v -= u
v >>= trailing_zeros(v)
end
r = u << k
# T(r) would throw InexactError; we want OverflowError instead
r > typemax(T) && __throw_gcd_overflow(a, b)
r % T
end
@noinline __throw_gcd_overflow(a, b) = throw(OverflowError("gcd($a, $b) overflows")) Interestingly, All variations in minimum time (using No code in the normal path changes, so my assumption performance issues would come more from branch prediction. Also if performance was harmed by a check, shouldn't the overflow error from above be removed? |
So, you can only add another checked_gcd method.Otherwise,gcd will not get correct result for arrays. julia> a = Int8[-128,126,12]
julia> gcd(a)
julia> 2 Using your checked_gcd methods in stead of original gcd: julia> a = Int8[-128,126,12]
julia> gcd(a)
julia> ERROR: LoadError: OverflowError: checked arithmetic: cannot compute |x| for x = -128::Int8 |
Can't that error already occur from I'd also agree with just not throwing overflow errors from gcd. But it currently throws one. I just think the behavior should be consistent about whether typemin is a valid result from |
Thought about this a bit more, and drafted an implementation of function checked_gcd(abc::AbstractArray{T}) where T<:Integer
idx = 1
a = zero(T)
n = length(abc)
while idx <= n && (abc[idx] == typemin(T) || abc[idx] == zero(T))
a = min(a, abc[idx])
idx += 1
end
while idx <= n
a = checked_gcd(a, abc[idx])
if a == one(T)
return a
end
idx += 1
end
return checked_abs(a)
end |
To add to this issue: currently My vague preference is that neither should throw an error, but I don't feel particularly strongly other than that these should be consistent. |
IMO,
That corresponds to the documentation of I think, we should prefer appropriate checks over performance, as we do in a similar situation comment. Concerning |
The currently implemented algorithms deviate from the previous claims only in these cases:
|
Interestingly, as the domain of
|
This issue has been fixed, I think in #35451 |
yes
|
Hi! Here's a quick example of the issue:
Since
gcd(typemin(T), typemin(T)) where T<:Signed
(per #15228) is an overflow error, and results should be positive I think this is a bug. I'd like to open a PR to fix this (I think anabs
to be swapped out with achecked_abs
). Does this sound reasonable?The text was updated successfully, but these errors were encountered: