Decimal strict mode should prevent Decimal("0.89") == 0.89

[timkay]

Bug report

Bug description:

Mixing Decimals and floats will often get you the wrong answer. By default, the Decimal library allows such behavior:

>>> import decimal
>>> decimal.Decimal(0.1)
Decimal('0.1000000000000000055511151231257827021181583404541015625')
>>> decimal.Decimal("0.1") < 0.1
True
>>> decimal.Decimal("0.1") == 0.1
False

Fortunately, you can turn on floating point strict mode, where mixing Decimals and floats is prohibited:

>>> decimal.getcontext().traps[decimal.FloatOperation] = True
>>> decimal.Decimal(0.1)
Traceback (most recent call last):
  File "<console>", line 1, in <module>
decimal.FloatOperation: [<class 'decimal.FloatOperation'>]
>>> decimal.Decimal("0.1") < 0.1
Traceback (most recent call last):
  File "<console>", line 1, in <module>
decimal.FloatOperation: [<class 'decimal.FloatOperation'>]

HOWEVER, for some reason, == and != are allowed in floating point strict mode, and produce wrong answers:

>>> decimal.Decimal("0.1") == 0.1
False
>>>

When floating point strict mode is on, why is == and != still allowed?

    if isinstance(other, float):
        context = getcontext()
        if equality_op:
            context.flags[FloatOperation] = 1
        else:
            context._raise_error(FloatOperation,
                "strict semantics for mixing floats and Decimals are enabled")
        return self, Decimal.from_float(other)

This code would be better without the if equality_op::

    if isinstance(other, float):
        context = getcontext()
        context._raise_error(FloatOperation,
            "strict semantics for mixing floats and Decimals are enabled")
        return self, Decimal.from_float(other)

Why are == and != allowed in floating point strict mode?

CPython versions tested on:

3.12

Operating systems tested on:

Linux

[skirpichev]

you can turn on floating point strict mode, where mixing Decimals and floats is prohibited

Not at all:
"If the signal is not trapped (default), mixing floats and Decimals is permitted in the Decimal constructor, create_decimal() and all comparison operators. Both conversion and comparisons are exact. Any occurrence of a mixed operation is silently recorded by setting FloatOperation in the context flags. Explicit conversions with from_float() or create_decimal_from_float() do not set the flag.

Otherwise (the signal is trapped), only equality comparisons and explicit conversions are silent. All other mixed operations raise FloatOperation."

HOWEVER, for some reason, == and != are allowed in floating point strict mode, and produce wrong answers:

Why do you think it's a wrong answer?! 1/10 != (binary approximation of)0.1; the 0.1 can't be represented exactly as binary floating-point number. See https://docs.python.org/3/tutorial/floatingpoint.html#floating-point-arithmetic-issues-and-limitations

So, current behaviour seems to be consistent and well documented. Your only argument against is wrong. And this will break backward compatibility.

To change things you need more arguments. Probably, this should be discussed first on https://discuss.python.org/

[mdickinson]

for some reason, == and != are allowed in floating point strict mode

There's a more general design principle at work here (though it's not one I think I've seen articulated clearly in the docs), not related to float or Decimal specifically: as a rule, an == comparison between two hashable objects shouldn't raise an exception. The reason is that equality checks are performed implicitly and unpredictably in hash-based collections like dict and set.

If == raised as you suggest, then it would be possible to create the set {Decimal("0.47013"), 8946670875749133.0}, but creating a set {Decimal("0.47012"), 8946670875749133.0} would raise (because the hashes of the two elements in the set match, and so an equality test would be invoked to determine whether the elements are actually different, and that equality test would raise). I think that would be rather surprising.

>>> from decimal import Decimal
>>> s = {Decimal("0.47012"), 8946670875749133.0}
>>> list(map(hash, s))
[8946670875749133, 8946670875749133]

[skirpichev]

There's a more general design principle at work here (though it's not one I think I've seen articulated clearly in the docs), not related to float or Decimal specifically: as a rule, an == comparison between two hashable objects shouldn't raise an exception.

Perhaps, most close to documenting it is a quote from here: "When no appropriate method returns any value other than NotImplemented, the == and != operators will fall back to is and is not, respectively."

It doesn't say why this default does exists, however. But I'm not sure it worth.

I think this can be closed.

[timkay]

There's a more general design principle at work here

The general design principle is that floating point strict mode should prevent coders from making mistakes. Sure it's not always a mistake to say Decimal(0.89), if that's what you really intended, but that's almost never what people really want.

>>> Decimal(0.89)
Decimal('0.89000000000000001332267629550187848508358001708984375')

Decimal has lots of use cases, and a common one is to handle dollars-and-cents. If you say

account.balance < amount

You can run into trouble because

>>> Decimal("0.89") < 0.89
True

This sort of problem is flagged by turning on floating point strict mode, and the runtime will flag that statement as incorrect.

The exact same argument applies to equality. For many use cases, equality between Dollar and float is a coding error, and floating point strict mode should catch it.

The idea that things like sets use equality implicitly is interesting. Problem is, most people would be surprised by

>>> 0.89 in set([Decimal("0.89")])
False

What is the use case where equality makes sense between Decimal and float?

As for breaking changes, we could introduce a new flag, StrictMode, which would do what FloatOperation does, but includes equality.

[skirpichev]

What is the use case where equality makes sense between Decimal and float?

When they are mathematically equal.

>>> import decimal
>>> decimal.Decimal(0.1) == 0.1
True
>>> decimal.Decimal(0.89) == 0.89
True

We don't loose anything, unless you apply context settings with insufficient precision:

>>> (+decimal.Decimal(0.1)) == 0.1
False
>>> decimal.Decimal(0.1).as_integer_ratio()
(3602879701896397, 36028797018963968)
>>> (0.1).as_integer_ratio()
(3602879701896397, 36028797018963968)
>>> (+decimal.Decimal(0.1)).as_integer_ratio()
(1000000000000000055511151231, 10000000000000000000000000000)
>>> decimal.getcontext().prec=100
>>> (+decimal.Decimal(0.1)).as_integer_ratio()
(3602879701896397, 36028797018963968)

we could introduce a new flag, StrictMode, which would do what FloatOperation does, but includes equality.

Mark remind us above why we can't raise an exception here.

So, in this case you suggest to fall back on is and set 0.1 != decimal.Decimal(0.1). What we gain here?

[timkay]

I wasn't suggesting that we fall back on is. I was responding to a previous point that equality is used implicitly sometimes, and that's why equality is white listed even in floating point strict mode. But it doesn't make any sense, as that example shows.

To your point, I ask, what is the use case for saying Decimal(0.1)? The correct use is Decimal("0.1").