You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When I run this in VS FSI (tested both x86 and x64, it only happens on x86) I get really odd behavior. In dotnet FSI it doesn't happen with this particular scenario (though the result of cos 1.0 is different for dotnet).
I'm aware that trig functions give different results depending on runtime, version and bitness, but within one session it should be stable. I put this in the "very odd" category for now, though I hope I'm just wrong and missing the obvious:
Repro steps
Copy these lines one by one:
>letx= cos 1.0;;valx:float =0.5403023059> x =0.54030230586813977;;// use literalvalit:bool =true // expected> cos 1.0=0.54030230586813977;;// use exactly the same literalvalit:bool =false // certainly NOT expected
Sanity check:
>letx= cos 1.0in x =0.54030230586813977;;valit:bool =false // certainly NOT expected
How can the assigned floating point value be different from the floating point value used in the last comparison? I would expect this to be stable.
EDIT: marked a regression, as I cannot repro this in VS 2017 using the same settings, so this was introduced at some point after 2017.
The text was updated successfully, but these errors were encountered:
abelbraaksma
changed the title
Assigned value from trig function compares equal, direct comparison compare unequal
Regression: Assigned value from trig function compares equal, direct comparison compare unequal
Jun 22, 2020
This is actually by design. Floating point equality just has a lot of vagaries like this and it is not expected in .NET floating point that you can accurately do equality in all situations and get exact results like you're expecting.
I don't know the best reference for this - but I believe it's common enough knowledge that examples like the ones you're showing will happen depending on register allocations etc.
@dsyme, tx, I agree. I've done a fair bit of research since and thought I closed it already.
It's odd, of course, that the value you read as a constant does not compare equal to the same constant, but I've come to understand that this can happen if your compiler compiles with, say, netfx, and your runtime env is Netcore.
It can even happen on cpu level, where one architecture rounds slightly different than another architecture. I've changed certain tests a few months back in the F# repo to test with a certain wiggle room.
When I run this in VS FSI (tested both x86 and x64, it only happens on x86) I get really odd behavior. In dotnet FSI it doesn't happen with this particular scenario (though the result of
cos 1.0
is different for dotnet).I'm aware that trig functions give different results depending on runtime, version and bitness, but within one session it should be stable. I put this in the "very odd" category for now, though I hope I'm just wrong and missing the obvious:
Repro steps
Copy these lines one by one:
Sanity check:
How can the assigned floating point value be different from the floating point value used in the last comparison? I would expect this to be stable.
EDIT: marked a regression, as I cannot repro this in VS 2017 using the same settings, so this was introduced at some point after 2017.
The text was updated successfully, but these errors were encountered: