-
Notifications
You must be signed in to change notification settings - Fork 6.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Calculator output losing precision #18574
Comments
@htcfreek Did you mean to link to this line? |
I think it has moved after merging the other PR today. |
Ahh. Which line were you intended to point to? |
65 |
Oh wow. I know what's causing this. It's a double to decimal conversion bug in .NET. dotnet/runtime#68042 (comment). Believe it or not, the reason I was using the calculator with those inputs is because I was working on fixing this exact bug. I should have noticed that the calculator was displaying a suspiciously familiar output. Long story short, this issue should be fixed soon once I fix that bug. In the meantime, I'll have to use a calculator not built on .NET 😄. |
@dakersnar
|
@htcfreek Just the first. As Tanner mentions in that comment, the overflow "bug" brought up by the original post is actually expected. The precision loss, however, is a bug. |
@dakersnar |
@crutkas , @jaimecbernardo |
@htcfreek I was not aware of that issue. In the example you linked, do you happen to know the exact double that is being passed into the decimal converter? If the output of the converter is inaccurate, and we can reproduce it, I can add it as a unit test to ensure I fix it during my refactor. |
@dakersnar I expected that without any calculation that input equals the output. But there comes one decimal digit where the output starts to change. Please test it number per number. Not sure how many you need. |
@htcfreek I am able to reproduce that in PT, but not in C# with a simple conversion. https://dotnetfiddle.net/cB36u7 (Notably, I don't include the "cultureInfo" argument that the PT code does, although I don't know if that changes things.) Looking at the output of the unit test I wrote, would you agree that the overflow bug is not specifically caused by double to decimal conversion? Please let me know if you find a flaw in my thinking. Interestingly, it looks like there is some precision loss with this test, so while I can't reproduce the overflow bug, this does provide another test case for my original bug. |
@dakersnar can't do it on my mobile |
We have more precision here, which backs up my thought that the conversion precision bug affects this input. |
@dakersnar
|
Sounds good. And like I said, I'll keep you updated when I fix the precision bug. |
Aaah. The input in the test code here becomes incorrect on |
Does it? I want to point out that |
I thought the output of x has to stop at the last 5 or at least fill with zero. Wonder where all these digits come from. Or is there something mathematics that I don't have in mind? |
Floating-point numbers are inherently inaccurate, so in this case, that is the closest memory representation of the input we give it. You can see in this online calculator that it tells us that the "Most accurate representation" of our input is similar to what we see in the C# output. https://www.binaryconvert.com/result_double.html?decimal=048046053053053053053053053053053053053053053053053053053053053 |
@jaimecbernardo , @dakersnar |
@jaimecbernardo |
Looks like mages has a number of problems. I think #2265 would solve this (?) |
I think moving to calc engine would help us centralize issues. Mage's is open source as well so we could do the PR against Mages but 100% we should get their opinion on the issue just like we like feedback |
end of day, this is a /dup #2265 as that is where we've been funneling these against |
Hi! We've identified this issue as a duplicate of another one that already exists on this Issue Tracker. This specific instance is being closed in favor of tracking the concern over on the referenced thread. Thanks for your report! |
@crutkas Is there any information about the state of calc engine and how long it need to get ready for our use case? |
The bug in mages is fixed (FlorianRappl/Mages#110) and the fix will be available in v2.0.1 of mages. Can we then update the engine reference and test this? |
Hi @htcfreek , I think the presented calculations in this issue are still not working. (might be something on our end, though) |
Microsoft PowerToys version
0.57.0
Running as admin
Area(s) with issue?
PowerToys Run
Steps to reproduce
Type
9007199254740991 * 2^43
into the PowerToys Run bar.✔️ Expected Behavior
❌ Actual Behavior
Other Software
No response
The text was updated successfully, but these errors were encountered: