Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

計算途中の浮動小数点誤差をなくす #18

Merged
merged 1 commit into from
Oct 13, 2023

Conversation

ciscorn
Copy link
Member

@ciscorn ciscorn commented Oct 13, 2023

計算中の経緯度を実際の経緯度の30倍にしておくことで、計算過程での浮動小数点誤差の発生をなくします。

説明

そのまま計算すると、計算中の 3 による除算1回と 10 による除算1回によって近似値が発生してしまう。あらかじめ内部的な数値に30をかけておけばそれらを回避できる。そのほかの、2 や 8 による除算(2の累乗による除算)では、2進法における近似値は発生しない。

注意

今後他の特殊なメッシュ(100mメッシュなど)に対応する場合は注意が必要。

@codecov
Copy link

codecov bot commented Oct 13, 2023

Codecov Report

Attention: 15 lines in your changes are missing coverage. Please review.

Comparison is base (c16024c) 26.91% compared to head (3ad955f) 26.98%.
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #18      +/-   ##
==========================================
+ Coverage   26.91%   26.98%   +0.06%     
==========================================
  Files           9        9              
  Lines         561      567       +6     
==========================================
+ Hits          151      153       +2     
- Misses        410      414       +4     
Flag Coverage Δ
unittests 26.98% <31.81%> (+0.06%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
japanese_grids/algorithms/utils/grid_square.py 12.37% <31.81%> (+1.38%) ⬆️

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ciscorn ciscorn merged commit 4c228c4 into main Oct 13, 2023
4 checks passed
@ciscorn ciscorn deleted the feature/eliminate-fp-error branch October 13, 2023 17:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant