Skip to content
This repository has been archived by the owner on Oct 29, 2024. It is now read-only.

dataframe client precision "ns" causing round error #340

Closed
jevans121 opened this issue Jun 14, 2016 · 0 comments · Fixed by #811
Closed

dataframe client precision "ns" causing round error #340

jevans121 opened this issue Jun 14, 2016 · 0 comments · Fixed by #811

Comments

@jevans121
Copy link

It appears, that the line,
_dataframe_client.py:145
'time': int(ts.value / precision_factor)

Is causing a time rounding error when precision_factor is 'ns'. It is likely to be insufficient precision in the double64 type to hold this many seconds (inc 9 decimal places). The rounding error is in the order of (<100ns). This is consistent with the 15 sig.figures of the double64.

The issue is not present when using the line without the division e.g.:
'time': ts.value

but only for precision_factor, None or 'ns'. Other precisions (unsurprisingly) create time out of bounds on the POST to influxdb.

This line was added at commit:
02ddf0e

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant