Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

U64 as displayed in the UI are wrong. #2013

Open
fulmicoton opened this issue Sep 26, 2022 · 6 comments
Open

U64 as displayed in the UI are wrong. #2013

fulmicoton opened this issue Sep 26, 2022 · 6 comments
Labels
bug Something isn't working

Comments

@fulmicoton
Copy link
Contributor

As spotted in #2011.

U64 field are very nice for users indexing ids.
They can be a nicer solution than strings, for instance if we do grouping and the number of distinct values does not justify dicitonary encoding.

Javascript will deserialize their JSON value as a javascript Number, which has a limited precision.
The number as displayed in the UI will hence be erroneous.

This limitation is not really a JSON issue (AFAIK JSON does not
restrict the range of integer values), but is shared by many other programming language.

I am not entirely sure what we should do here tbh. :)

We also probably support string as an input for u64.

On serialize, a common solution to this problem is to add a string version of int fields.

"id": 23423432432424324,
"id_str": "23423432432424324"

But then we could have some collision problems.
We could add a serialization option to serializes all ints as a string.

@fulmicoton fulmicoton added the bug Something isn't working label Sep 26, 2022
@saroh
Copy link
Collaborator

saroh commented Sep 26, 2022

Twitter answered both types and settled on returning strings for integers. https://developer.twitter.com/en/docs/twitter-ids

My 2 cents:

We could add a serialization option to serializes all ints as a string.

If you need to expose an API and not just write an UI, I feel this is best.
Creating data duplication does not seem natural ("I send an id field but need to read an id_str". You can make the point that: "I send a integer value and get a string back" is not really better, at least it gets parsed)
It's something you could set globally in your config / add a specific endpoint for or override via an HTTP parameter.

@PSeitz
Copy link
Contributor

PSeitz commented Sep 27, 2022

We could use a custom parser in the browser https://github.com/josdejong/lossless-json
Parsing is probably much slower though.

@fulmicoton
Copy link
Contributor Author

@PSeitz yeah but other users will suffer right?
@saroh Elastic has u64 and i32 types nowdays right? Do they return string for u64?

@PSeitz
Copy link
Contributor

PSeitz commented Sep 27, 2022

@PSeitz yeah but other users will suffer right?

Yes, every browser UI would need to have some solution for that or may run into the same issue.

@saroh
Copy link
Collaborator

saroh commented Sep 27, 2022

@saroh Elastic has u64 and i32 types nowdays right? Do they return string for u64?

i64 an i32 yes. They keep the data as is meaning the client may have surprises. The same issue is opened with Kibana elastic/kibana#40183

image

@fulmicoton
Copy link
Contributor Author

@saroh thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants