Proper JSON Serialization of Long/BigInt/Int64

[travisgosselin]

From @nickclarity

Regarding serialization and types in JSON, how should we represent a Long/Big Int/64 Bit Int? The SIP specifically refers to Number as 32 bit, would that imply Longs should be represented as Strings?

https://atlassian.spscommerce.com/wiki/display/SIP/API+-+Serialization#APISerialization-Number

I know as far as JSON is concerned it doesn't matter, it's just a serialization format. But what if the client of the service is JavaScript?

[travisgosselin]

Great callout @nickclarity. I believe your interpretation to be closest to the original design of the standards.

Basically, 64-bit integers are a problem for compatibility in some API consumers, mainly we are talking JavaScript as you suggest (I'm not sure I can think of another). BUT that is a pretty big concern for some APIs. So some of this depends on knowing the consumer of your API to make the best decision. However, building an API platform moving forward, it is fair to expect that you have to build for the lowest common parser, and I have updated the API standards to indicate that with some more specific details that apply here.

A quick overview:

• Numbers should only be 32-bit integers - anything else is a string. 64-bit integers are too big and floating-point (decimal, percentages) may lose precision in some consumers. Sure you could get nitpicky and pick apart certain scenarios where its ok, or suggest you don't need all 64-bit... but ideally it is just simply to stick to the well-known cross-compatible conventions. And sure BigInt is widely supported these days (https://caniuse.com/bigint) ... but perhaps not as well adopted by the developer as we might hope :). This can be easily represented in JSON schema/ Open API data types by specifying the type as "string" and the format as "int64" for example - and this methodology is a well understand best practice.
• If you are in a situation and require a 64-bit value and you simply MUST have it as a number/integer type in serialization, then the suggestion would be to provide both (i.e. largeValue and largeValueString). Kind of gross, but if you are at the point of needing the necessary evil, it works and is pretty clear.
• Special case if you are trying to serialize an ID! IDs ideally in a perfect world are never something we expose as integers as architecturally it relies on centralization of the ID curation, allows for ID guessing flaws/security, and also implies a level of usage when you see it incrementing. But in reality, we have them everywhere. We should always serialize IDs as strings to indicate they are opaque values not intended to be parsed to many anything more. The worst-case scenario I can imagine is a 64-bit integer ID value (serialized as such) coming back to a JS front-end that has a loss of precision on it and interprets the value as an entirely different value then being passed back to update or save (assuming it would just fail without access to that ID, but the potential for updating the completely wrong value). I may get flack on this one - open to hearing more thoughts...

Updates: https://atlassian.spscommerce.com/wiki/display/SIP/API+-+Serialization#APISerialization-Number

[travisgosselin]

@nickclarity can you provide more detail on how you intend to use the 64-bit integer and the context of the property?

[nickclarity]

DEX has a number of endpoints where we take in values and just pass them along as requests to other APIs. We don't currently use 64-bit integers "natively" as we don't expose any entities.

The example I ran into yesterday was regarding the DC4 Company ID and the NCS service. The "id" is define as a 64-bit integer and serialized as a Number. As Evan pointed out, the Rules Management API also has several spots where it has defined IDs as 64-bit integer and serialized as a Numbers.

[eggilbert]

As someone involved in both of those projects, that was just a bit of the implementation details leaking in the Swagger/OpenAPI generation. Both of those projects model the database primary key as a java Long and so the OpenAPI generator sees that makes it a number with a format of $int64. In neither case do the numbers actually get that big (but in theory they could, someday).

[shifr]

We have orgId in the Identity service represented to clients as a string but the value is a decimal with max digits at 40. So, I agree with the overview, internally we could handle numbers as we want but expose it as a string.

[JamesStauffer]

"IDs ideally in a perfect world are never something we expose as integers"
Is that is an SPS standard somewhere?

[travisgosselin]

@JamesStauffer - great question. My distant and longer-term opinion of API design was leaking a bit into this. There is no formal standard that expresses this opinion yet. Though, as we mature in API Design it would be something I would be interesting in pursuing more in the future as a best practice part of these standards. The general sentiment of IDs and interoperability and developer experience are portrayed pretty welll as an overview in this article: https://apihandyman.io/how-to-choose-ids-and-codes-to-build-userfriendly-and-interoperable-apis/

There is also some discussion hidden in here: #5 addressing a longer term need for a standard URN style SPS interopable ID... like spsItemId that would be a specialized format: "sps:item:123456".

Again nothing introduced formally at SPS about these more general industry trends.