Should created and modified times be precise to exactly 3 digits after the decimal place in seconds?

[rpiazza]

For created and modified properties the spec says:

The object creator can use the time it deems most appropriate as the time the object was created, but it MUST be precise to the nearest millisecond (exactly three digits after the decimal place in seconds).

Maybe the spec didn't mean to imply this - but it seems to say that the time should not have more than three digits after the decimal place in seconds

During implementation of python_stix2, when testing the new_version functionality, two versions were made within the same millisecond - which means that these two object versions are indistinguishable. The solution was to add a millisecond if a version already exists with the exact same modified time.

I wonder if this is the best way to deal with this situation. If we could have more precision (i.e., more than 3 digits), we could distinguish the two without putting one of them into "the future".
This could get even messier if you could make "n" versions within the same millisecond.

So instead of:
"2020-02-07T14:02:17.833Z"
"2020-02-07T14:02:17.834Z"

it would be:

"2020-02-07T14:02:17.833Z"
"2020-02-07T14:02:17.8331Z"

And the spec text could read:

The object creator can use the time it deems most appropriate as the time the object was created, but it MUST be precise to the at_least the nearest millisecond (three digits after the decimal place in seconds).

[StephenOTT]

nice find!

[johnwunder]

This was intentional, I can help explain the original reasoning behind it. Like many things, it goes back to a compromise...in this case, the Great Timestamp Precision Debate of 2017 (or whenever).

We were worried about the ambiguity of the exact situation you're mentioning:

• One timestamp is 17.883Z
• One timestamp is 17.8831Z

Per our precision rules, the first timestamp isn't implicitly 17.88300000..., instead, it's any time to the specified level of precision. In other words, the first timestamp is really saying anything between 17.8830 and 7.8831.

Given the lack of precision, if you follow the rules exactly you wouldn't know for sure whether the second one should come before or after the first.

Obviously we can go back and revisit...I think in a few cases we wrote ourselves into weird corners in an effort to make as many people happy as possible.

[rpiazza]

@johnwunder -
Yes, John - I remember the Great Timestamp Precision Debate of 2017 - and I was hesitant to bring this up again. The implementation solution of adding a millisecond might be the best answer - but it made me uneasy - so I though it was worth talking about --- again. Additionally, during that debate there were few, if any implementations - so we were discussing this as a "hypothetical". Now we have a real use case - I'm not sure if it was one of the corner cases considered previously.

[jordan2175]

I think the key is that our original compromise was based on a hypothetical and theoretical discussion. Now we are starting to get real implementations. Also the debate was around, what should we do and some people saying that their devices can not produce timestamps with more precision than a 1 second level. Others said that milli second was sufficient.

Maybe we should say that people SHOULD use micro second level precision but MAY milli second if their product does not support microsecond level precision.

[johnwunder]

Yeah it seems like it should be changed. One option is to just say that when comparing values for versioning purposes you just assume infinite precision (e.g. "27.5" becomes "27.500000000..." and is therefore smaller than "27.51"). This is probably what would happen when you parsed those numbers in most libraries anyway.

[jordan2175]

Talked about this on the 2020-02-18 working call and the consensus was to make a change that said you "MUST use at least mili-second precision". John-Mark is going to propose some text to cover a corner case.

[jmgnc]

Suggested additional text:
The minimum precision MUST be milliseconds, but may be more precise. Two times MUST NOT be equal if the more precise time is truncated to the less precise time. For example, it is invalid to have one object that has it's sub-second precision be .384, and a second object with .384834. If you truncated the second time to match the millisecond precision of the first, they would be equal. This prevents confusion on ordering and storing, but allows the author to increase precision in the future if needed.

Note that precisions larger than nanosecond may not be very interoperable or usable, as conversion to UNIX epoch time stamps are often either limited to nanoseconds, or when stored as a floating point number, would lose precision in single precision.

Not sure if we want to add an implementation consideration about incrementing modified time of the new object or not.

[jordan2175]

We added this text to 2.16.1

Note when using precisions greater than nanoseconds there may be implications for interoperability as they may be truncated when stored as a UNIX timestamp or floating point number due to the fundamental precision of those formats.

[jordan2175]

We replaced text in 3.2 for created and modified to say

The object creator can use the time it deems most appropriate as the time the object was created. The minimum precision MUST be milliseconds (three digits after the decimal place in seconds), but may be more precise.

and

The object creator can use the time it deems most appropriate as the time this version of the object was modified. The minimum precision MUST be milliseconds (three digits after the decimal place in seconds), but may be more precise.

[jordan2175]

Rich P and I talked about the other suggested text and we do not feel that it is actually required due to the fact that the timestamp value is made up of more than just the precision. So if, in the future a producer creates a new version of the object, and is now using micro second precision, then the rest of the timestamp value will be different, not just the precision.

If we are missing something, or did not understand your suggestion correctly, please let us know.