-
Notifications
You must be signed in to change notification settings - Fork 25k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Esql compare nanos and millis #118027
Esql compare nanos and millis #118027
Changes from 20 commits
8b17ecb
7544b5d
25646bc
1bc3707
b5dae19
74b970f
edc4e3a
020b83c
9932bc7
35f2d4a
cae21a3
a4a2806
add783a
a7a30be
2d03fdc
1395a80
292d8b6
7eef795
d4dc498
3bb1417
4dbdff9
78e8b9d
7048e44
44790fd
af8dab8
b520da0
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
pr: 118027 | ||
summary: Esql compare nanos and millis | ||
area: ES|QL | ||
type: enhancement | ||
issues: | ||
- 116281 |
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -293,6 +293,17 @@ public static long toMilliSeconds(long nanoSecondsSinceEpoch) { | |
return nanoSecondsSinceEpoch / 1_000_000; | ||
} | ||
|
||
public static int compareNanosToMillis(long nanos, long millis) { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this comparing sub-second durations, or epoch times? I think it's the former, but please add javadocs to make clear the expectations for input and output. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Javadoc added. The intent is in fact to compare epoch dates in the two different formats. The intent here is to support an ES|QL use case comparing fields of different types. This seemed like a good place to put it, since we have a bunch of other functions for dealing with ES date_nanos here, but I can move it if this class isn't appropriate. |
||
if (millis < 0) { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If this is sub-second duration (see question above), how can millis be negative? If it's epoch times, why would negative millis the nanos are always greater? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Elasticsearch doesn't allow nanosecond resolution dates before epoch, so a millisecond date before epoch will, by definition, be before any valid nanosecond date. See, for example, There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We didn't support passing negative epoch millis until recently. Rather than make this assumption, why not make the method correctly compare the two? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I didn't do that because we then need to deal with the long overflow case in the comparison. Right now, The assumption that nanosecond dates will never be negative is fairly deeply baked into the system, and we would need to change a lot of things before we could support that, not just this method. I could change this to read There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Could you add an assert that nanos >= 0? That would then break if we try adding negative support in the future, so modifying this method is not missed. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. good idea. Done. |
||
return 1; | ||
} | ||
if (millis > MAX_NANOSECOND_IN_MILLIS) { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This seems like it should be a precondition. Why would we have millis outside the expected input bounds? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The intent here is to support comparing user data stored in the two different resolutions. So in this case, the millis aren't outside the expected input bounds, as any long-representable datetime should be a valid input. A user might have a query like |
||
return -1; | ||
} | ||
long diff = nanos - (millis * 1_000_000); | ||
return diff == 0 ? 0 : diff < 0 ? -1 : 1; | ||
} | ||
|
||
/** | ||
* Rounds the given utc milliseconds sicne the epoch down to the next unit millis | ||
* | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -726,7 +726,7 @@ public static Literal randomLiteral(DataType type) { | |
case UNSIGNED_LONG, LONG, COUNTER_LONG -> randomLong(); | ||
case DATE_PERIOD -> Period.of(randomIntBetween(-1000, 1000), randomIntBetween(-13, 13), randomIntBetween(-32, 32)); | ||
case DATETIME -> randomMillisUpToYear9999(); | ||
case DATE_NANOS -> randomLong(); | ||
case DATE_NANOS -> randomLongBetween(0, Long.MAX_VALUE); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is unrelated, but has been wrong for a while. I changed how we were logging dates in tests, which then triggered an issue with negative values here, thus needing to fix it now. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nit, could be simplified to |
||
case DOUBLE, SCALED_FLOAT, COUNTER_DOUBLE -> randomDouble(); | ||
case FLOAT -> randomFloat(); | ||
case HALF_FLOAT -> HalfFloatPoint.sortableShortToHalfFloat(HalfFloatPoint.halfFloatToSortableShort(randomFloat())); | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the main implementation. All the specific operators are implemented in terms of this method.