Add WindowStart and WindowEnd UDFs (#1993) #2090
Conversation
* add WindowStartTime UDAF (cherry picked from commit 82b603d)
| @@ -272,6 +272,21 @@ counting/aggregation step per region. | |||
| WINDOW SESSION (60 SECONDS) \ | |||
| GROUP BY regionid; | |||
|
|
|||
| Sometimes you may want to include the bounds of the current window in the result so that it is | |||
There was a problem hiding this comment.
| Sometimes you may want to include the bounds of the current window in the result so that it is | |
| Sometimes, you may want to include the bounds of the current window in the result, so that it is |
| @@ -272,6 +272,21 @@ counting/aggregation step per region. | |||
| WINDOW SESSION (60 SECONDS) \ | |||
| GROUP BY regionid; | |||
|
|
|||
| Sometimes you may want to include the bounds of the current window in the result so that it is | |||
| more easily accessible to consumers of the data. The statement below extracts the start and | |||
There was a problem hiding this comment.
| more easily accessible to consumers of the data. The statement below extracts the start and | |
| more easily accessible to consumers of the data. The following statement extracts the start and |
| +------------------------+---------------------------+------------+---------------------------------------------------------------------+ | ||
| | WindowEnd | ``WindowEnd()`` | Stream | Extract the end time of the current window, in milliseconds. | | ||
| | | | Table | If the query is not windowed the function will return null. | | ||
| +------------------------+---------------------------+------------+---------------------------------------------------------------------+ |
There was a problem hiding this comment.
I have some concern about presenting this as an Aggregate Fn / UDAF, at least to the end-user. It's a little bit of a grey-zone and i understand why, mechanically, it's easier to implement this way but semantically it's not a "reducing function" that operates to combine the values from multiple grouped values. Rather, it's a function which operates on the group as a whole, once it's been reduced to a single row. In my mind that makes this smell different than a UDAF.
I guess it's also not really a scalar fn either - although it's closer to being that than an aggregator - so possibly a third category of "Other Functions" is the best way to handle it. WDYT ?
Also, i realise i should probably have made this comment on the original PR but only just noticed it....
There was a problem hiding this comment.
I know what you mean, though don't have a strong feeling on this in terms of the documentation. So happy to hear from others.
I'm also looking to extend this functionality to more than just aggregates, so will look to update the docs more then if we've not come to any decision here.
There was a problem hiding this comment.
I can see two places where a user would be looking for this information:
- In a section on Windowing (aka windowed aggregations), because "I am working with windows, so show me in one place all info I need on how to deal with windows".
- In a section on built-in "Functions" (KDF, KDAF, etc.), because "I remember it's a function call to get this info, so I'll check the place in the docs where all functions are listed".
We don't have a dedicated docs section yet that covers Windowing (cf. (1) above), but @JimGalasyn is working on that.
Description
(cherry picked from commit 82b603d)
Original PR: #1993