Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update DATE and TIME functions to parse string input as datetime #991

Merged
merged 21 commits into from
Nov 8, 2022

Conversation

MitchellGale
Copy link
Contributor

@MitchellGale MitchellGale commented Oct 31, 2022

Description

Add option to accept datetime like string in both TIME and DATE (eg. accept "1999-01-02 12:12:12" for both TIME and DATE.

Strict check on date for testing for valid dates (eg. Don't accept Feb 30th as a valid date) and throws a SemanticCheckException.

opensearchsql> SELECT TIME("1999-02-30 21:07:32");                                                                                                                                 
TransportError(500, 'SemanticCheckException', {'error': {'type': 'SemanticCheckException', 'reason': 'Invalid Query', 'details': 'time:1999-02-30 21:07:32 in unsupported format, please use HH:mm:ss[.SSSSSSSSS]'}, 'status': 400})
opensearchsql> SELECT TIME("1999-02-28 21:07:32");                                                                                                                                 
fetched rows / total rows = 1/1
+-------------------------------+
| TIME("1999-02-28 21:07:32")   |
|-------------------------------|
| 21:07:32                      |
+-------------------------------+
opensearchsql> SELECT TIME("1999-02-19 21:07:32.12345");                                                                                                                           
fetched rows / total rows = 1/1
+-------------------------------------+
| TIME("1999-02-19 21:07:32.12345")   |
|-------------------------------------|
| 21:07:32.12345                      |
+-------------------------------------+
opensearchsql> SELECT DATE("1999-02-19 21:07:32.12345");                                                                                                                           
fetched rows / total rows = 1/1
+-------------------------------------+
| DATE("1999-02-19 21:07:32.12345")   |
|-------------------------------------|
| 1999-02-19                          |
+-------------------------------------+

opensearchsql> SELECT DATE("1999-02-19 21:07:32");                                                                                                                                                                                                          
fetched rows / total rows = 1/1
+-------------------------------+
| DATE("1999-02-19 21:07:32")   |
|-------------------------------|
| 1999-02-19                    |
+-------------------------------+
opensearchsql> SELECT DATE("1999-02-19");                                                                                                                                                                                                                   
fetched rows / total rows = 1/1
+----------------------+
| DATE("1999-02-19")   |
|----------------------|
| 1999-02-19           |
+----------------------+
opensearchsql> SELECT TIME("1999-02-19 21:07:32");                                                                                                                                                                                                          
fetched rows / total rows = 1/1
+-------------------------------+
| TIME("1999-02-19 21:07:32")   |
|-------------------------------|
| 21:07:32                      |
+-------------------------------+
opensearchsql> SELECT TIME("21:07:32");                                                                                                                                                                                                                     
fetched rows / total rows = 1/1
+--------------------+
| TIME("21:07:32")   |
|--------------------|
| 21:07:32           |
+--------------------+

Issues Resolved

#126

Check List

  • New functionality includes testing.
    • All tests pass, including unit test, integration test and doctest
  • New functionality has been documented.
    • New functionality has javadoc added
    • New functionality has user manual doc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

MitchellGale and others added 19 commits October 13, 2022 10:53
…y to parse time out of datetime or date out of datetime.

Signed-off-by: MitchellGale-BitQuill <[email protected]>
Signed-off-by: Yury-Fridlyand <[email protected]>
Signed-off-by: MitchellGale-BitQuill <[email protected]>
Signed-off-by: MitchellGale-BitQuill <[email protected]>
Signed-off-by: MitchellGale-BitQuill <[email protected]>
Signed-off-by: MitchellGale-BitQuill <[email protected]>
Signed-off-by: MitchellGale-BitQuill <[email protected]>
Signed-off-by: MitchellGale-BitQuill <[email protected]>
Signed-off-by: MitchellGale-BitQuill <[email protected]>
Signed-off-by: MitchellGale-BitQuill <[email protected]>
Signed-off-by: MitchellGale-BitQuill <[email protected]>
…prDateValue.java for timestampValue to use systemDefault instead of UTC time.

Signed-off-by: MitchellGale-BitQuill <[email protected]>
@MitchellGale MitchellGale requested a review from a team as a code owner October 31, 2022 09:58
@codecov-commenter
Copy link

codecov-commenter commented Oct 31, 2022

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.70%. Comparing base (b30d156) to head (68b9a26).
Report is 528 commits behind head on 2.x.

Additional details and impacted files
@@             Coverage Diff              @@
##                2.x     #991      +/-   ##
============================================
- Coverage     97.90%   95.70%   -2.21%     
- Complexity     3072     3344     +272     
============================================
  Files           293      336      +43     
  Lines          7588     9101    +1513     
  Branches        490      672     +182     
============================================
+ Hits           7429     8710    +1281     
- Misses          158      334     +176     
- Partials          1       57      +56     
Flag Coverage Δ
query-workbench 62.76% <ø> (?)
sql-engine 98.27% <100.00%> (+0.36%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

MaxKsyunz
MaxKsyunz previously approved these changes Oct 31, 2022
@dai-chen dai-chen added the enhancement New feature or request label Oct 31, 2022
MaxKsyunz
MaxKsyunz previously approved these changes Nov 3, 2022
@MitchellGale MitchellGale requested review from Yury-Fridlyand and removed request for Yury-Fridlyand November 3, 2022 18:27
@MitchellGale MitchellGale requested a review from dai-chen November 3, 2022 19:08
dai-chen
dai-chen previously approved these changes Nov 3, 2022
Copy link
Collaborator

@dai-chen dai-chen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a minor comment. Thanks for the changes!

Signed-off-by: MitchellGale-BitQuill <[email protected]>
@MitchellGale MitchellGale dismissed stale reviews from dai-chen and MaxKsyunz via 68b9a26 November 8, 2022 18:17
@MitchellGale MitchellGale requested a review from dai-chen November 8, 2022 18:44
Copy link
Collaborator

@dai-chen dai-chen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the changes!

@MitchellGale MitchellGale requested review from MaxKsyunz and Yury-Fridlyand and removed request for MaxKsyunz and Yury-Fridlyand November 8, 2022 19:25
@MitchellGale MitchellGale changed the title Date and Time function update Accept datetime like strings in time and date functions & allow time gotten from date value Nov 8, 2022
@MitchellGale MitchellGale changed the title Accept datetime like strings in time and date functions & allow time gotten from date value Accept datetime like strings in time and date functions Nov 8, 2022
@MitchellGale MitchellGale changed the title Accept datetime like strings in time and date functions Update DATE and TIME functions to parse string input as datetime Nov 8, 2022
@MaxKsyunz MaxKsyunz merged commit 8c91689 into opensearch-project:2.x Nov 8, 2022
@MaxKsyunz MaxKsyunz deleted the Integ-updateDateFunction branch November 8, 2022 19:41
dai-chen added a commit that referenced this pull request Nov 9, 2022
* Fix `FLOAT` -> `DOUBLE` cast. (#1025)

Signed-off-by: Yury-Fridlyand <[email protected]>

* Fix error messaging from prometheus. (#1029) (#1037)

Signed-off-by: vamsi-amazon <[email protected]>
(cherry picked from commit 4a9cef3)

Co-authored-by: vamsi-amazon <[email protected]>

* Add `query` function as alternate syntax to `query_string` function (#1010)

This maintains backwards compatibility with the v1 engine.

* Update DATE and TIME functions to parse string input as datetime (#991)

Add option to accept datetime like string in both TIME and DATE (eg. accept "1999-01-02 12:12:12" for both TIME and DATE.

Strict check on date for testing for valid dates (eg. Don't accept Feb 30th as a valid date) and throws a SemanticCheckException.

Co-authored-by: Yury-Fridlyand <[email protected]>
Signed-off-by: MitchellGale-BitQuill <[email protected]>

* back quote fix (#1041) (#1050)

Signed-off-by: vamsi-amazon <[email protected]>
(cherry picked from commit d3bb902)

Co-authored-by: vamsi-amazon <[email protected]>

* Catalog to Datasource changes (#1027) (#1049)

Signed-off-by: vamsi-amazon <[email protected]>
(cherry picked from commit 3e30379)

* Bump jackson to 2.14.0 (#1058)

Signed-off-by: Joshua Li <[email protected]>
(cherry picked from commit 5a1adb2)

* Add valueOf() to Expression (#1055)

Signed-off-by: Joshua Li <[email protected]>

Signed-off-by: Yury-Fridlyand <[email protected]>
Signed-off-by: MitchellGale-BitQuill <[email protected]>
Signed-off-by: Joshua Li <[email protected]>
Signed-off-by: Chen Dai <[email protected]>
Co-authored-by: Yury-Fridlyand <[email protected]>
Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>
Co-authored-by: vamsi-amazon <[email protected]>
Co-authored-by: GabeFernandez310 <[email protected]>
Co-authored-by: MitchellGale-BitQuill <[email protected]>
Co-authored-by: Joshua Li <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants