Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added New Fast DateTime parser implementation for RFC3339 Datetime format #11465

Merged
merged 1 commit into from
Jan 24, 2024

Conversation

CaptainDredge
Copy link
Contributor

@CaptainDredge CaptainDredge commented Dec 5, 2023

Description

This PR adds new rfc3339 format for datetime parsing whose implementation is not based on the usual java datetime formatters but is hand written to make it extremely fast. RFC3339 format is one of the most used format for datetime across internet. The parser implements a lenient version of rfc3339 which aligns more with W3C Note.
This implementation is 42x faster than strict_date_optional_time_format to parse rfc3339 compatible datetime strings.

Microbenchmark results

DocumentParsingBenchmark.benchRFC3339Parser               2023-01-01T23:38:34.000Z  avgt   30    66.321 ±  0.424  ns/op
DocumentParsingBenchmark.benchRFC3339Parser               1970-01-01T00:16:12.675Z  avgt   30    68.199 ±  1.341  ns/op
DocumentParsingBenchmark.benchRFC3339Parser               5050-01-01T12:02:01.123Z  avgt   30    66.213 ±  0.311  ns/op
DocumentParsingBenchmark.strictDateOptionalTimeFormatter  2023-01-01T23:38:34.000Z  avgt   30  2732.462 ± 10.043  ns/op
DocumentParsingBenchmark.strictDateOptionalTimeFormatter  1970-01-01T00:16:12.675Z  avgt   30  2794.871 ±  8.151  ns/op
DocumentParsingBenchmark.strictDateOptionalTimeFormatter  5050-01-01T12:02:01.123Z  avgt   30  2779.538 ± 13.909  ns/op

Related Issues

Resolves #8361

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)
  • Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link
Contributor

❌ Gradle check result for 9be85bf: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

✅ Gradle check result for c325fb2: SUCCESS

@CaptainDredge
Copy link
Contributor Author

@reta can you please take a final look now?

@mgodwan mgodwan self-requested a review January 24, 2024 14:13
@reta
Copy link
Collaborator

reta commented Jan 24, 2024

@reta can you please take a final look now?

@CaptainDredge Just a question, rfc3339_strict we decided not to implement for now, right? thanks!

@CaptainDredge
Copy link
Contributor Author

@CaptainDredge Just a question, rfc3339_strict we decided not to implement for now, right? thanks!

Yes exactly @reta. I can take it up in a follow up PR. I've one follow up PR for supporting local datetime fully which most logging libraries use after this and then will see if I can add support for rfc3339 strict as well

@reta reta merged commit 829215c into opensearch-project:main Jan 24, 2024
34 checks passed
@reta reta added the backport 2.x Backport to 2.x branch label Jan 24, 2024
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.x
# Create a new branch
git switch --create backport/backport-11465-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 829215c4a4660ee65026c40a0a01ebe100b3ee2c
# Push it to GitHub
git push --set-upstream origin backport/backport-11465-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-11465-to-2.x.

@reta
Copy link
Collaborator

reta commented Jan 24, 2024

@CaptainDredge could you please backport manually to 2.x? thank you

@CaptainDredge
Copy link
Contributor Author

Thanks for merging! I'll raise backport to 2.x manually soon

@vmmusings
Copy link
Member

There is a unit test failure in sql with the introduction of a new Format Name. We have backported our fix to 2.x but core 2.x still doesn't have the new FormatName. @CaptainDredge can we expedite backport to 2.x as 2.12 is nearing the code freeze date.

CaptainDredge added a commit to CaptainDredge/OpenSearch that referenced this pull request Jan 31, 2024
Signed-off-by: Prabhat Sharma <[email protected]>
Co-authored-by: Prabhat Sharma <[email protected]>
(cherry picked from commit 829215c)
CaptainDredge added a commit to CaptainDredge/OpenSearch that referenced this pull request Jan 31, 2024
Signed-off-by: Prabhat Sharma <[email protected]>
Co-authored-by: Prabhat Sharma <[email protected]>
(cherry picked from commit 829215c)
CaptainDredge added a commit to CaptainDredge/OpenSearch that referenced this pull request Jan 31, 2024
Signed-off-by: Prabhat Sharma <[email protected]>
Co-authored-by: Prabhat Sharma <[email protected]>
(cherry picked from commit 829215c)
CaptainDredge added a commit to CaptainDredge/OpenSearch that referenced this pull request Jan 31, 2024
Signed-off-by: Prabhat Sharma <[email protected]>
Co-authored-by: Prabhat Sharma <[email protected]>
(cherry picked from commit 829215c)
@CaptainDredge
Copy link
Contributor Author

@vamsi-amazon I've raised the backport #12107. Thanks!

reta pushed a commit that referenced this pull request Jan 31, 2024
Signed-off-by: Prabhat Sharma <[email protected]>
Co-authored-by: Prabhat Sharma <[email protected]>
(cherry picked from commit 829215c)
peteralfonsi pushed a commit to peteralfonsi/OpenSearch that referenced this pull request Mar 1, 2024
rayshrey pushed a commit to rayshrey/OpenSearch that referenced this pull request Mar 18, 2024
shiv0408 pushed a commit to Gaurav614/OpenSearch that referenced this pull request Apr 25, 2024
Signed-off-by: Prabhat Sharma <[email protected]>
Co-authored-by: Prabhat Sharma <[email protected]>
Signed-off-by: Shivansh Arora <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch backport-failed enhancement Enhancement or improvement to existing feature or request Indexing Indexing, Bulk Indexing and anything related to indexing v2.12.0 Issues and PRs related to version 2.12.0 v3.0.0 Issues and PRs related to version 3.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Performance] Improve performance for date field parsing
6 participants