Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Dissect ingest processor cannot parse empty value with brackets #7223

Closed
gaobinlong opened this issue Apr 18, 2023 · 4 comments
Closed
Assignees
Labels
bug Something isn't working Search:Relevance Search Search query, autocomplete ...etc

Comments

@gaobinlong
Copy link
Collaborator

Describe the bug
Dissect ingest processor cannot parse empty value with brackets like [], {}, or () correctly, when there exists an empty value with brackets in the text incidentally, the dissect processor throws exception Unable to find match for dissect pattern....

To Reproduce
Steps to reproduce the behavior:

  1. Create an ingest pipeline:
PUT _ingest/pipeline/test
{
  "processors": [
      {
        "dissect": {
          "field": "message",
          "pattern": "[%{@timestamp}][%{logLevel}][%{class}]"
        }
      }
    ]
}
  1. Ingest some documents and using the above pipeline:

normal case:

POST test1/_doc?pipeline=test
{
  "message":"[2023-04-13T15:09:03,947][INFO][o.o.n.Node]"
}

abnormal case, the field message contains an empty value with brackets:

POST test1/_doc?pipeline=test
{
  "message":"[2023-04-13T15:09:03,947][INFO][]"
}

you can see the error below:

{
  "error": {
    "root_cause": [
      {
        "type": "find_match",
        "reason": "Unable to find match for dissect pattern: [%{@timestamp}][%{logLevel}][%{class}] against source: [2023-04-13T15:09:03,947][INFO][]"
      }
    ],
    "type": "find_match",
    "reason": "Unable to find match for dissect pattern: [%{@timestamp}][%{logLevel}][%{class}] against source: [2023-04-13T15:09:03,947][INFO][]"
  },
  "status": 500
}

Expected behavior
The dissect ingest processor can handle the text containing empty value with brackets gracefully, assign the value to empty string for the matched field.

Screenshots
image

Host/Environment (please complete the following information):

  • OS: [Mac OS]
  • Version [OpenSearch 2.6.0]
@gaobinlong gaobinlong added bug Something isn't working untriaged labels Apr 18, 2023
@dblock
Copy link
Member

dblock commented Apr 19, 2023

Looks like a bug :( Want to try to turn it into a YAML test and maybe fix?

@gaobinlong
Copy link
Collaborator Author

Sure, I will try to fix it and add some test code.

@msfroh
Copy link
Collaborator

msfroh commented Sep 18, 2023

@gaobinlong -- are you still planning to work on this one?

Nevermind, I see the linked PR. Will review that.

@gaobinlong
Copy link
Collaborator Author

Closed by #9255.

@github-project-automation github-project-automation bot moved this from 🆕 New to ✅ Done in Search Project Board Jan 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Search:Relevance Search Search query, autocomplete ...etc
Projects
Archived in project
Development

No branches or pull requests

6 participants