Fix Ingest Pipelines Grok processor to accept patterns that contain escaped characters #137245
Conversation
…scaped characters.
cjcenizal
added
release_note:fix
Team:Kibana Management
|
Pinging @elastic/platform-deployment-management (Team:Deployment Management) |
💚 Build Succeeded
Metrics [docs]Async chunks
History
To update your PR or re-run it, just comment with: |
| // Add "field" value | ||
| form.setInputValue('fieldNameField.input', 'test_grok_processor'); | ||
|
|
||
| // Add the escaped value of \[%{HTTPDATE:timestamp}\]%{SPACE}\"%{WORD:http_method}%{SPACE}HTTP/%{NUMBER:http_version}\" |
There was a problem hiding this comment.
nit: seems a bit redundant to me to write the whole escaped string in this comment
| // Add the escaped value of \[%{HTTPDATE:timestamp}\]%{SPACE}\"%{WORD:http_method}%{SPACE}HTTP/%{NUMBER:http_version}\" | |
| // Add the escaped value into the input |
There was a problem hiding this comment.
I hear you. It's definitely repetitive. TBH I liked having it because it saved me the trouble of having to read the escaped string, parse it for every backslash and then mentally unescape it. This was there's no mental computation necessary, the value is right there for me to read.
…scaped characters. (elastic#137245) (cherry picked from commit 8b01dcf)
…scaped characters. (elastic#137245) (cherry picked from commit 8b01dcf)
💔 Some backports could not be created
Note: Successful backport PRs will be merged automatically after passing CI. Manual backportTo create the backport manually run: Questions ?Please refer to the Backport tool documentation |
…scaped characters. (#137245) (#137866) (cherry picked from commit 8b01dcf) Co-authored-by: CJ Cenizal <[email protected]>
…scaped characters. (#137245) (#137867) (cherry picked from commit 8b01dcf) Co-authored-by: CJ Cenizal <[email protected]>
Fixes #124027
Summary
The original issue reported this Grok pattern as the failure case:
%{IPORHOST:ip}%{SPACE}-%{SPACE}%{DATA:user_name}%{SPACE}\[%{HTTPDATE:timestamp}\]%{SPACE}\"%{WORD:http_method}%{SPACE}%{DATA:url}%{SPACE}HTTP/%{NUMBER:http_version}\"%{SPACE}%{NUMBER:response_code}%{SPACE}%{NUMBER:body_sent_bytes}%{SPACE}\"%{DATA:referrer}\"%{SPACE}\"%{DATA:agent}\"%{SPACE}\"%{IPORHOST:client_ip}\However, it contains a trailing backslash. I ran it through Grok Debugger, which rejected it, so I believe the trailing backslash is invalid syntax anyway.
I implemented the solution proposed by JL and added a test with a new case:
\[%{HTTPDATE:timestamp}\]%{SPACE}\"%{WORD:http_method}%{SPACE}HTTP/%{NUMBER:http_version}\"This is valid grok syntax, but the backslashes get rejected by the JSON string validator in the original code. Removing that validator solves the validation problem, and then it looks like no additional escaping or unescaping of slashes is necessary. Reviewers can manually test this fix by saving a Grok processor with the new pattern, above.
Release note
Ingest Pipelines will now accept valid grok syntax for Grok processor patterns, and will no longer incorrectly reject them with an "Invalid JSON string" error.