Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Ingest Pipelines Grok processor to accept patterns that contain escaped characters #137245

Merged
merged 2 commits into from
Aug 2, 2022
Merged

Fix Ingest Pipelines Grok processor to accept patterns that contain escaped characters #137245

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f583c7a
Fix Ingest Pipelines Grok processor to accept patterns that contain e…
Jul 26, 2022
979d97e
Merge branch 'main' into bug/ingest-pipelines-grok-pattern-encoding
Aug 1, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions ..._pipelines/public/application/components/pipeline_editor/__jest__/processors/grok.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,4 +104,26 @@ describe('Processor: Grok', () => {
patterns: ['pattern1', 'pattern2', 'pattern3'],
});
});

test('accepts grok pattern that contains escaped characters', async () => {
const {
actions: { saveNewProcessor },
form,
} = testBed;

// Add "field" value
form.setInputValue('fieldNameField.input', 'test_grok_processor');

// Add the escaped value of \[%{HTTPDATE:timestamp}\]%{SPACE}\"%{WORD:http_method}%{SPACE}HTTP/%{NUMBER:http_version}\"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: seems a bit redundant to me to write the whole escaped string in this comment

Suggested change
// Add the escaped value of \[%{HTTPDATE:timestamp}\]%{SPACE}\"%{WORD:http_method}%{SPACE}HTTP/%{NUMBER:http_version}\"
// Add the escaped value into the input

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hear you. It's definitely repetitive. TBH I liked having it because it saved me the trouble of having to read the escaped string, parse it for every backslash and then mentally unescape it. This was there's no mental computation necessary, the value is right there for me to read.

const escapedValue =
'\\[%{HTTPDATE:timestamp}\\]%{SPACE}\\"%{WORD:http_method}%{SPACE}HTTP/%{NUMBER:http_version}\\"';
form.setInputValue('droppableList.input-0', escapedValue);

// Save the field
await saveNewProcessor();

const processors = getProcessorValue(onUpdate, GROK_TYPE);

expect(processors[0][GROK_TYPE].patterns).toEqual([escapedValue]);
});
});
1 change: 0 additions & 1 deletion ...ication/components/pipeline_editor/components/processor_form/processor_form.container.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,6 @@ export const ProcessorFormContainer: FunctionComponent<Props> = ({

if (isValid) {
const { type, fields: options } = data as FormData;

unsavedFormState.current = options;

onSubmit({
Expand Down
7 changes: 2 additions & 5 deletions ...blic/application/components/pipeline_editor/components/processor_form/processors/grok.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@
* 2.0.
*/

import { flow } from 'lodash';
import React, { FunctionComponent } from 'react';
import { i18n } from '@kbn/i18n';

Expand All @@ -23,7 +22,7 @@ import { XJsonEditor, DragAndDropTextList } from '../field_components';

import { FieldNameField } from './common_fields/field_name_field';
import { IgnoreMissingField } from './common_fields/ignore_missing_field';
import { FieldsConfig, to, from, EDITOR_PX_HEIGHT, isJSONStringValidator } from './shared';
import { FieldsConfig, to, from, EDITOR_PX_HEIGHT } from './shared';

const { isJsonField, emptyField } = fieldValidators;

Expand All @@ -49,7 +48,6 @@ const patternsValidation: ValidationFunc<any, string, ArrayItem[]> = ({ value, f

const patternValidations: Array<ValidationFunc<any, string, string>> = [
emptyField(valueRequiredMessage),
isJSONStringValidator,
];

const fieldsConfig: FieldsConfig = {
Expand All @@ -58,8 +56,7 @@ const fieldsConfig: FieldsConfig = {
label: i18n.translate('xpack.ingestPipelines.pipelineEditor.grokForm.patternsFieldLabel', {
defaultMessage: 'Patterns',
}),
deserializer: flow(String, to.escapeBackslashes),
serializer: from.unescapeBackslashes,
deserializer: String,
helpText: i18n.translate('xpack.ingestPipelines.pipelineEditor.grokForm.patternsHelpText', {
defaultMessage:
'Grok expressions used to match and extract named capture groups. Uses the first matching expression.',
Expand Down