Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution][Timeline Templates] KQLSyntaxError when Template Field is not present on Alert document #129958

Closed
spong opened this issue Apr 11, 2022 · 2 comments · Fixed by #140735
Assignees
Labels
8.3 candidate bug Fixes for quality problems that affect the customer experience Feature:Timeline Security Solution Timeline feature Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Threat Hunting:Investigations Security Solution Investigations Team Team:Threat Hunting Security Solution Threat Hunting Team

Comments

@spong
Copy link
Member

spong commented Apr 11, 2022

Summary

Observed in 8.2 when using the Investigate in Timeline action, if the configured Timeline Template has a Template Field that is not present in the Alert document, the Timeline will throw a KQLSyntaxError.

Steps to recreate

  1. Create Timeline Template with Template Field you know won't be in your source documents
  2. Create Rule and assign above Template
  3. Generate Alerts
  4. Verify Investigate in Timeline action show KQLSyntaxError as below

Note: May be helpful to use this sample Timeline Template, as it works out of the box with the node x-pack/plugins/security_solution/scripts/endpoint/resolver_generator.js data generation scripts.

Sample Template
{"savedObjectId":"c548bf70-b654-11ec-8a31-7ba9a7f2f52c","version":"WzE5MjE4NSwyXQ==","columns":[{"columnHeaderType":"not-filtered","id":"@timestamp","type":"number"},{"columnHeaderType":"not-filtered","id":"event.action"},{"aggregatable":true,"description":"Short name or login of the user.","columnHeaderType":"not-filtered","id":"source.user.name","category":"source","type":"string","example":"a.einstein"},{"columnHeaderType":"not-filtered","id":"source.ip"},{"aggregatable":true,"description":"Bytes sent from the source to the destination.","columnHeaderType":"not-filtered","id":"source.bytes","category":"source","type":"number"},{"columnHeaderType":"not-filtered","id":"destination.ip"},{"aggregatable":true,"description":"Port of the destination.","columnHeaderType":"not-filtered","id":"destination.port","category":"destination","type":"number"},{"aggregatable":true,"description":"Bytes sent from the destination to the source.","columnHeaderType":"not-filtered","id":"destination.bytes","category":"destination","type":"number"}],"dataProviders":[{"excluded":false,"and":[],"kqlQuery":"","name":"{source.ip}","queryMatch":{"field":"source.ip","value":"{source.ip}","operator":":"},"id":"timeline-1-1210648b-8456-4012-8fae-92597b54c9c7","type":"template","enabled":true}],"description":"View outbound blocked traffic through the Palo","eventType":"all","filters":[],"kqlMode":"filter","timelineType":"template","kqlQuery":{"filterQuery":{"serializedQuery":"{\"bool\":{\"filter\":[{\"bool\":{\"should\":[{\"match\":{\"event.module\":\"panw\"}}],\"minimum_should_match\":1}},{\"bool\":{\"should\":[{\"bool\":{\"should\":[{\"match\":{\"event.action\":\"flow_dropped\"}}],\"minimum_should_match\":1}},{\"bool\":{\"should\":[{\"match\":{\"event.action\":\"flow_denied\"}}],\"minimum_should_match\":1}}],\"minimum_should_match\":1}}]}}","kuery":{"expression":"event.module : panw and event.action : (flow_dropped or flow_denied) ","kind":"kuery"}}},"title":"Palo Outbound Blocked","sort":[{"columnType":"number","sortDirection":"desc","columnId":"@timestamp"}],"templateTimelineId":"2829c772-fe8e-470c-bac4-7b960ac797a1","templateTimelineVersion":1,"created":1649324049393,"createdBy":"soc","updated":1649424529242,"updatedBy":"soc","dateRange":{"start":"2022-04-06T09:10:18.488Z","end":"2022-04-07T09:10:18.488Z"},"indexNames":["filebeat-*"],"eqlOptions":{"tiebreakerField":"","size":100,"query":"","eventCategoryField":"event.category","timestampField":"@timestamp"},"favorite":[],"savedQueryId":null,"dataViewId":"filebeat-*","eventNotes":[],"globalNotes":[],"pinnedEventIds":[]}
Error

Data Provider disabled as expected

cc @paulewing @kqualters-elastic @andrew-goldstein

@spong spong added bug Fixes for quality problems that affect the customer experience triage_needed Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Feature:Timeline Security Solution Timeline feature Team:Threat Hunting:Investigations Security Solution Investigations Team labels Apr 11, 2022
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-threat-hunting (Team:Threat Hunting)

@logeekal logeekal self-assigned this Sep 12, 2022
logeekal added a commit that referenced this issue Sep 22, 2022
Fixes : #129958

When the data provider was disabled, the final query getting created is not syntactically correct and throws a syntax error as show in the screenshot below: 

![](https://user-images.githubusercontent.com/2946766/162839613-88320f35-ec0d-4df3-aa66-167593ef4955.png)
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Sep 22, 2022
…0735)

Fixes : elastic#129958

When the data provider was disabled, the final query getting created is not syntactically correct and throws a syntax error as show in the screenshot below:

![](https://user-images.githubusercontent.com/2946766/162839613-88320f35-ec0d-4df3-aa66-167593ef4955.png)

(cherry picked from commit ec1fe0a)
kibanamachine added a commit that referenced this issue Sep 22, 2022
…141424)

Fixes : #129958

When the data provider was disabled, the final query getting created is not syntactically correct and throws a syntax error as show in the screenshot below:

![](https://user-images.githubusercontent.com/2946766/162839613-88320f35-ec0d-4df3-aa66-167593ef4955.png)

(cherry picked from commit ec1fe0a)

Co-authored-by: Jatin Kathuria <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
8.3 candidate bug Fixes for quality problems that affect the customer experience Feature:Timeline Security Solution Timeline feature Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Threat Hunting:Investigations Security Solution Investigations Team Team:Threat Hunting Security Solution Threat Hunting Team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants