Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution][Detection Engine] Fixes critical date time format issues #79911

Merged
merged 2 commits into from
Oct 8, 2020

Conversation

FrankHassanabad
Copy link
Contributor

@FrankHassanabad FrankHassanabad commented Oct 7, 2020

Summary

Fixes #79865

Also fixes:

  • Timestamp override not being pushed down into threshold rules to use
  • Timestamp override not being used for lastValidDate
  • The return format of the date time might have been different depending on the customer mapping for both the override and the regular @timestamp so this fixes that as well.
  • Fixes one small type issue with fields.

Checklist

Delete any items that are not applicable to this PR.

@elasticmachine
Copy link
Contributor

Pinging @elastic/siem (Team:SIEM)

@FrankHassanabad FrankHassanabad added bug Fixes for quality problems that affect the customer experience v7.10.0 v7.11.0 v8.0.0 labels Oct 7, 2020
@FrankHassanabad FrankHassanabad changed the title [Security Solution][Detection Engine] Fixes date time format issues [Security Solution][Detection Engine] fixes critical date time format issues Oct 7, 2020
Copy link
Member

@spong spong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked out, tested locally with multiple test records (some with @timestamp, some with only specific time fields like event.ingested (in epoch as well)), and all cases outlined in the PR description appear to be functioning as intended.

In testing I did find a few issues, but not related to this PR, so LGTM! 👍 😉

Related issues:

  • Threshold rules are looking like they just generate an id for signal.parent.id and signal.parents[].id. We should verify this implementation, and consider not setting a parent id as there isn't a single event that the alert corresponds to (but rather a bucket of events). cc @marshallmain

@FrankHassanabad FrankHassanabad merged commit 7732a21 into elastic:master Oct 8, 2020
@FrankHassanabad FrankHassanabad deleted the add-time-format branch October 8, 2020 00:53
FrankHassanabad added a commit to FrankHassanabad/kibana that referenced this pull request Oct 8, 2020
…lastic#79911)

## Summary

Fixes elastic#79865

Also fixes:
* Timestamp override not being pushed down into threshold rules to use
* Timestamp override not being used for lastValidDate
* The return format of the date time might have been different depending on the customer mapping for both the override and the regular @timestamp so this fixes that as well.
* Fixes one small type issue with fields.


### Checklist

Delete any items that are not applicable to this PR.

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
FrankHassanabad added a commit to FrankHassanabad/kibana that referenced this pull request Oct 8, 2020
…lastic#79911)

## Summary

Fixes elastic#79865

Also fixes:
* Timestamp override not being pushed down into threshold rules to use
* Timestamp override not being used for lastValidDate
* The return format of the date time might have been different depending on the customer mapping for both the override and the regular @timestamp so this fixes that as well.
* Fixes one small type issue with fields.


### Checklist

Delete any items that are not applicable to this PR.

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
@FrankHassanabad FrankHassanabad added the Feature:Detection Rules Security Solution rules and Detection Engine label Oct 8, 2020
@FrankHassanabad FrankHassanabad changed the title [Security Solution][Detection Engine] fixes critical date time format issues [Security Solution][Detection Engine] Fixes critical date time format issues Oct 8, 2020
@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

FrankHassanabad added a commit that referenced this pull request Oct 8, 2020
…79911) (#79965)

## Summary

Fixes #79865

Also fixes:
* Timestamp override not being pushed down into threshold rules to use
* Timestamp override not being used for lastValidDate
* The return format of the date time might have been different depending on the customer mapping for both the override and the regular @timestamp so this fixes that as well.
* Fixes one small type issue with fields.


### Checklist

Delete any items that are not applicable to this PR.

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
FrankHassanabad added a commit that referenced this pull request Oct 8, 2020
…79911) (#79964)

## Summary

Fixes #79865

Also fixes:
* Timestamp override not being pushed down into threshold rules to use
* Timestamp override not being used for lastValidDate
* The return format of the date time might have been different depending on the customer mapping for both the override and the regular @timestamp so this fixes that as well.
* Fixes one small type issue with fields.


### Checklist

Delete any items that are not applicable to this PR.

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
gmmorris added a commit to gmmorris/kibana that referenced this pull request Oct 8, 2020
* master: (217 commits)
  Fix dashboard "snapshot share" is not sharing panel state in view mode (elastic#79837)
  fix can't edit a scripted field with special char (elastic#79842)
  [ML] clear selection action (elastic#79834)
  [TSVB] Show tooltip on external pointer events (elastic#77306)
  Fixes bug where the same index was being passed in (elastic#79949)
  Adds date time query and return fields for timestamps and overrides (elastic#79911)
  [Security Solution][Detections] Reverts rules table tag filter to use AND operator (elastic#79920)
  add the correct class to truncate the names (elastic#79921)
  [kbn/optimizer] report limits with ci metrics (elastic#78205)
  [release notes] extract "dev docs" comment too (elastic#79351)
  Revert "skips test failing promotion (elastic#79777)" (elastic#79904)
  share tslib across bundles (elastic#79915)
  remove entire suite as partial skips aren't doing the trick
  skip flaky suite (elastic#78689)
  Skip failing suite (elastic#79522)
  skip flaky suite (elastic#79910)
  [es/mappings] remove doc_values from text fields (elastic#79869)
  remove skipped snapshots
  skip flaky tests (elastic#79891)
  chore(NA): add missing branches into backportrc configuration file (elastic#79848)
  ...
@MindyRS MindyRS added the Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. label Sep 23, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Fixes for quality problems that affect the customer experience Feature:Detection Rules Security Solution rules and Detection Engine release_note:fix Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:SIEM v7.10.0 v7.11.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Security Solutions][Detection Engine] Throws date time format errors on custom mappings
5 participants