[SIEM][Detection Engine] Import/Export REST endpoints

[FrankHassanabad]

Summary

• Adds Import and Export REST endpoints
• Fixes minor misc issues with types
• Changes camel case from bulk api to become snake_case

For the API and testing it is very similar to the saved objects API

For import:

POST /api/detection_engine/rules/_import

With a ndjson body of:

{"created_at":"2020-01-09T01:38:00.740Z","updated_at":"2020-01-09T01:38:00.740Z","created_by":"elastic_kibana","description":"Query with a rule_id that acts like an external id","enabled":true,"false_positives":[],"from":"now-6m","id":"6688f367-1aa2-4895-a5a8-b3701eecf57d","immutable":false,"interval":"5m","rule_id":"query-rule-id-1","language":"kuery","output_index":".siem-signals-frank-hassanabad-default","max_signals":100,"risk_score":1,"name":"Query with a rule id Number 1","query":"user.name: root or user.name: admin","references":[],"severity":"high","updated_by":"elastic_kibana","tags":[],"to":"now","type":"query","threats":[],"version":1}
{"created_at":"2020-01-09T01:38:00.745Z","updated_at":"2020-01-09T01:38:00.745Z","created_by":"elastic_kibana","description":"Query with a rule_id that acts like an external id","enabled":true,"false_positives":[],"from":"now-6m","id":"7a912444-6cfa-4c8f-83f4-2b26fb2a2ed9","immutable":false,"interval":"5m","rule_id":"query-rule-id-2","language":"kuery","output_index":".siem-signals-frank-hassanabad-default","max_signals":100,"risk_score":2,"name":"Query with a rule id Number 2","query":"user.name: root or user.name: admin","references":[],"severity":"low","updated_by":"elastic_kibana","tags":[],"to":"now","type":"query","threats":[],"version":1}
{"exported_count":2,"missing_rules":[],"missing_rules_count":0}

If you want to overwrite existing objects you can use the overwrite query parameter like so:

POST /api/detection_engine/rules/_import?overwrite=true

See and run the scripts of:

import_rules.sh
import_rules_no_overwrite.sh

For exporting everything:

POST /api/detection_engine/rules/_export

For exporting just a handful of things you would send a body like so:

POST /api/detection_engine/rules/_export
{
  "objects": [
    {
      "rule_id": "query-rule-id-1"
    },
    {
      "rule_id": "query-rule-id-2"
    }
  ]
}

To change either the filename of the file that gets downloaded or to remove the extra appended export details you can do the following:

POST /api/detection_engine/rules/_export?exclude_export_details=true&file_name=my_file.ndjson"

See the scripts of:

export_rules.sh
export_rules_by_rule_id.sh
export_rules_by_rule_id_to_file.sh
export_rules_to_file.sh

Checklist

Use strikethroughs to remove checklist items you don't feel are applicable to this PR.

- [ ] This was checked for cross-browser compatibility, including a check against IE11

- [ ] Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support

- [ ] Documentation was added for features that require explanation or tutorials

• Unit or functional tests were updated or added to match the most common scenarios

- [ ] This was checked for keyboard-only and screenreader accessibility

For maintainers

- [ ] This was checked for breaking API changes and was labeled appropriately

• This includes a feature addition or change that requires a release note and was labeled appropriately

[elasticmachine]

Pinging @elastic/siem (Team:SIEM)

[kibanamachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request
• Commit: 07ad87a

History

• 💔 Build #18933 failed 0e259a8

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[dhurley14]

This is neat. Didn't know Joi had this functionality.

[dhurley14]

Tested the scripts locally + code review. LGTM. Thanks for the new feature!