[Security Solution] Incorrect count is displayed when export the prebuilt + custom rules

[ghost]

Describe the bug
Incorrect count is displayed when export the prebuilt + custom rules

Build info

VERSION : 8.4.0 BC2
Build: 55166
COMMIT: 9e9e0d6a685cbc2858a85a357f93dcb76259fdee

Preconditions

1. Kibana should be running
2. Prebuilt and custom rules should be exist

Steps to Reproduce

1. Navigate to security > Rules page
2. Select the custom rules and prebuilt rules
3. Click on bulk action and select the export
4. Observe that incorrect count is displayed for custom rule
5. How ever correct count is displayed when click on export button

Actual Result
Incorrect count is displayed when export the prebuilt + custom rules

Expected Result
Correct count should be displayed when export the prebuilt + custom rules

Screen-cast

prebuild.rule.mp4

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[banderror]

@vitaliidm In the environment access to which @deepikakeshav-qasource kindly shared with me, I was able to reproduce the bug. This is what I noticed.

The bulk action request body was:

{"action":"export","ids":["adf8c7a0-13ee-11ed-9617-b1cad46473e2","066e8050-13ef-11ed-9617-b1cad46473e2","20b90840-1480-11ed-9617-b1cad46473e2","513bbd80-148c-11ed-9617-b1cad46473e2","e2182400-170d-11ed-bf82-2f6ce8fb01fa","ae8e77f0-13ee-11ed-9617-b1cad46473e2","ae8e50e0-13ee-11ed-9617-b1cad46473e2","ae8ddbb0-13ee-11ed-9617-b1cad46473e2","ae8db4a0-13ee-11ed-9617-b1cad46473e2","ae8d6680-13ee-11ed-9617-b1cad46473e2","ae8d3f70-13ee-11ed-9617-b1cad46473e2","ae8cca40-13ee-11ed-9617-b1cad46473e2","ae8c06f0-13ee-11ed-9617-b1cad46473e2","ae8c5510-13ee-11ed-9617-b1cad46473e2","ae8b1c90-13ee-11ed-9617-b1cad46473e2","ae8bb8d0-13ee-11ed-9617-b1cad46473e2","ae8a8050-13ee-11ed-9617-b1cad46473e2","ae8b6ab0-13ee-11ed-9617-b1cad46473e2","ae8c7c20-13ee-11ed-9617-b1cad46473e2","ae8af580-13ee-11ed-9617-b1cad46473e2"]}

The response body was:

{"id":"20b90840-1480-11ed-9617-b1cad46473e2","updated_at":"2022-08-09T06:57:03.178Z","updated_by":"396189040","created_at":"2022-08-05T05:33:25.577Z","created_by":"396189040","name":"Endpoint Security [Duplicate]","tags":["Elastic","Endpoint Security"],"interval":"5s","enabled":true,"description":"Generates a detection alert each time an Elastic Endpoint Security alert is received. Enabling this rule allows you to immediately begin investigating your Endpoint alerts.","risk_score":47,"severity":"medium","license":"Elastic License v2","output_index":"","meta":{"from":"15m"},"rule_name_override":"message","timestamp_override":"event.ingested","author":["Elastic"],"false_positives":[],"from":"now-905s","rule_id":"50a40106-d9c9-4ce4-8fbd-1be81851741f","max_signals":10000,"risk_score_mapping":[{"field":"event.risk_score","value":"","operator":"equals"}],"severity_mapping":[{"severity":"low","field":"event.severity","value":"21","operator":"equals"},{"severity":"medium","field":"event.severity","value":"47","operator":"equals"},{"severity":"high","field":"event.severity","value":"73","operator":"equals"},{"severity":"critical","field":"event.severity","value":"99","operator":"equals"}],"threat":[],"to":"now","references":[],"version":6,"exceptions_list":[{"id":"endpoint_list","list_id":"endpoint_list","namespace_type":"agnostic","type":"endpoint"},{"id":"58075450-14a8-11ed-9617-b1cad46473e2","list_id":"bcd7f2f1-89b9-45a9-9217-c4a1fb7c75d2","type":"detection","namespace_type":"single"}],"immutable":false,"related_integrations":[],"required_fields":[],"setup":"","type":"query","language":"kuery","index":["logs-endpoint.alerts-*"],"query":"event.kind:alert and event.module:(endpoint and not endgame)\n","filters":[],"throttle":"no_actions","actions":[]}
{"id":"e2182400-170d-11ed-bf82-2f6ce8fb01fa","updated_at":"2022-08-09T07:00:40.727Z","updated_by":"elastic","created_at":"2022-08-08T11:33:12.198Z","created_by":"396189040","name":"custom","tags":[],"interval":"30s","enabled":true,"description":"test","risk_score":21,"severity":"low","license":"","output_index":"","meta":{"from":"30000h","kibana_siem_app_url":"https://bc2-975f77.kb.europe-west1.gcp.cloud.es.io:9243/app/security"},"timestamp_override":"mytimestamp","timestamp_override_fallback_disabled":false,"author":[],"false_positives":[],"from":"now-108000030s","rule_id":"c9822c76-716d-4f9d-989f-f6b85c7991c6","max_signals":100,"risk_score_mapping":[],"severity_mapping":[],"threat":[],"to":"now","references":[],"version":1,"exceptions_list":[],"immutable":false,"related_integrations":[],"required_fields":[],"setup":"","type":"query","language":"kuery","index":["index1"],"query":"host.name: *","filters":[],"throttle":"no_actions","actions":[]}
{"id":"066e8050-13ef-11ed-9617-b1cad46473e2","updated_at":"2022-08-09T07:00:41.033Z","updated_by":"elastic","created_at":"2022-08-04T12:14:44.713Z","created_by":"396189040","name":"custom rule","tags":[],"interval":"10s","enabled":true,"description":"test","risk_score":21,"severity":"low","license":"","output_index":"","meta":{"from":"1m","kibana_siem_app_url":"https://bc2-975f77.kb.europe-west1.gcp.cloud.es.io:9243/app/security"},"author":[],"false_positives":[],"from":"now-70s","rule_id":"a950246a-a0df-48b7-8b3b-8e2101a56940","max_signals":100,"risk_score_mapping":[],"severity_mapping":[],"threat":[],"to":"now","references":[],"version":10,"exceptions_list":[],"immutable":false,"related_integrations":[],"required_fields":[],"setup":"","type":"query","language":"kuery","index":["apm-*-transaction*","auditbeat-*","endgame-*","filebeat-*","logs-*","packetbeat-*","traces-apm*","winlogbeat-*","-*elastic-cloud-logs-*"],"query":"process.name: \"cmd.exe\"","filters":[],"throttle":"no_actions","actions":[]}
{"id":"513bbd80-148c-11ed-9617-b1cad46473e2","updated_at":"2022-08-09T05:38:31.272Z","updated_by":"396189040","created_at":"2022-08-05T07:00:41.453Z","created_by":"396189040","name":"Data view custom rule","tags":[],"interval":"30s","enabled":true,"description":"test","risk_score":21,"severity":"low","license":"","output_index":"","meta":{"from":"30000h","kibana_siem_app_url":"https://bc2-975f77.kb.europe-west1.gcp.cloud.es.io:9243/app/security"},"author":[],"false_positives":[],"from":"now-108000030s","rule_id":"c1333a3c-22c4-4651-879a-41927e44d377","max_signals":100,"risk_score_mapping":[],"severity_mapping":[],"threat":[],"to":"now","references":[],"version":2,"exceptions_list":[{"list_id":"450b56a4-c7fd-46c4-9b2e-fb36d733dc31","namespace_type":"single","id":"59c65320-148c-11ed-9617-b1cad46473e2","type":"detection"}],"immutable":false,"related_integrations":[],"required_fields":[],"setup":"","type":"query","language":"kuery","data_view_id":"3bfa11b7-9ea4-4970-accf-532b0adf22e3","query":"*","filters":[],"throttle":"no_actions","actions":[]}
{"_version":"WzMyNzc3MywxXQ==","created_at":"2022-08-05T10:21:18.229Z","created_by":"elastic","description":"Generates a detection alert each time an Elastic Endpoint Security alert is received. Enabling this rule allows you to immediately begin investigating your Endpoint alerts.","id":"58075450-14a8-11ed-9617-b1cad46473e2","immutable":false,"list_id":"bcd7f2f1-89b9-45a9-9217-c4a1fb7c75d2","name":"Endpoint Security [Duplicate]","namespace_type":"single","os_types":[],"tags":[],"tie_breaker_id":"1517a505-2990-48d4-85ff-c464660b7bf5","type":"detection","updated_at":"2022-08-05T10:21:18.234Z","updated_by":"elastic","version":1}
{"_version":"WzMyNzc3OSwxXQ==","comments":[],"created_at":"2022-08-05T10:23:04.826Z","created_by":"elastic","description":"Memory Threat Detection Alert: Multi.EICAR.Not-a-virus - exception list item","entries":[{"field":"source.ip","operator":"included","type":"exists"}],"id":"9790bda0-14a8-11ed-9617-b1cad46473e2","item_id":"4594dd60-8fd2-4688-9e04-5d9cd0714caf","list_id":"bcd7f2f1-89b9-45a9-9217-c4a1fb7c75d2","name":"Memory Threat Detection Alert: Multi.EICAR.Not-a-virus - exception list item","namespace_type":"single","os_types":[],"tags":[],"tie_breaker_id":"f78c007d-e6c4-40db-840b-1b8618f335ae","type":"simple","updated_at":"2022-08-05T10:23:04.829Z","updated_by":"elastic"}
{"_version":"WzMyNzc3OCwxXQ==","comments":[],"created_at":"2022-08-05T10:23:04.809Z","created_by":"elastic","description":"Memory Threat Detection Alert: Multi.EICAR.Not-a-virus - exception list item","entries":[{"field":"process.name","operator":"excluded","type":"match","value":"peazip.exe"}],"id":"978e2590-14a8-11ed-9617-b1cad46473e2","item_id":"2a805fdb-e544-45ba-ac07-76017cc7fb4c","list_id":"bcd7f2f1-89b9-45a9-9217-c4a1fb7c75d2","name":"Memory Threat Detection Alert: Multi.EICAR.Not-a-virus - exception list item","namespace_type":"single","os_types":[],"tags":[],"tie_breaker_id":"aa6374fd-02dc-4685-a0c9-a62625377a2d","type":"simple","updated_at":"2022-08-05T10:23:04.818Z","updated_by":"elastic"}
{"_version":"WzMyNzc3NywxXQ==","comments":[],"created_at":"2022-08-05T10:23:04.806Z","created_by":"elastic","description":"Memory Threat Detection Alert: Multi.EICAR.Not-a-virus - exception list item","entries":[{"field":"host.ip","operator":"excluded","type":"match_any","value":["10.0.5.125"]}],"id":"978db060-14a8-11ed-9617-b1cad46473e2","item_id":"faa70cdc-335a-4938-b389-b1bf7fda2413","list_id":"bcd7f2f1-89b9-45a9-9217-c4a1fb7c75d2","name":"Memory Threat Detection Alert: Multi.EICAR.Not-a-virus - exception list item","namespace_type":"single","os_types":[],"tags":[],"tie_breaker_id":"c158260b-ab07-4749-88ca-914a78e79959","type":"simple","updated_at":"2022-08-05T10:23:04.815Z","updated_by":"elastic"}
{"_version":"WzMyNzc3NiwxXQ==","comments":[],"created_at":"2022-08-05T10:23:04.804Z","created_by":"elastic","description":"Memory Threat Detection Alert: Multi.EICAR.Not-a-virus - exception list item","entries":[{"field":"destination.ip","operator":"excluded","type":"exists"}],"id":"978d6240-14a8-11ed-9617-b1cad46473e2","item_id":"c27d5178-6302-40b3-8fe4-a6fcf673874f","list_id":"bcd7f2f1-89b9-45a9-9217-c4a1fb7c75d2","name":"Memory Threat Detection Alert: Multi.EICAR.Not-a-virus - exception list item","namespace_type":"single","os_types":[],"tags":[],"tie_breaker_id":"fe2f27a7-d362-4dc9-9a82-7c51751ba5b5","type":"simple","updated_at":"2022-08-05T10:23:04.812Z","updated_by":"elastic"}
{"_version":"WzMyNzc3NSwxXQ==","comments":[],"created_at":"2022-08-05T10:23:04.803Z","created_by":"elastic","description":"Memory Threat Detection Alert: Multi.EICAR.Not-a-virus - exception list item","entries":[{"field":"file.path","operator":"included","type":"match_any","value":["C:\\Users\\zeus\\Desktop\\Files\\Malware\\Windows\\mimikatz.exe"]}],"id":"978d3b30-14a8-11ed-9617-b1cad46473e2","item_id":"09e8a290-a770-498a-9fed-f7315bc05cb9","list_id":"bcd7f2f1-89b9-45a9-9217-c4a1fb7c75d2","name":"Memory Threat Detection Alert: Multi.EICAR.Not-a-virus - exception list item","namespace_type":"single","os_types":[],"tags":[],"tie_breaker_id":"c35f1e58-c6d4-40db-8c92-31ba4ec53fb8","type":"simple","updated_at":"2022-08-05T10:23:04.807Z","updated_by":"elastic"}
{"_version":"WzMyNzc3NCwxXQ==","comments":[],"created_at":"2022-08-05T10:21:33.436Z","created_by":"elastic","description":"Memory Threat Detection Alert: Multi.EICAR.Not-a-virus - exception list item","entries":[{"field":"host.name","operator":"included","type":"match","value":"Deepika1-Win10x64"}],"id":"6117e2d0-14a8-11ed-9617-b1cad46473e2","item_id":"9ef76f78-b829-4ea8-8911-3df8fd6f1cf8","list_id":"bcd7f2f1-89b9-45a9-9217-c4a1fb7c75d2","name":"Memory Threat Detection Alert: Multi.EICAR.Not-a-virus - exception list item","namespace_type":"single","os_types":[],"tags":[],"tie_breaker_id":"448eb455-8df7-4efd-889b-10d29b6b4d8b","type":"simple","updated_at":"2022-08-05T10:21:33.441Z","updated_by":"elastic"}
{"_version":"WzMyNzczOCwxXQ==","created_at":"2022-08-05T07:00:55.250Z","created_by":"396189040","description":"test","id":"59c65320-148c-11ed-9617-b1cad46473e2","immutable":false,"list_id":"450b56a4-c7fd-46c4-9b2e-fb36d733dc31","name":"Data view custom rule","namespace_type":"single","os_types":[],"tags":[],"tie_breaker_id":"36866d08-50fb-4f6e-bf26-4e33519e3d63","type":"detection","updated_at":"2022-08-05T07:00:55.255Z","updated_by":"396189040","version":1}
{"exported_count":12,"exported_rules_count":4,"missing_rules":[{"rule_id":"e2f9fdf5-8076-45ad-9427-41e0e03dc9c2"},{"rule_id":"60884af6-f553-4a6c-af13-300047455491"},{"rule_id":"b83a7e96-2eb3-4edf-8346-427b6858d3bd"},{"rule_id":"11013227-0301-4a8c-b150-4db924484475"},{"rule_id":"ad88231f-e2ab-491c-8fc6-64746da26cfe"},{"rule_id":"9a1a2dae-0b5f-4c3d-8305-a268d404c306"},{"rule_id":"5bb4a95d-5a08-48eb-80db-4c3a63ec78a8"},{"rule_id":"5370d4cd-2bb3-4d71-abf5-1e1d0ff5a2de"},{"rule_id":"4ed493fc-d637-4a36-80ff-ac84937e5461"},{"rule_id":"c7ce36c0-32ff-4f9a-bfc2-dcb242bf99f9"},{"rule_id":"31295df3-277b-4c56-a1fb-84e31b4222a9"},{"rule_id":"96e90768-c3b7-4df6-b5d9-6237f8bc36a8"},{"rule_id":"4d50a94f-2844-43fa-8395-6afbd5e1c5ef"},{"rule_id":"f0b48bbc-549e-4bcf-8ee0-a7a72586c6a7"},{"rule_id":"c0429aa8-9974-42da-bfb6-53a0a515a145"},{"rule_id":"42bf698b-4738-445b-8231-c834ddefd8a0"}],"missing_rules_count":16,"exported_exception_list_count":2,"exported_exception_list_item_count":6,"missing_exception_list_item_count":0,"missing_exception_list_items":[],"missing_exception_lists":[],"missing_exception_lists_count":0}

See "exported_count":12,"exported_rules_count":4,?

The endpoint exported 12 objects, only 4 of which were rules, there were exception lists there as well.

We should fix and cover it with e2e tests. If possible, let's try to squeeze the fix into 8.4.0.

[MadameSheema]

Pending to be validated on BC5.

@deepikakeshav-qasource can you please validate this fix in latest 8.4 branch while BC5 is not available? Thanks!!

[vitaliidm]

fixed in #138598

[ghost]

Hi Team,

We have validated this issue on 8.4.0 Branch and observed that issue is Fixed.

Please find the below Testing Details:

Build info

Version:8.4.0 Branch
Commit:b52d44a7823999e96aad8e6496f410307d68e272

Screen-cast

export.mp4

We will going to retest this issue on BC5 so keeping the ticket open till then.

cc: @MadameSheema

Thanks!!

[ghost]

Hi @vitaliidm,

We have Validated this issue on 8.4.0 BC5 Build and Observed that issue is Fixed. 🟢

Please find the below Testing Details:

Build info

Version:8.4.0 BC5
BUILD: 55374
COMMIT: f12954223a8ad66bbbf77becc4f0557ffd1c92c3

Screen-cast

export.rules.mp4

Hence, We are closing this issue and marking as QA Validated!!

Thanks!!