[Security Solution] Restores Alerts table local storage persistence and the Remove Column action

[andrew-goldstein]

[Security Solution] Restores Alerts table local storage persistence and the Remove Column action

This PR implements the following changes summarized below to address #113090, as proposed here:

• Configures the Columns popover to be consistent with Discover
• Changes the Hide column action to Remove column, to be consistent with Discover
• Persists updates to the Columns popover order in local storage
• Restores the feature to persist column widths in local storage

Configures the Columns popover to be consistent with Discover

• We now pass false to the allowHide EuiDataGrid API:

This makes all EuiDataGrid-based views in the Security Solution consistent with Discover's use of the EuiDataGrid Columns popover.

In 7.15, the Columns popover includes the hide column toggle, as shown in the screenshot below:

Above: The Columns popover in the 7.15 Alerts table

The Columns popover in Discover's EuiDataGrid-based table does not display the hide column toggle, as shown the screenshot below:

Above: The EuiDataGrid Columns popover in Discover, in master

Passing false to the allowHide EuiDataGrid API API makes the Columns popover in all EuiDataGrid-based views in the Security Solution consistent with Discover, as illustrated by the screenshot below:

Above: The Columns popover is now consistent with Discover

Changes the Hide column action to Remove column, to be consistent with Discover

• The Hide column action shown in the 7.15 alerts table is changed to Remove column, making it consistent with Discover's use of EuiDataGrid

In 7.15, the Alerts table has a Hide column action, as shown in the screenshot below:

Above: The Hide Column action in the 7.15 Alerts table

In 7.15, clicking the Hide Column action shown in the screenshot above hides the column, but does not remove it.

In 7.15, columns may only be removed by un-checking them in the Fields browser, or by un-toggling them in the Alerts / Events details popover. Both of those methods require multiple clicks, and require uses to re-find the field in the modal or popover before it may be toggled for removal.

In Discover, users don't hide columns.

In Discover, users directly remove columns by clicking the Remove column action, shown in the screenshot below:

Above: The Remove column action in Discover's use of EuiDataGrid in master

All EuiDataGrid-based views in the Security Solution were made consistent with Discover by replacing the Hide column action with Remove column, per the screenshot below:

Above: The Remove column action in the Alerts table

Note: the Remove column action shown above appears as the last item in the popover because it's specified via the EuiDataGrid EuiDataGridColumnActions > additonal API, which appends additonal actions to the end of popover, after the built-in actions:

Persists updates to the Columns popover order in local storage

• Persist column order updates to local storage when users update the order of columns via the Columns popover

The following PR #110685 restored partial support for persisting columns across page refreshes via local storage, but the Redux store was not updated when users sort columns via the Columns popover, an shown in the animated gif below:

Above: Ordering via the Columns popover is not persisted to local storage in 7.15

This PR utilizes the setVisibleColumns EuiDataGrid API API as a callback to update Redux when the columns are sorted, which will in-turn update local storage to persist the new order across page refreshes:

Restores the feature to persist column widths in local storage

In previous releases, resized column widths were peristed in local storage to persist across page refreshes, as documented in #110524 :

{
   "detections-page":{
      "id":"detections-page",
      "activeTab":"query",
      "prevActiveTab":"query",
      "columns":[
         {
            "category":"base",
            "columnHeaderType":"not-filtered",
            "description":"Date/time when the event originated. This is the date/time extracted from the event, typically representing when the event was generated by the source. If the event source has no original timestamp, this value is typically populated by the first time the event was received by the pipeline. Required field for all events.",
            "example":"2016-05-23T08:05:34.853Z",
            "id":"@timestamp",
            "type":"date",
            "aggregatable":true,
            "width":190
         },
         {
            "category":"cloud",
            "columnHeaderType":"not-filtered",
            "description":"The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier.",
            "example":"666777888999",
            "id":"cloud.account.id",
            "type":"string",
            "aggregatable":true,
            "width":180
         },
         {
            "category":"cloud",
            "columnHeaderType":"not-filtered",
            "description":"Availability zone in which this host is running.",
            "example":"us-east-1c",
            "id":"cloud.availability_zone",
            "type":"string",
            "aggregatable":true,
            "width":180
         },
         // ...
         }
      ],
      // ...
   }
}

Above: column widths were persisted to local storage in previous release, (going at least back to 7.12)

In this PR, we utilize the onColumnResize EuiDataGrid API API as a callback to update Redux when the columns are sorted via the Columns popover. Updating Redux will in-turn update local storage, so resized columns widths will persist across page refreshes:

Other changes

The Alerts page Trend chart and table were updated to include the following additional Stack by fields (CC @paulewing):

process.name
file.name
hash.sha256

per the before / after screenshots below:

Above: The Alerts Trend Stack by fields in 7.15 (before)

Above: The Alerts Trend Stack by fields (after the addition of the process.name, file.name, and hash.sha256 fields)

CC: @monina-n @paulewing

[monina-n]

all looks good to design 👍

[kqualters-elastic]

desk tested both in security solution and observability, with all changes working as described. lgtm 👍

[andrew-goldstein]

@elasticmachine merge upstream

[kibanamachine]

💛 Build succeeded, but was flaky

• continuous-integration/kibana-ci/pull-request
• Commit: 291894f
• Storybooks Preview
• Documentation Changes
• Flaky suites:
  • xpack-kibana-ciGroup8

---

Test Failures

Kibana Pipeline / general / Performance Tests.x-pack/test/performance/tests/reporting_dashboard·ts.performance reporting dashbaord downloaded PDF has OK status

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has failed 3 times on tracked branches: https://github.com/elastic/kibana/issues/110470

[00:00:00]     │
[00:00:00]       └-: performance
[00:00:00]         └-> "before all" hook in "performance"
[00:00:00]         └-: reporting dashbaord
[00:00:00]           └-> "before all" hook for "downloaded PDF has OK status"
[00:00:00]           └-> "before all" hook for "downloaded PDF has OK status"
[00:00:00]             │ debg resolved import for x-pack/test/performance/kbn_archives/reporting_dashboard to /dev/shm/workspace/parallel/6/kibana/x-pack/test/performance/kbn_archives/reporting_dashboard.json
[00:00:00]             │ info importing 4 saved objects { space: undefined }
[00:00:00]             │ info [o.e.c.m.MetadataMappingService] [node-01] [.kibana_8.0.0_001/PkHg8n_IS-2nlw8sC9AsvA] update_mapping [_doc]
[00:00:00]             │ info [o.e.c.m.MetadataMappingService] [node-01] [.kibana_8.0.0_001/PkHg8n_IS-2nlw8sC9AsvA] update_mapping [_doc]
[00:00:00]             │ info [o.e.c.m.MetadataMappingService] [node-01] [.kibana_8.0.0_001/PkHg8n_IS-2nlw8sC9AsvA] update_mapping [_doc]
[00:00:00]             │ info [o.e.c.m.MetadataMappingService] [node-01] [.kibana_8.0.0_001/PkHg8n_IS-2nlw8sC9AsvA] update_mapping [_doc]
[00:00:00]             │ info [o.e.c.m.MetadataMappingService] [node-01] [.kibana_8.0.0_001/PkHg8n_IS-2nlw8sC9AsvA] update_mapping [_doc]
[00:00:00]             │ succ import success
[00:00:00]             │ info [x-pack/test/performance/es_archives/reporting_dashboard] Loading "mappings.json"
[00:00:00]             │ info [x-pack/test/performance/es_archives/reporting_dashboard] Loading "data.json.gz"
[00:00:00]             │ info [o.e.c.m.MetadataCreateIndexService] [node-01] [foo] creating index, cause [api], templates [], shards [1]/[1]
[00:00:00]             │ info [x-pack/test/performance/es_archives/reporting_dashboard] Created index "foo"
[00:00:00]             │ debg [x-pack/test/performance/es_archives/reporting_dashboard] "foo" settings {"index":{"number_of_replicas":"1","number_of_shards":"1"}}
[00:00:02]             │ info [x-pack/test/performance/es_archives/reporting_dashboard] Indexed 10000 docs into "foo"
[00:00:02]           └-> downloaded PDF has OK status
[00:00:02]             └-> "before each" hook: global before each for "downloaded PDF has OK status"
[00:00:02]             │ debg navigating to dashboards url: http://localhost:6161/app/dashboards
[00:00:02]             │ debg navigate to: http://localhost:6161/app/dashboards
[00:00:03]             │ debg browser[INFO] http://localhost:6161/login?next=%2Fapp%2Fdashboards%3F_t%3D1634406162127 281 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:00:03]             │
[00:00:03]             │ debg browser[INFO] http://localhost:6161/bootstrap.js 41:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:00:03]             │ debg ... sleep(700) start
[00:00:03]             │ debg ... sleep(700) end
[00:00:03]             │ debg returned from get, calling refresh
[00:00:05]             │ debg browser[INFO] http://localhost:6161/login?next=%2Fapp%2Fdashboards%3F_t%3D1634406162127 281 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:00:05]             │
[00:00:05]             │ debg browser[INFO] http://localhost:6161/bootstrap.js 41:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:00:05]             │ debg currentUrl = http://localhost:6161/login?next=%2Fapp%2Fdashboards%3F_t%3D1634406162127
[00:00:05]             │          appUrl = http://localhost:6161/app/dashboards
[00:00:05]             │ debg TestSubjects.find(kibanaChrome)
[00:00:05]             │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"]') with timeout=60000
[00:00:06]             │ debg Found login page
[00:00:06]             │ debg TestSubjects.setValue(loginUsername, test_user)
[00:00:06]             │ debg TestSubjects.click(loginUsername)
[00:00:06]             │ debg Find.clickByCssSelector('[data-test-subj="loginUsername"]') with timeout=10000
[00:00:06]             │ debg Find.findByCssSelector('[data-test-subj="loginUsername"]') with timeout=10000
[00:00:06]             │ warn browser[SEVERE] http://localhost:6161/api/licensing/info - Failed to load resource: the server responded with a status of 401 (Unauthorized)
[00:00:06]             │ debg TestSubjects.setValue(loginPassword, changeme)
[00:00:06]             │ debg TestSubjects.click(loginPassword)
[00:00:06]             │ debg Find.clickByCssSelector('[data-test-subj="loginPassword"]') with timeout=10000
[00:00:06]             │ debg Find.findByCssSelector('[data-test-subj="loginPassword"]') with timeout=10000
[00:00:06]             │ debg TestSubjects.click(loginSubmit)
[00:00:06]             │ debg Find.clickByCssSelector('[data-test-subj="loginSubmit"]') with timeout=10000
[00:00:06]             │ debg Find.findByCssSelector('[data-test-subj="loginSubmit"]') with timeout=10000
[00:00:06]             │ debg Find.waitForDeletedByCssSelector('.kibanaWelcomeLogo') with timeout=10000
[00:00:06]             │ proc [kibana] [2021-10-16T17:42:45.938+00:00][INFO ][plugins.security.routes] Logging in with provider "basic" (basic)
[00:00:07]             │ debg browser[INFO] http://localhost:6161/app/dashboards?_t=1634406162127 281 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:00:07]             │
[00:00:07]             │ debg browser[INFO] http://localhost:6161/bootstrap.js 41:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:00:07]             │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"]') with timeout=60000
[00:00:09]             │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"] nav:not(.ng-hide)') with timeout=60000
[00:00:10]             │ debg browser[INFO] http://localhost:6161/app/dashboards?_t=1634406168424 281 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:00:10]             │
[00:00:10]             │ debg browser[INFO] http://localhost:6161/bootstrap.js 41:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:00:10]             │ debg Finished login process currentUrl = http://localhost:6161/app/dashboards
[00:00:10]             │ debg ... sleep(501) start
[00:00:11]             │ debg ... sleep(501) end
[00:00:11]             │ debg in navigateTo url = http://localhost:6161/app/dashboards
[00:00:11]             │ debg Waiting up to 20000ms for dashboard landing page...
[00:00:11]             │ debg onDashboardLandingPage
[00:00:11]             │ debg TestSubjects.exists(dashboardLandingPage)
[00:00:11]             │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="dashboardLandingPage"]') with timeout=5000
[00:00:11]             │ debg Load Saved Dashboard dashboard
[00:00:11]             │ debg gotoDashboardLandingPage
[00:00:11]             │ debg onDashboardLandingPage
[00:00:11]             │ debg TestSubjects.exists(dashboardLandingPage)
[00:00:11]             │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="dashboardLandingPage"]') with timeout=5000
[00:00:11]             │ debg searchForItemWithName: dashboard
[00:00:11]             │ debg TestSubjects.find(tableListSearchBox)
[00:00:11]             │ debg Find.findByCssSelector('[data-test-subj="tableListSearchBox"]') with timeout=10000
[00:00:12]             │ debg isGlobalLoadingIndicatorVisible
[00:00:12]             │ debg TestSubjects.exists(globalLoadingIndicator)
[00:00:12]             │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="globalLoadingIndicator"]') with timeout=1500
[00:00:12]             │ debg TestSubjects.exists(globalLoadingIndicator-hidden)
[00:00:12]             │ debg Find.existsByCssSelector('[data-test-subj="globalLoadingIndicator-hidden"]') with timeout=100000
[00:00:14]             │ debg TestSubjects.click(dashboardListingTitleLink-dashboard)
[00:00:14]             │ debg Find.clickByCssSelector('[data-test-subj="dashboardListingTitleLink-dashboard"]') with timeout=10000
[00:00:14]             │ debg Find.findByCssSelector('[data-test-subj="dashboardListingTitleLink-dashboard"]') with timeout=10000
[00:00:14]             │ debg isGlobalLoadingIndicatorVisible
[00:00:14]             │ debg TestSubjects.exists(globalLoadingIndicator)
[00:00:14]             │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="globalLoadingIndicator"]') with timeout=1500
[00:00:14]             │ debg browser[INFO] http://localhost:6161/app/dashboards#/view/37b49c50-2dc6-11eb-8af3-cb3aa84dbabd?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-15m,to:now)) 281 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:00:14]             │
[00:00:14]             │ debg browser[INFO] http://localhost:6161/bootstrap.js 41:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:00:16]             │ debg --- retry.tryForTime error: [data-test-subj="globalLoadingIndicator"] is not displayed
[00:00:16]             │ debg TestSubjects.exists(globalLoadingIndicator-hidden)
[00:00:16]             │ debg Find.existsByCssSelector('[data-test-subj="globalLoadingIndicator-hidden"]') with timeout=100000
[00:00:16]             │ debg TestSubjects.missingOrFail(dashboardLandingPage)
[00:00:16]             │ debg Find.waitForDeletedByCssSelector('[data-test-subj="dashboardLandingPage"]') with timeout=10000
[00:00:17]             │ debg openPdfReportingPanel
[00:00:17]             │ debg openShareMenuItem title:PDF Reports
[00:00:17]             │ debg TestSubjects.exists(shareContextMenu)
[00:00:17]             │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="shareContextMenu"]') with timeout=2500
[00:00:18]             │ info [o.e.c.m.MetadataCreateIndexService] [node-01] [.async-search] creating index, cause [auto(bulk api)], templates [], shards [1]/[0]
[00:00:20]             │ debg --- retry.tryForTime error: [data-test-subj="shareContextMenu"] is not displayed
[00:00:20]             │ info [o.e.c.m.MetadataMappingService] [node-01] [.kibana_8.0.0_001/PkHg8n_IS-2nlw8sC9AsvA] update_mapping [_doc]
[00:00:20]             │ debg TestSubjects.click(shareTopNavButton)
[00:00:20]             │ debg Find.clickByCssSelector('[data-test-subj="shareTopNavButton"]') with timeout=10000
[00:00:20]             │ debg Find.findByCssSelector('[data-test-subj="shareTopNavButton"]') with timeout=10000
[00:00:21]             │ debg Find.findByCssSelector('div.euiContextMenuPanel') with timeout=10000
[00:00:22]             │ debg TestSubjects.click(sharePanel-PDFReports)
[00:00:22]             │ debg Find.clickByCssSelector('[data-test-subj="sharePanel-PDFReports"]') with timeout=10000
[00:00:22]             │ debg Find.findByCssSelector('[data-test-subj="sharePanel-PDFReports"]') with timeout=10000
[00:00:23]             │ debg Find.waitForElementStale with timeout=10000
[00:00:23]             │ debg TestSubjects.click(generateReportButton)
[00:00:23]             │ debg Find.clickByCssSelector('[data-test-subj="generateReportButton"]') with timeout=10000
[00:00:23]             │ debg Find.findByCssSelector('[data-test-subj="generateReportButton"]') with timeout=10000
[00:00:24]             │ info [o.e.c.m.MetadataMappingService] [node-01] [.kibana_8.0.0_001/PkHg8n_IS-2nlw8sC9AsvA] update_mapping [_doc]
[00:00:24]             │ debg getReportURL
[00:00:24]             │ debg TestSubjects.getAttribute(downloadCompletedReportButton, href, tryTimeout=120000, findTimeout=60000)
[00:00:24]             │ debg TestSubjects.find(downloadCompletedReportButton)
[00:00:24]             │ debg Find.findByCssSelector('[data-test-subj="downloadCompletedReportButton"]') with timeout=60000
[00:01:25]             │ debg --- retry.tryForTime error: Waiting for element to be located By(css selector, [data-test-subj="downloadCompletedReportButton"])
[00:01:25]             │      Wait timed out after 61222ms
[00:01:26]             │ debg TestSubjects.find(downloadCompletedReportButton)
[00:01:26]             │ debg Find.findByCssSelector('[data-test-subj="downloadCompletedReportButton"]') with timeout=60000
[00:02:27]             │ debg --- retry.tryForTime error: Waiting for element to be located By(css selector, [data-test-subj="downloadCompletedReportButton"])
[00:02:27]             │      Wait timed out after 61201ms
[00:02:28]             │ debg Find.findByCssSelector('[data-test-errorText]') with timeout=10000
[00:02:38]             │ info Taking screenshot "/dev/shm/workspace/parallel/6/kibana/x-pack/test/functional/screenshots/failure/performance reporting dashbaord downloaded PDF has OK status.png"
[00:02:38]             │ info Current URL is: http://localhost:6161/app/dashboards#/view/37b49c50-2dc6-11eb-8af3-cb3aa84dbabd?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-15m,to:now))
[00:02:38]             │ info Saving page source to: /dev/shm/workspace/parallel/6/kibana/x-pack/test/performance/failure_debug/html/performance reporting dashbaord downloaded PDF has OK status.html
[00:02:38]             └- ✖ fail: performance reporting dashbaord downloaded PDF has OK status
[00:02:38]             │      TimeoutError: Waiting for element to be located By(css selector, [data-test-errorText])
[00:02:38]             │ Wait timed out after 10055ms
[00:02:38]             │       at /dev/shm/workspace/parallel/6/kibana/node_modules/selenium-webdriver/lib/webdriver.js:842:17
[00:02:38]             │       at runMicrotasks (<anonymous>)
[00:02:38]             │       at processTicksAndRejections (internal/process/task_queues.js:95:5)
[00:02:38]             │ 
[00:02:38]             │ 

Stack Trace

TimeoutError: Waiting for element to be located By(css selector, [data-test-errorText])
Wait timed out after 10055ms
    at /dev/shm/workspace/parallel/6/kibana/node_modules/selenium-webdriver/lib/webdriver.js:842:17
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:95:5) {
  remoteStacktrace: ''
}

---

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
timelines	329	330	+1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
securitySolution	4.6MB	4.6MB	+171.0B
timelines	240.3KB	241.4KB	+1.1KB
total			+1.3KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
securitySolution	104.0KB	104.1KB	+144.0B
timelines	156.6KB	157.4KB	+773.0B
total			+917.0B

History

• 💔 Build #161678 failed a1429b1
• 💔 Build #161270 failed c7c467398393dc82a59a3083de63b986c73d5865
• 💛 Build #160279 was flaky 5b2f83d140083d43949f021482663e323b7d5c9b

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @andrew-goldstein

[kibanamachine]

💚 Backport successful

Status	Branch	Result
✅	7.x

This backport PR will be merged automatically after passing CI.

[elasticmachine]

Pinging @elastic/security-threat-hunting (Team:Threat Hunting)