[Ingest Manager] Allow to force unenroll from the UI #72386

nchaulet · 2020-07-20T00:41:46Z

Summary

Resolves elastic/beats#20006. Resolves elastic/beats#18825.

When an agent do not confirm the unenrollment, an user can invalidate all the ApiKeys for that user doing a force unenroll, that functionality was not exposed in the UI.

Done in this PR:

Expose the force unenroll feature for the agent in the unenrolling state.

UI Change

elasticmachine · 2020-07-20T00:41:56Z

Pinging @elastic/ingest-management (Team:Ingest Management)

nchaulet · 2020-07-20T00:42:18Z

@elasticmachine merge upstream

EricDavisX · 2020-07-20T21:32:35Z

This is excellent. I want to be sure I understand the changes, please. @ph
@nchaulet or @mostlyjason can you translate for me (and anyone who doesn't know) the impact of whats happening when the user elects to do the 'force unenroll'?

...Meaning can we use language not specific to Ingest Manager to make sure we're on the same page, please. I get confused still a little bit still about enrolling and api-key usage and how we use the terms.

@dedemorton when we've confirmed, It may be nice to have it in 7.9 docs if we didn't have a line item for it in the main 7.9 docs tracker ticket

nchaulet · 2020-07-21T13:02:48Z

@EricDavisX sure I am trying to explain what the "force unenroll" and the "unenroll" do.

Unenroll

When a user click on unenroll,

Fleet send a message to the agent, and we wait the agent to answer to confirm the unenrollment
The agent confirm (ack) the unenrollment
Fleet invalidate all the API keys related to the agent and change the status to inactive (disappear from the UI)

Force unenroll

When a user click on force unenroll:

Fleet invalidate all the API keys related to the agent and change the status to inactive (disappear from the UI). The agent cannot communicate with Fleet and send date to ES anymore.

mostlyjason · 2020-07-21T13:05:11Z

@nchaulet can you explain the need to distinguish between these two cases? Is it for Endpoint where we have to uninstall it? Would it eventually end up at same state the next time it checks in and discovers its API keys are invalid? I imagine this will be the case for offline agents as well?

If users would want to do a force unenroll every time, then it simplifies the UX to combine those steps.

nchaulet · 2020-07-21T13:09:05Z

Normally user will not have to do a force unenroll, it's to correct an error scenario where the agent is not able to confirm the enrollment.

The difference is with the normal unenrollment, the agent can finish to send data, correctly uninstall everything.

For the force unenroll it's more for agent that are not online anymore and an user that want to clean the API keys.

In an ideal scenario user should never had to do a force unenroll.

EricDavisX · 2020-07-21T13:54:07Z

This is helpful, I think I see where I was confused too. I got worried that unenrolling one agent would invalidate all keys for a given user overall or configuration entirely (not just for that agent). Thank you. And so, to another point - I think the reason we wanted to see the Force Unenroll was because standard unenroll was not working in common cases that worked prior. I think that bug and this feature support are different... @michalpristas are you still tracking the bug in elastic/beats#20006

Either way, I'm looking forward to getting this pr in. Thanks Nicolas!

mostlyjason · 2020-07-21T14:22:49Z

The difference is with the normal unenrollment, the agent can finish to send data, correctly uninstall everything.

Being able to finish sending the data makes sense, particularly once we have disk spooling. Is the purpose of the unenrolling state in Fleet clear out the queues?

Can we also confirm that after an Agent is force unenrolled that Endpoint gets uninstalled next time the Agent checks in?

Lastly, it'd be good to have design take a look at this. I'll reopen this issue to incorporate force unenroll into the confirmation modal design #72653

jen-huang

Small suggestions, otherwise code LGTM.

Thanks @mostlyjason for creating a design issue to improve the user messaging here.

...lugins/ingest_manager/public/applications/ingest_manager/components/context_menu_actions.tsx

…er/components/context_menu_actions.tsx Co-authored-by: Jen Huang <[email protected]>

kibanamachine · 2020-07-21T21:19:44Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: ca2e7fc

Build metrics

async chunks size

id	value	diff	baseline
ingestManager	1.1MB	+1.6KB	1.1MB

History

💚 Build #63382 succeeded 5a47dd3
💚 Build #62836 succeeded 22ebc7f

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

* master: (23 commits) Stabilize closing toast (elastic#72097) stabilize failing test (elastic#72086) Stabilize filter bar test (elastic#72032) Unskip vislib tests (elastic#71452) [ML] Fix layout of anomaly chart tooltip for long field values (elastic#72689) fix preAuth/preRouting mocks (elastic#72663) [Security Solution] Hide KQL bar (all pages) and alerts filters (Detections) when Resolver is full screen (elastic#72788) [Uptime] Rename Whitelist to Allowlist in parse_filter_map (elastic#71584) [Security Solution] Fixes exception modal not loading content (elastic#72770) [Security Solution][Exceptions] - Require non empty entries and non empty string values in exception list items (elastic#72748) [Detections] Add validation for Threshold value field (elastic#72611) [SIEM][Detection Engine][Lists] Adds version and immutability data structures (elastic#72730) [Security Solution][Detections] Validate file type of value lists (elastic#72746) [pre-req] New Component Layout proposal (elastic#72385) [ML] do not throw an error when agg is not supported by UI (elastic#72685) [Resolver] Origin process (elastic#72382) [Ingest Manager] Allow to force unenroll from the UI (elastic#72386) skip 6.8 branch when triggering baseline-capture builds (elastic#72706) [CI] In-progress PR comments (elastic#72211) Fix sorting of scripted string fields (elastic#72681) ...

EricDavisX · 2020-07-22T23:13:10Z

the 'Force unenroll' is in BC3 and seems to work - will need more time to review the nuances of the changes and Jason's questions. anyone available?

ph · 2020-07-23T15:19:46Z

Being able to finish sending the data makes sense, particularly once we have disk spooling. Is the purpose of the unenrolling state in Fleet clear out the queues?

We send a stop command to the process with a timeout, processes are expected to push they events to Elasticsearch and stop.

Can we also confirm that after an Agent is force unenrolled that Endpoint gets uninstalled next time the Agent checks in?

Yes, unenrolling will uninstall endpoint.

Lastly, it'd be good to have design take a look at this. I'll reopen this issue to incorporate force unenroll into the confirmation modal design #72653

ghost · 2020-07-27T10:18:11Z

Hi @EricDavisX ,

We have validated this ticket on 7.9 BC4 cloud environment with 7.9 agent integrated with security app as well on Windows 10 Host OS.

We have created 11 test cases under Un-enrolled agent section in Testrail and 02 test cases are moved from fleet to "Un-enrolled agent" section.

Moreover, we have executed 12 test cases under Allow to force an Unenroll from UI test run and 01 test case is pending due to query #1:

Please find below our observations and queries.

Observation:

We observed that agent successfully gets un-enrolled after clicking "Un-enroll" option and move to Inactive state. However, it took 1-2 minute for disappearing from agent list.

Screenshot1:

We observed that agent successfully gets un-enrolled on clicking "Force Unenroll" option and agent successfully gets disappear from agent list and move to Inactive state.

Screenshot 2:

Query:

We observed that when we "force un-enrolled" the agent from Fleet tab and then re-enrolled with default config that is already integrated with endpoint security, then agent successfully enrolled under Fleet tab, but when we navigate to endpoint security>Admin>Host tab, then still page to enable the endpoint security by enrolling the agent appears, is it expected or a bug?

Screenshot:

Additionally, we have reported below 02 defects while exploring the "Un-enrolled agent" functionality.

#73236
#73237

Please let us if anything else we need to verify for this ticket.

EricDavisX · 2020-07-27T13:20:31Z

I've asked @kamalpreetpahwa-qasource to log a separate bug for this last concern. we'll track it separately. thanks for the testing and test case work

ghost · 2020-07-28T06:04:40Z

@EricDavisX : I have reported Defect #73272 for the same.

Also, marked 01 test case as failed due to the reported issue #73272 under test pan Allow to force an Unenroll from UI

Please let us know if anything else need to verify.

[Ingest Manager] Allow to force unenroll from the UI

1b8500e

nchaulet requested a review from a team July 20, 2020 00:41

botelastic bot added the Team:Fleet Team label for Observability Data Collection Fleet team label Jul 20, 2020

nchaulet self-assigned this Jul 20, 2020

Merge branch 'master' into feature-ui-force-unenroll

22ebc7f

nchaulet added v7.10.0 v7.9.0 v8.0.0 release_note:skip Skip the PR/issue when compiling release notes labels Jul 20, 2020

ph mentioned this pull request Jul 20, 2020

[Ingest Manager] When I unenroll the Agent it doesn't disappear from UI (remains in unenrolling state) elastic/beats#20006

Closed

mdelapenya mentioned this pull request Jul 20, 2020

fix: use the status of the agent by hostname elastic/e2e-testing#181

Merged

This was referenced Jul 21, 2020

[Ingest Manager] Improve unenroll agent UX #67704

Closed

[Ingest Manager] Bulk unenroll and force unenroll #72653

Closed

nchaulet added the bug Fixes for quality problems that affect the customer experience label Jul 21, 2020

nchaulet requested a review from jen-huang July 21, 2020 18:17

jen-huang approved these changes Jul 21, 2020

View reviewed changes

...lugins/ingest_manager/public/applications/ingest_manager/components/context_menu_actions.tsx Outdated Show resolved Hide resolved

...lugins/ingest_manager/public/applications/ingest_manager/components/context_menu_actions.tsx Outdated Show resolved Hide resolved

Update x-pack/plugins/ingest_manager/public/applications/ingest_manag…

5a47dd3

…er/components/context_menu_actions.tsx Co-authored-by: Jen Huang <[email protected]>

hbharding mentioned this pull request Jul 21, 2020

[Ingest Manager] Add bulk actions #72712

Closed

Fix after review

ca2e7fc

nchaulet merged commit bb7d128 into elastic:master Jul 21, 2020

nchaulet deleted the feature-ui-force-unenroll branch July 21, 2020 21:34

nchaulet mentioned this pull request Jul 21, 2020

[7.x] [Ingest Manager] Allow to force unenroll from the UI (#72386) #72750

Merged

nchaulet added a commit to nchaulet/kibana that referenced this pull request Jul 21, 2020

[Ingest Manager] Allow to force unenroll from the UI (elastic#72386)

5e06f69

nchaulet mentioned this pull request Jul 21, 2020

[7.9] [Ingest Manager] Allow to force unenroll from the UI (#72386) #72751

Merged

nchaulet added a commit to nchaulet/kibana that referenced this pull request Jul 21, 2020

[Ingest Manager] Allow to force unenroll from the UI (elastic#72386)

bfaf453

nchaulet added a commit that referenced this pull request Jul 22, 2020

[Ingest Manager] Allow to force unenroll from the UI (#72386) (#72751)

0e7ad59

nchaulet added a commit that referenced this pull request Jul 22, 2020

[Ingest Manager] Allow to force unenroll from the UI (#72386) (#72750)

a5d9840

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Ingest Manager] Allow to force unenroll from the UI #72386

[Ingest Manager] Allow to force unenroll from the UI #72386

nchaulet commented Jul 20, 2020 •

edited by jen-huang

Loading

elasticmachine commented Jul 20, 2020

nchaulet commented Jul 20, 2020

EricDavisX commented Jul 20, 2020

nchaulet commented Jul 21, 2020

mostlyjason commented Jul 21, 2020 •

edited

Loading

nchaulet commented Jul 21, 2020

EricDavisX commented Jul 21, 2020

mostlyjason commented Jul 21, 2020 •

edited

Loading

jen-huang left a comment

kibanamachine commented Jul 21, 2020

EricDavisX commented Jul 22, 2020

ph commented Jul 23, 2020

ghost commented Jul 27, 2020 •

edited by ghost

Loading

EricDavisX commented Jul 27, 2020

ghost commented Jul 28, 2020 •

edited by ghost

Loading

[Ingest Manager] Allow to force unenroll from the UI #72386

[Ingest Manager] Allow to force unenroll from the UI #72386

Conversation

nchaulet commented Jul 20, 2020 • edited by jen-huang Loading

Summary

UI Change

elasticmachine commented Jul 20, 2020

nchaulet commented Jul 20, 2020

EricDavisX commented Jul 20, 2020

nchaulet commented Jul 21, 2020

Unenroll

Force unenroll

mostlyjason commented Jul 21, 2020 • edited Loading

nchaulet commented Jul 21, 2020

EricDavisX commented Jul 21, 2020

mostlyjason commented Jul 21, 2020 • edited Loading

jen-huang left a comment

Choose a reason for hiding this comment

kibanamachine commented Jul 21, 2020

💚 Build Succeeded

Build metrics

async chunks size

History

EricDavisX commented Jul 22, 2020

ph commented Jul 23, 2020

ghost commented Jul 27, 2020 • edited by ghost Loading

EricDavisX commented Jul 27, 2020

ghost commented Jul 28, 2020 • edited by ghost Loading

nchaulet commented Jul 20, 2020 •

edited by jen-huang

Loading

mostlyjason commented Jul 21, 2020 •

edited

Loading

mostlyjason commented Jul 21, 2020 •

edited

Loading

ghost commented Jul 27, 2020 •

edited by ghost

Loading

ghost commented Jul 28, 2020 •

edited by ghost

Loading