Agent doesn't uninstall Endpoint when it is uninstalled from the Agent's command line

[ferullo]

I was using 7.10.0 BC2 and I ran the following commands. They show that Endpoint was not uninstalled when Agent was uninstalled. Agent installed Endpoint because of I had an Endpoint Security integration enabled. I tested this on Linux (Ubuntu 18.04 x86_64), I didn't try other OSes.

I was manually able to uninstall Endpoint after uninstall Agentt so I don't think that Endpoint refused to uninstall when Agent tried.

vagrant@ubuntu:/tmp/elastic-agent-7.10.0-linux-x86_64$ ls /opt/
vagrant@ubuntu:/tmp/elastic-agent-7.10.0-linux-x86_64$ sudo ./elastic-agent install -f --kibana-url=https://<REDACTED>:443 --enrollment-token=<REDACTED>
The Elastic Agent is currently in BETA and should not be used in production

2020-10-15T16:39:47.624-0400	DEBUG	kibana/client.go:170	Request method: POST, path: /api/fleet/agents/enroll
Successfully enrolled the Elastic Agent.
Installation was successful and Elastic Agent is running.
vagrant@ubuntu:/tmp/elastic-agent-7.10.0-linux-x86_64$ ls /opt/Elastic/
Agent  Endpoint
vagrant@ubuntu:/tmp/elastic-agent-7.10.0-linux-x86_64$ sudo /usr/bin/elastic-agent uninstall
Elastic Agent will be uninstalled from your system at /opt/Elastic/Agent. Do you want to continue? [Y/n]:y

Elastic Agent has been uninstalled.
vagrant@ubuntu:/tmp/elastic-agent-7.10.0-linux-x86_64$ 
vagrant@ubuntu:/tmp/elastic-agent-7.10.0-linux-x86_64$ ls /opt/Elastic/
Endpoint
vagrant@ubuntu:/tmp/elastic-agent-7.10.0-linux-x86_64$ ls /opt/Elastic/Endpoint/
cache  elastic-endpoint  elastic-endpoint.yaml  state
vagrant@ubuntu:/tmp/elastic-agent-7.10.0-linux-x86_64$

[elasticmachine]

Pinging @elastic/ingest-management (Team:Ingest Management)

[botelastic]

This issue doesn't have a Team:<team> label.

[blakerouse]

I think we need to determine if an uninstall should also perform an unenroll and in that case should we also have an unenroll command. When you re-enroll an already enrolled agent does it perform unenroll on the previous Fleet connection?

The reason I mention those questions is because if we implement client-side unenroll. We can have the uninstall command just perform unenroll first. Then do the actual uninstall. Because unenroll will uninstall Endpoint and then uninstall of the agent will follow.

To handle unenroll/uninstall the uninstall command will send a GRPC call through the control socket to the running daemon to perform the unenroll/uninstallation of Endpoint. Then signal back to the calling process that its complete and then the process can finish the uninstallation.

[ph]

@blakerouse Can you lead that discussion it seems a mix of technical and pm @ruflin @mostlyjason.

[ruflin]

I like the idea of an "unenroll" command on the Agent.

[EricDavisX]

hi. we discussed in the team weekly meeting and I don't have any concerns now with this knowing the good news that it should work fine to uninstall Endpoint if a user does an ‘unenroll’ via the Fleet ui. The problem only shows when the 'uninstall' command is done via the Agent command line first.

[mostlyjason]

+1 on an unenroll command on the agent. otherwise, we leave a bunch of junk on the fleet page that users have to force unenroll. I think its lower priority because the UI works, but would still be nice to clean up on uninstall.

[EricDavisX]

FYI - I am seeing that if you uninstall from the Agent on the host first, and then uninstall the Endpoint, you don't get any feedback that it didn't work immediately. And the current state can block the user from doing subsequent re-installs of Agent (like if they want to start over for some reason). Dan mentions you can do this to uninstall Endpoint once in this state:

copy c:\Program Files\Elastic\Endpoint\elastic-endpoint.exe my-elastic-endpoint.exe
my-elastic-endpoint.exe uninstall
del my-elastic-endpoint.exe

I think the situation is a little messy, and is something we'll run into with more usage for sure. I'm going to bump it to 'high' priority. But even as i encounter it now, I think the design change / discussion is too impactful for 7.11 so I'm bumping to 7.12. I have pinged Dan / PH / Blake in chat.

[EricDavisX]

I am finishing up testing on this and can cite I was tripped up by the Agent 'upgrade' watcher process hanging around and being protected by Endpoint (new in 7.11+) and it all combined to leave me in a bad state. but it actually is likely not a common scenario - and after avoiding that (by waiting 10 minutes for upgrade 'watcher' to close out) then this work-around worked just fine. And Agent could be deleted / removed too, so it was not a problem. putting this back to the normal urgency we had on this.

It remains a feature we need to design and one that may not be prioritized high unless the work-around is bad, which isn't the case as I had thought it was

[ferullo]

can this be priorized for 7.13?

[ph]

@michalpristas Can add an e2e for this?

[EricDavisX]

looks like it is in PR for 7.13 / 8.0 - woo! we'll test it as soon as the 7.12 cycle relaxes. nice!
@dikshachauhan-qasource this will require some test suite update and testing across multiple OS types and artifact types. we can chat it more.

[michalpristas]

@EricDavisX was this verified as fixed and can this issue be closed

[dikshachauhan-qasource]

Hi @EricDavisX

We have validated this issue on 8.0 build and found fixed for Mac and Linux agents with Endpoint.

• Now endpoint gets removed when agent is uninstalled.

We are blocked to validate same on Windows due to defect elastic/kibana#93910.

Screenshot:

Thanks
QAS

[EricDavisX]

We have seen Windows working and testable, apart from the noted issue above - if you are seeing it we should follow through to see if we can resolve it sooner as opposed to later. We can add it to the urgent-issues list.

[amolnater-qasource]

Hi @EricDavisX
As per feedback have revalidated this on 7.14.0 BC-3 and had below observations:

All agents were installed with policy having System and Endpoint Security.
We are able to uninstall agents on mac and linux with: ./elastic-agent uninstall -f command from installation directories.

OS	Path	Command	Re-install
MAC	/Library/Elastic/Agent/*	./elastic-agent uninstall -f	Yes
Linux	/opt/Elastic/Agent/	./elastic-agent uninstall -f	Yes
Windows	C:\Program Files\Elastic\Agent	.\elastic-agent uninstall -f	Yes

Please let us know if anything else is required from our end.

Thanks
QAS

[noc101]

Hello... My FireEye license has expired. And I was not able to uninstall the agents from my dashboard. Now I cannot login the portal... I can do a manual uninstallation on the machines, but my problem is all of those are under on Production.... Is there a Major Impact if I just leave the agent installed on the nodes?.. I cant do a reboot after uninstallation of the agents...