Elastic Agent uses rename to move files, cannot move across filesystem boundaries

[pebrc]

In trying to work around #24160 we experimented with agent.download.install_path and agent.download.target_directory Specifically, running on Kubernetes, we tried to mount an emptyDir volume to use for both these locations as this would have allowed us to sidestep the noexec property of the main data path.

The problem with that approach seems that Agent uses a temporary directory in $path.home/data/tmp where it stages the programs to run to then move them to their final location (presumable with os.Rename ) which fails because source and target are not on the same filesystem

rename /var/lib/elastic-agent/data/tmp/elastic-agent-install414079180/metricbeat-7.11.1-linux-x86_64 /install/metricbeat-7.11.1-linux-x86_64: invalid cross-device link  

I think either this restriction needs to be documented/flagged up when mis-configured by the user or Elastic Agent should take the os.Open(src)/os.Create(tgt)/io.Copy(tgt, src)/os.Remove(src) approach instead of renaming.

[elasticmachine]

Pinging @elastic/agent (Team:Agent)