Temporary Directory Issues Cause Installation Failure

[radeusgd]

General Summary

I have found several issues regarding handling of the temporary directory which cause the installation to sometimes fail.

Steps to Reproduce

1. Make sure that the temporary directory is cluttered:
   i. The clutter needs to be specific, for example create a directory graalvm-ce-java11-21.1.0 and put some files of the runtime here.
   ii. Due to other errors described below, you can just run enso install engine 0.2.12-SNAPSHOT.2021-06-15 followed by enso uninstall engine 0.2.12-SNAPSHOT.2021-06-15.
2. Run enso install engine 0.2.12-SNAPSHOT.2021-06-15 or open a project in IDE that will require that version.

Expected Result

The engine should be correctly installed.

Actual Result

The launcher fails with:

[error] [2021-06-15T10:50:33.278Z] [org.enso.launcher.cli.Main] A fatal error has occurred: java.io.IOException: Unable to delete directory /home/radeusgd/.local/share/enso/tmp.
java.io.IOException: Unable to delete directory /home/radeusgd/.local/share/enso/tmp.
  at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1127)
  at org.enso.runtimeversionmanager.FileSystem$.removeDirectory(FileSystem.scala:99)
  at org.enso.runtimeversionmanager.distribution.TemporaryDirectoryManager.$anonfun$tryCleaningTemporaryDirectory$1(TemporaryDirectoryManager.scala:57)
  at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
  at org.enso.runtimeversionmanager.locking.ResourceManager.tryWithExclusiveTemporaryDirectory(ResourceManager.scala:139)
  at org.enso.runtimeversionmanager.distribution.TemporaryDirectoryManager.tryCleaningTemporaryDirectory(TemporaryDirectoryManager.scala:51)
  at org.enso.launcher.cli.LauncherApplication$.$anonfun$installEngineCommand$2(LauncherApplication.scala:323)
  at org.enso.launcher.cli.LauncherApplication$.$anonfun$installEngineCommand$2$adapted(LauncherApplication.scala:322)
  at org.enso.cli.arguments.Application.$anonfun$run$1(Application.scala:99)
  at scala.util.Either.flatMap(Either.scala:352)
  at org.enso.cli.arguments.Application.run(Application.scala:86)
  at org.enso.cli.arguments.Application.run(Application.scala:64)
  at org.enso.launcher.cli.Main$.runAppHandlingParseErrors(Main.scala:18)
  at org.enso.launcher.cli.Main$.$anonfun$main$1(Main.scala:36)
  at org.enso.launcher.upgrade.LauncherUpgrader$.recoverUpgradeRequiredErrors(LauncherUpgrader.scala:416)
  at org.enso.launcher.cli.Main$.main(Main.scala:36)
  at org.enso.launcher.cli.Main.main(Main.scala)

and the Project Manager fails with:

[error] [2021-06-15T10:18:07.945Z] [org.enso.projectmanager.requesthandler.ProjectOpenHandler] Request ProjectOpen with Number(1) failed due to ProjectOpenFailed(Cannot install the required engine. java.nio.file.NoSuchFileException: /home/radeusgd/.local/share/enso/tmp/graalvm-ce-java11-21.1.0/bin/gu).

Enso Version

This happens on Launcher/PM from the latest (2021-06-15) nightly build.

Analysis

The above bug has uncovered a few (more-or-less related) issues with how temporary directory handling during installation/uninstallation:

1. It seems that uninstalling the latest runtime fails for some reason - the uninstallation is done in two steps to be 'as atomic as possible' - first the runtime is (hopefully atomically) moved to the tmp directory and only then the files are actually removed. The move succeeds but the deletion seems to fail on some files. It looks like it may be related to symlinks, but that has to be investigated.
2. The temporary directory is not cleared by the Project Manager (that is why it has a weird failure because the new installation sees the temporary directory polluted by the old installation and that unleashes ancient dragons of corrupted directory structure).
3. The temporary directory mechanism does not take into account that a directory with the same name may already exist in there - the tmp dir is cleared only opportunistically (and not at all by the Project Manager), so it is very possible that there is already a directory with the same name as used by a package that is currently being installed. This causes the corruption as described above.

Necessary solutions:

1. The clearing mechanism should be fixed to not fail in this scenario - there may be an issue with removing symlinks, this has to be investigated and the clear procedure improved so that it can handle it (rm is able to remove it even without -f so it should definitely be doable).
2. The Project Manager should also clean the temporary directory when possible. One place to add this could be in the function DistributionConfiguration.makeRuntimeVersionManager just before creating an instance of RuntimeVersionManager.
3. The install/uninstall processes should be resilient to unexpected files in the temporary directory. I'd suggest modifying slightly the logic for accessing the temporary directory and always creating a subdirectory with a randomly generated suffix (and ensuring that it does not yet exist), so that there are no collisions with leftover files.