Dotfiles install script execution should be cached

[schickling]

Is your feature request related to a problem? Please describe

As mentioned in this issue executing the install.sh script of my dotfiles currently takes 2-3 minutes (mostly by running Nix home-manager). This means using Gitpod with my dotfiles enabled is basically not practical for my workflows (i.e. waiting 2-3 minutes every time I'm opening a workspace).

Describe the behaviour you'd like

One solution to this problem I can think of is to cache the result of the dotfiles install script.

Describe alternatives you've considered

Maybe some kind of layered file systems (e.g. via Docker images)?

Additional context

• Dotfiles install.sh script doesn't fully run #7579
• Related Discord chat

[stemount]

I've not used Nix, would it be more suitable in this scenario for you to fork the GitPod default workspace (or similar) as I am guessing it's not the dotfiles themselves but just that this environment is being built everytime you use GitPod.

[schickling]

I've not used Nix, would it be more suitable in this scenario for you to fork the GitPod default workspace (or similar) as I am guessing it's not the dotfiles themselves but just that this environment is being built everytime you use GitPod.

Would that approach still work for every repo/container I'm opening regardless of whether I have control over its .gitpod.yml?

[csweichel]

Caching the dotfile execution in such long-running cases would be sensible indeed. I'm wondering how this would work if it were based on a filesystem layer model: in filesystem layers files cannot really be changed, merely overwritten, we'd need to run the install script at least once for every workspace image - and when the dotfile repo changes. I'm not sure this would yield a good user experience, as users would still be waiting a lot for their dotfile scripts.

Part of what makes this wait so bad is that right now the dotfile install script has to complete before we start the IDE (it's likely the dotfiles modify $PATH and the likes). What if your dotfile install script did some of those install operations outside of its "hot path", i.e. did not block until they're complete?

[axonasif]

@csweichel: What if your dotfile install script did some of those install operations outside of its "hot path", i.e. did not block until they're complete?

Can we just spawn a subprocess for long running commands? (E.g sudo apt-get upgrade &)
Would the IDE launch process still be blocked and wait for the dotfiles install script to complete/exit?

[schickling]

I'm wondering how this would work if it were based on a filesystem layer model

Since I'm using nix for my system setup, I think this approach would actually work really well. cc @ghuntley

[axonasif]

@csweichel: we'd need to run the install script at least once for every workspace image - and when the dotfile repo changes. I'm not sure this would yield a good user experience, as users would still be waiting a lot for their dotfile scripts.

I can think of two approaches(more like init prebuilds):

• A dedicated script (I.e install-init.sh) that only runs once on change. A filesystem change snapshot may be taken for file modifications performed by this script.
• A gp cli command where you pass your one-time commands from your install.sh, then take the snapshots of the file changes and combine multiple snapshots together (Might be a rather complicated solution)
  It could be like: gp snap <identifier-name> <some-cli-command>.
  An example could be like: gp snap install-firefox sudo apt-get install firefox

What if your dotfile install script did some of those install operations outside of its "hot path", i.e. did not block until they're complete?

I went through experimenting with a bunch of hacks but either way, you have to wait to get started with your work. Whether you stay stuck in the IDE loading(Starting G) screen for a while or inject the shell changes in terminals after the installation is done, you have to wait either way.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[schickling]

Still relevant.

[david-bakin]

When implemented, the gitpod env vars including user defined env vars (incl per project env vars) should be part of the cache key. So you could pass an env var (either set in the gitpod settings or via URL) and have that affect the dotfile execution. See for example the discussion at https://discord.com/channels/816244985187008514/839379835662368768/995280980812255322

[loujaybee]

I wonder if the request to optimise dotfiles points to the absence of user-specific image configurations. Right now, Gitpod assumes that a team collaborates together on getting the right tools installed for a project, however in some cases users will want their own per-user tools or configurations for themselves (just as they do on their local machine). It might be worth exploring having a per-use base image setup where a user can define their own tools that's cached and incorporated with prebuilds, rather than being bundled in the dotfiles (which is the current workaround for per-user installations).

[axonasif]

I personally prefer the to install/configure things from my dotfiles, I use my dotfiles locally too and I would usually run my installation script for the first time I setup a machine. It's easier to manage one singe source of my dotfiles instead of having a detached Gitpod-specific configuration elsewhere.