Add XDG support

[drlkf]

• Any changes that could be relevant to users have been recorded in ChangeLog.md.
• The documentation has been updated, if necessary

This patch implements basic XDG support, as requested in #4243, with the
following logic:

• Environment variable override takes precedence as before.
• If the old-style ~/.stack directory exists, use it.
• Otherwise, put the global config.yaml in a subdirectory of
  XDG_CONFIG_HOME, and the rest in a subdirectory of XDG_DATA_HOME.

I tested this in a clean VM with no prior stack installed, with the following commands:

vagrant@bullseye:~$ tree -a
.
├── .bash_logout
├── .bashrc
├── .profile
└── .ssh
    └── authorized_keys

1 directory, 4 files
vagrant@bullseye:~$ /vagrant/build/stack new test
[...]
vagrant@bullseye:~$ tree -a
.
├── .bash_logout
├── .bashrc
├── .config
│   └── stack
│       └── config.yaml
├── .local
│   └── share
│       └── stack
│           ├── pantry
│           │   ├── global-hints-cache.yaml
│           │   ├── pantry.sqlite3
│           │   └── pantry.sqlite3.pantry-write-lock
│           ├── stack.sqlite3
│           ├── stack.sqlite3.pantry-write-lock
│           └── templates
│               └── new-template.hsfiles
├── .profile
├── .ssh
│   └── authorized_keys
└── test
    ├── app
    │   └── Main.hs
    ├── CHANGELOG.md
    ├── .gitignore
    ├── LICENSE
    ├── package.yaml
    ├── README.md
    ├── Setup.hs
    ├── src
    │   └── Lib.hs
    ├── stack.yaml
    ├── test
    │   └── Spec.hs
    └── test.cabal

12 directories, 22 files

Backwards-compatibility test:

vagrant@bullseye:~$ tree -a
.
├── .bash_logout
├── .bashrc
├── .profile
├── .ssh
│   └── authorized_keys
└── .stack
    ├── config.yaml
    ├── pantry
    │   ├── global-hints-cache.yaml
    │   ├── pantry.sqlite3
    │   └── pantry.sqlite3.pantry-write-lock
    ├── stack.sqlite3
    ├── stack.sqlite3.pantry-write-lock
    └── templates
        └── new-template.hsfiles

4 directories, 11 files
vagrant@bullseye:~$ /vagrant/build/stack new test
[...]
vagrant@bullseye:~$ tree -a
.
├── .bash_logout
├── .bashrc
├── .profile
├── .ssh
│   └── authorized_keys
├── .stack
│   ├── config.yaml
│   ├── pantry
│   │   ├── global-hints-cache.yaml
│   │   ├── pantry.sqlite3
│   │   └── pantry.sqlite3.pantry-write-lock
│   ├── stack.sqlite3
│   ├── stack.sqlite3.pantry-write-lock
│   └── templates
│       └── new-template.hsfiles
└── test
    ├── app
    │   └── Main.hs
    ├── CHANGELOG.md
    ├── .gitignore
    ├── LICENSE
    ├── package.yaml
    ├── README.md
    ├── Setup.hs
    ├── src
    │   └── Lib.hs
    ├── stack.yaml
    ├── test
    │   └── Spec.hs
    └── test.cabal

8 directories, 22 files

Although this is not much, I think it's indicative enough that stack is
reading/writing files in the correct locations. If the maintainers think written
tests need to be added, I am open to suggestions.

Some typo fixed while reading CONTRIBUTING.md too! :o)

A TODO comment asks if some Path.IO exception should be handled because I
don't know what is the general expected behavior of stack here.

There are several other problems and questions that I'm unable to answer alone:

• IMO this should be the default behavior, but I can't speak for the
  maintainers. For instance, it seems Travis caching will be broken by this
  change. I don't know your policy about this kind of questions.
• The documentation question depends on the above too, since the default
  behavior should be documented and, if XDG becomes default, I personally don't
  think legacy behavior should be cluttering the documentation.
• I have not tested this on Windows, nor do I know how to do it since I am
  unfamiliar with this OS. If you have a standard way, I'm open to suggestions.

[mpilgrem]

@drlkf, many thanks.

Am I correct to understand the following?

• this introduces the idea of a 'global config root', distinct from the rest of the traditional Stack root
• if the STACK_ROOT environment variable is defined, then that is respected as both the Stack root and the 'global config root'
• if the traditional Stack root already exists, that is respected as both the Stack root and the 'global config root'
• otherwise, if the XDG_DATA_HOME and XDG_CONFIG_HOME environment variables exist, the former is used as the base for Stack root and the latter as the base for the 'global config root'.
• otherwise, %APPDIR%\stack is used on Windows (as before); and ~/.local/share and ~/.config are used on Unix-like operating systems (a change in Stack's behaviour).

I think some extension will be needed to the stack path command, as stack path --stack-root will no longer help the user to know where the 'global config root' is located. Perhaps a stack path --global-config command? (I refer to 'global config' because the existing stack path --config-location refers to the project-level YAML configuration file.)

[hasufell]

My opinion is that this shouldn't be automagic, but the user should set an env var (like STACK_XDG=true). Otherwise this can lead to confusing behavior and bugs in CI.

[drlkf]

Mike Pilgrem ***@***.***> wrote:

Am I correct to understand the following? * this introduces the idea of a 'global config root', distinct from the rest of the traditional Stack root

This is correct, as discussed in #4243 XDG introduces separation of concerns that is not yet present in `stack`.

* if the `STACK_ROOT` environment variable is defined, then that is respected as both the Stack root and the 'global config root' * if the traditional Stack root already exists, that is respected as both the Stack root and the 'global config root' * otherwise, if the `XDG_DATA_HOME` and `XDG_CONFIG_HOME` environment variables exist, the former is used as the base for Stack root and the latter as the base for the 'global config root'.

Correct, XDG is pretty much the last option `stack` would resort to, when everything else has been ruled out.

* otherwise, if the `XDG_DATA_HOME` and `XDG_CONFIG_HOME` environment variables exist, the former is used as the base for Stack root and the latter as the base for the 'global config root'. * otherwise, `%APPDIR%\stack` is used on Windows (as before); and `~/.local/share` and `~/.config` are used on Unix-like operating systems (a change in Stack's behaviour).

Correct, if you meant `%APPDATA%\stack` for Windows.

I think some extension will be needed to the `stack path` command, as `stack path --stack-root` will no longer help the user to know where the 'global config root' is located. Perhaps a `stack path --global-config` command? (I refer to 'global config' because the existing `stack path --config-location` refers to the project-level YAML configuration file.)

I have added it to the best of my ability in 6054fd1, let me know if something is missing/wrong.

[mpilgrem]

@drlkf, many thanks for the explanations.

I think @hasufell is correct at #5919 (comment). A lot of CI will currently assume it has to cache ~/.stack on Linux and so some express 'opt-in' for a shift to ~/.local/share/stack and ~.config/stack might, at the least, smooth the introduction. The presence of the environment variable STACK_XDG seems like a way to do that. (GHCup uses GHCUP_USE_XDG_DIRS - https://www.haskell.org/ghcup/guide/#xdg-support, by way of another example.)

[drlkf]

Mike Pilgrem ***@***.***> wrote:

I think @hasufell is correct at #5919 (comment). A lot of CI will currently assume it has to cache `~/.stack` on Linux and so some express 'opt-in' for a shift to `~/.local/share/stack` and `~.config/stack` might, at the least, smooth the introduction. The presence of the environment variable `STACK_XDG` seems like a way to do that. (GHCup uses `GHCUP_USE_XDG_DIRS` - https://www.haskell.org/ghcup/guide/#xdg-support, by way of another example.)

Seems sensible to me, I've added this change, and the documentation and changelog entries. Could you tell me if those are ok ?

[mpilgrem]

Looks good to me! Thanks.

[galenhuntington]

I may be a bit late to the party, but it seems that XDG_DATA_HOME is the wrong place for nearly all of the non-config files. Most look like they qualify as cache and so belong in XDG_CACHE_HOME. This was the original proposal in #4243, putting everything in cache except for the config and templates.

This description of the XDG spec sums up the distinction:

If you would cry, running frenetically to your backup, it means that it belongs to XDG_DATA_HOME.

If you would just think « It’s bloody slow those days » then it is obviously part of XDG_CACHE_HOME.

Of course, I’m sure there is some corner cases. But take the time to choose carefully instead of just putting everything in XDG_DATA_HOME, defeating the purpose of this specification.

Deleting everything, excepting the config and templates, only means re-downloading a bunch of stuff on future Stack runs, which makes it cache.

On the other hand, global-project/stack.yaml looks like it belongs in XDG_CONFIG_HOME while (IIUC) this PR also puts it in XDG_DATA_HOME.

[mpilgrem]

@galenhuntington, this development was said to introduce 'basic' XDG support, and STACK_XDG means people chose to 'opt in' to that. If somebody wants to develop a 'more comprehensive' version of support, still on an 'opt in' basis, I don't see a barrier to that. As a practical matter, I suppose it would be good to do that before the 'basic' version gets widely adopted.

It is perhaps debatable whether the implicit global project (the project-level configuration file of 'last resort') is conceptually application 'configuration'. On the one hand it is common to all projects that do not have another project-level configuration file in their ancestry. On the other hand, it is a project-level configuration file, like any other.

As for 'non-essential' data files (cache), I suppose 'essential', like beauty, can be in the eye of the beholder. Yes, Stack can rebuild things if it needs to, but in some cases that would involve a lot of compiling - the 'cost' of losing the 'snapshot' data might be felt keenly by some.

[galenhuntington]

As for 'non-essential' data files (cache), I suppose 'essential', like beauty, can be in the eye of the beholder. Yes, Stack can rebuild things if it needs to, but in some cases that would involve a lot of compiling - the 'cost' of losing the 'snapshot' data might be felt keenly by some.

I'd say that's a central example of cache. Cached files are useful, sometimes very useful, to have around but can always be reconstituted. It's not like losing your PhD thesis or baby pictures. A criterion is what to back up. If you're archiving to Tarsnap, there's bandwidth and storage cost to keeping tens of gigs around for years, and if your drive crashes, it is not necessarily more convenient to restore from back-up than to rebuild with Stack. Indeed, in the case of Stack, I recompile often anyway, such as when GHC is bumped, and my .stack is brimming with old versions of stuff that will never be used again.

I agree it would be good to get the XDG layout right before it gets wide adoption. Sure, it's not always clear-cut what's config (vs. data or state), and despite using Stack for years I still feel I don't understand a lot of fine points.

If I had to spitball a backward-compatible solution, I'd suggest using XDG directories going forward, but for new installs, unless overridden by something like STACK_XDG, adding symlinks from files and directories under .stack/ to the appropriate XDG location. That's perhaps a big change and ambitious, and certainly I'm not in a position to implement it. The current arrangement with STACK_XDG may be adequate and will probably stay.