Stack should be more clear about packages vs. dependencies

[bitemyapp]

I realize it may be convenient, but including two different kinds of things under the header of "packages", where extra-dep: true flips the switch from one default to the other is possibly counter-productive from a UX point of view. I just helped another Stack user who is quite comfortable with Haskell and Cabal get unstuck because their stack ghci was broken. It was broken because they had intended to add local/git repos as additional dependencies, but didn't know they needed extra-dep: true to make them not be primary targets of the Stack build environment.

The documentation does touch on this, but doesn't exactly come right out and say it, and honestly, I think getting people to read the docs before diving in is going to be a bit hard as Stack is easy enough to use that they won't really ever need to pause until they run into a problem.

I think a more lasting solution is having separate top-level stanzas for build targets and dependencies coming from a filepath or git repository. The overlap of attributes for these is possibly galling, but you could console yourself with the idea that this is a sum-of-products representation :)

Another issue is the word packages itself. This is sort of a "bring your own preconceptions" word where to some it means "build target" and others it means "dependency you install". I think this is intentional in order to make it a catch-all, but doesn't help with the clarity issue here.

[mgsloan]

How about having extra-deps take the same format as packages? Except instead of taking some_directory, it would expect the package-version info currently used by extra-deps.

[bitemyapp]

@mgsloan I've spent a good 5 minutes puzzling over what you mean, could you write some strawman yaml to demonstrate what you have in mind?

[mgsloan]

@bitemyapp Sorry for the confusion, I was being overly terse. What I mean is that the extra-dep: true flag seems very ugly to me. extra-deps should be in the extra-deps list, local packages should be in the packages list.

Taking an example from https://github.com/commercialhaskell/stack/blob/master/doc/yaml_configuration.md#packages :

packages:
- local-package
- location: vendor/binary
  extra-dep: true
- location:
    git: [email protected]:yesodweb/wai
    commit: 2f8a8e1b771829f4a8a77c0111352ce45a14c30f
  subdirs:
  - auto-update
  - wai

Would instead be written

packages:
- local-package
- location:
    git: [email protected]:yesodweb/wai
    commit: 2f8a8e1b771829f4a8a77c0111352ce45a14c30f
  subdirs:
  - auto-update
  - wai
extra-deps:
- location: vendor/binary

This does not solve the issue of the user not understanding deps vs local packages, but I think it's much cleaner. What I was trying to say with my second sentence is that you will be able to use the same syntax in packages and extra-deps. However, extra-deps will need to have strings parsed as a package + version, whereas packages have string parsed as a directory.

[bitemyapp]

However, extra-deps will need to have strings parsed as a package + version, whereas packages have string parsed as a directory.

Perhaps trying to avoid syntactic overlap was part of the motivation for this? I'm planning to make a video tutorial for Slack with my coauthor soon, so perhaps at least that could help.

It seems like it might be more clear, but I'm not convinced the change would be worth the breakage. I'll stew on it. Please pipe up if you have any ideas.

[mgsloan]

To soften the breakage, we can certainly have a deprecation of a few major versions.

[bitemyapp]

It might seem petty, but perhaps projects rather than packages would make it more clear? I don't think most programmers are accustomed to thinking of packages as anything but dependencies.

projects:
- local-project
- location:
    git: [email protected]:yesodweb/wai
    commit: 2f8a8e1b771829f4a8a77c0111352ce45a14c30f
  subdirs:
  - auto-update
  - wai
dependencies:
- location: vendor/binary

[VictorTaelin]

I'm really sorry for interrupting, but as another confused user, what is the difference between those? I have no idea what this means:

but didn't know they needed extra-dep: true to make them not be primary targets of the Stack build environment.

[mgsloan]

@bitemyapp That does look nice! I'm not sure if a name change is worth it, though, maybe if the new names were absolutely perfect :) . These names have the following issues:

• "project" in stack land tends to refer to a multi-package repo with a stack.yaml
• extra-deps makes it clearer that this is not a full dependency list, just extra deps atop your resolver.

@MaiaVictor Hmm, yeah, the documentation could use more clarity on this point.

The difference is that packages are directly part of your project. All the commands that accept TARGETs are looking for names of local packages (build, test, bench, ide start, ghci, hpc report), and by default they will use all of the local packages.

It also affects how build options are applied. The default behavior of ghc-options recently changed to only apply to your local packages, but with the ability to also apply them to extra-deps / snapshot packages. Some aspects of this are still in flux: #849 .

It would definitely be good to write down all the distinctions which are based on whether a package is local / in the extra deps / in the snapshot. This might be informative for design as well, to ensure things behave as consistently as possible.

[borsboom]

Related: #1265 (Unify extra-deps and extensible snapshots)

[chreekat]

FWIW, I myself experience mental churn every time I hit the word "extra-deps". It does not mean what I initially think it means. Maybe it's because "extra" has the connotation of "inessential"? What about "other-deps" or "additional-deps" or "additional-dependencies"?

Mmm "other-deps" is just as bad, so strike that one. :)

I agree with @borsboom that "packages" is the right word: One cabal file for one package. Multiple packages per project. One stack.yaml per project. But I also agree that it's a rather fine distinction.

[mgsloan]

Closing this, as we now have a design for clearing this up in (3) of #1265, and the following discussion. I've also opened a documentation issue: #1501