Prevent extra-deps / local packages from overriding wired-in packages (only when ghc is a dep?)

[mgsloan]

Similarly to #1486, we need to treat GHC's wired-in packages specially when it comes to overriding the version in the global DB. Not checking for these cases leads to confusing error messages such as the ones described in #1522 and #1025. I think that this is only necessary when ghc is a transitive dependency, but I'm not sure about that, so part of the work here is to check this.

Since #1486 also deals with similar additional checking, it should probably get implemented along with this ticket. Note that the situation is a bit different for GHCJS, though. The two checks differ in the following ways:

1. GHC's wired-in packages are only special if we have a transitive dependency on GHC. GHCJS's boot packages are always special, because they are patched.

2. It's potentially acceptable to override a GHCJS boot package with a local package. There should probably be a warning letting the user know that it should be a patched version of the package, though.

Also, note that the checks for GHC's wired-in packages also apply when using GHCJS. Does GHCJS have extra wired-in packages? This is another thing that would be good to check.

[tolysz]

Just a note!
Packages used to boot GHCJS is not a constant list you can hard code! And this is simply wrong to do so. It is like saying aeson can not have any additional dependencies ever. The list needs to be queried from ghcjs-boot each time. we

• need a way that ghcjs-boot print this list

The long term is to re-boot ghcjs each time dependencies are updated... only a couple of packages is special.

• we need that ghcjs can use a custom location where to cache boot

if one overrides hardwired package we need to resolve all dependencies and build new boot.tar and boot.

[mchaver]

When you build something with GHCJS, there is already an assumption of what packages can be used for the project you can use because GHCJS was built with GHC and some version of text, bytestring, aeson, etc.. However, that dependency is not clear when you use stack. If you want to build a package with text-1.0 but GHCJS was built with text-0.11 stack won't build but it also won't notify you of what is going on. Right now it looks like this:

compiler: ghcjs-0.2.0.20160414_ghc-7.10.3
compiler-check: match-exact
setup-info:
  ghcjs:
    source:
      ghcjs-0.2.0.20160414_ghc-7.10.3:
        url: https://s3.amazonaws.com/ghcjs/ghcjs-0.2.0.20160414_ghc-7.10.3.tar.gz
        sha1: 6d6f307503be9e94e0c96ef1308c7cf224d06be3

I think it can be extended further to something like this:

compiler: ghcjs-0.2.0.20160414_ghc-7.10.3
compiler-check: match-exact
setup-info:
  ghcjs:
    source:
      ghcjs-0.2.0.20160414_ghc-7.10.3:
        url: https://s3.amazonaws.com/ghcjs/ghcjs-0.2.0.20160414_ghc-7.10.3.tar.gz
        sha1: 6d6f307503be9e94e0c96ef1308c7cf224d06be3
    deps:
      "bytestring-0.10"
      "aeson-0.11.0.0"

where deps must be a list of dependencies from ghcjs.cabal that are in valid ranges. For any non-declared dependency for building ghcjs, that can be derived from the resolver, packages and the packages in extra-deps. Now stack needs to make sure that ghcjs build dependencies, resolver, packages, and extra-deps packages do not conflict, inform user if they do.

Then you can create a hash key or some unique value to represent that combination of ghcjs dependencies and tag that to the end of the compiler name. Once you determine that ghcjs and the package can be built because of a match in packages, then you look up compiler name + package combination hash key and see if that is built or not. If it exist you don't need to rebuild it.

[tolysz]

The packages' sources are stored in libs/cache/boot.tar and which packages are used libs/etc/boot.yaml ... it can be extracted by machines

[mgsloan]

Yeah, I agree that it would better to extract the info from the tarball. @mchaver 's approach would work, but it would be a lot of manual labor.

[mchaver]

That sounds easier.

I was thinking of a more flexible solution but it pushes some optional manual labor to the stack.yaml. It would be nice to be able to point to one GHCJS tarball and swap out some boot dependencies at the stack.yaml level and anything not explicitly declared would use the dependency that comes with the tarball. That way I wouldn't have to manage multiple GHCJS tarballs for projects which have slightly different dependencies. However, that might just be a custom requirement, I can certainly come up with a way to do that with shake and stack for my own use cases.

[tolysz]

Aeson being part of ghcjs boot libraries has different dependencies across versions thus upgrading requires different wired in libraries