Architecture and requirements for this library

[chrisdone]

Here is the list of things that our Stackage library needs to be able to do:

• Download the package index. Options: use all-cabal-files (optionally with GPG verification), download 00-index from Hackage, use the new TUF stuff, etc. Currently there is a Hackage downloader reference implementation in Stackage.PackageIndex.
• Download an individual package source. Options: download from Hackage, download from S3. This should be careful to reject downloads that exceed the size specified in the all-cabal-files metadata about that package. There is currently a little reference implementation of this in Stackage.Fetch.
  • We should support verification of integrity via the all-cabal-files repo.
  • Optionally support verification of authenticity via the sig-archive archive.
• Determine the dependencies to install for a given set of package targets. Options: refuse to install dependencies at all, ask stackage.org (or the stackage-build-plan package) which packages to install based on an LTS or NIghtly release, use a dependency solver (Nathan's solution), ask cabal install --dry-run. There is currently an implementation of asking stackage.org in Stackage.Resolve.
• Determine the package databases to reference for packages, and the database to register new packages into. Options: global/user like normal, cabal sandboxes, something completely custom. We currently have some work done in this in stackage-cli, i.e. having snapshot-specific sandboxes. I think we can have that be the default, and when inside a Docker container, the snapshot location will come with the image, initally setup.
• Modifications to PATH/where to install binaries. Options: no mods and install in ~/.cabal/bin, use a cabal sandbox /bin/, something completely custom. In stackage-cli, packages are installed into the snapshot location under ~/.stackage/sandboxes/ghc-7.8.4/lts-2.1/bin, ../lib, etc.
• How to build packages. I don't think there's any reason right now to tweak this, as using runghc Setup ... seems like the right call. stackage-build has an implementation of this that has yet to be extracted.
• Where does the build occur. Options: the exact same place stackage-build is being run, inside a Docker container, theoretically in the future: a completely different machine that we rsync files to. In any case I think that the build process should be agnostic and whatever containment is happening should setup the environment and then run the build inside it.
• Reading and exposing package information after resolving flags properly (used by fpbuild, fpview, stackage-curator, etc.).
• Unpacking packages (and properly updating the cabal file). There's a basic reference implementation in Stackage.Fetch.
• Make sure that everything is logged, in a MonadLogger m => m, which in the usual case will probably be a runNoLoggingT but as a library can be configured to write somewhere, and when used as a program can, with -v, log to stdout.
• Ability to query the ghc-pkg database. Related to the other issue above. There are some limited wrapper functions for this in Stackage.GhcPkg.

These tasks can all be in stackage-common, but optionally can be in other packages. In any case they will come under the same module hierarchy. The fact that this library is used for commandline programs is a matter of incidence rather than design -- for example, fpbuild, fpview, stackage-curator will use these as well as stackage-sandbox and stackage-setup.

[chrisdone]

Moved all the particular modules into separate issues.

[chrisdone]

We can probably close this issue in favour of our more fine-grained tracking now.

[snoyberg]

Agreed