Detect extra-deps missing from indices

[luigy]

Stack used to silently accept extra-deps from config or cli that were missing from indices.
It now errors out when they are not found
(will add package suggestions for typos in another PR)

fixes #1521

[mgsloan]

Awesome, thanks!

Can these please get turned into a warning? Having it as an error like this is indeed more straightforward, however, having this as an error rules out the following use cases:

• Using an environment that does not have all of the required packages. For example, if you usually run with a docker container that has proprietary packages, but would still like to be able to build some packages outside of the container.
• If the stack.yaml is required to build tools / run scripts which populate the package DB with some more packages, satisfying the extra-deps.

This may be a bit tricky when they're also accompanied by build plan failures as in the example in #1521 . Ideally they'd come at the end of the output in that case, since it's probably the issue. I haven't looked at how the code would need to change to do that. Feel free to keep this as an exception constructor even if it's never thrown.

[luigy]

Oh, I see! yes, definitely this should be delayed till later.

This may be a bit tricky when they're also accompanied by build plan failures as in the example in #1521 . Ideally they'd come at the end of the output in that case, since it's probably the issue.

stack can have loadSourceMap return these as warnings and catch build plan failures in Stack.Build.build to display the messages in a better order. A change like this to loadSourceMap could also be used by a possible solution to #1752 #1722 that also depend, in a way, on loadSourceMap not throwing and holding off till wether the plan succeeds or not. Stack will have to make it clear that is doing this

Hmm should extra-deps coming through the cli still error out, though, to match stack's original behavior? and only warn on the ones coming from the config?

[luigy]

looks like something for #1745 could help with display order of warnings

[mgsloan]

Yep, I encourage you to work on #1745 if you want to! Particularly if it makes the implementation of this cleaner (it probably would).

I haven't thought about it thoroughly, but this could look like adding some IORef [Message] type to Env. runStackT would always output these messages (even on exception), so that they don't get lost.

There's no need to go and convert all the existing warnings and such to this system, that could be done incrementally. It might make sense to write something that just handles this particular usecase well, and generalize from there.

[mgsloan]

@luigy Hey! I'm doing a bit of a PR cleanup sweep, how's it going?

This change seems a bit related to #1839 . If it makes it easier, feel free to merge them into one PR (potentially a new PR if you like)

[mgsloan]

@luigy I pondered how to implement this given all the various concerns, and it was indeed quite tricky! Turns out we needed to plumb the extraDeps info out of loadSourceMap and then check it against the installed packages. Here's the commit: ced6ea8

Thanks for working on this!