Make a way to build execs, tests, benches, without running them

[lukehoersten]

stack build --test should build the test, for example, without running the test. stack test should build the tests if not built and then run them.

We also need a stack build --exec or something to build the exec sections without running them.

[drwebb]

We already have some options to specify this behavior

$ stack test --help
...
 --no-run-tests           Disable running of tests. (Tests will still be
                           built.)
...

, and there is a similar one for bench

These should be able to be specified in a stack.yaml once the build opts are finished moving around. I've made some progress the last few days and am nearing completion with that.

[lukehoersten]

What about building exec sections?

Also, a shortcut (like stack test) for stack build --test --no-run-tests --bench --no-run-benchmarks plus the execs would be nice.

stack build for "lib + exe + test + bench" is what I'm looking for.

[drwebb]

stack builds exes by default, am I getting it wrong what you are asking?

If you need more fine grained control, there is some command line options that you can use at the moment:
http://docs.haskellstack.org/en/v1.0.2/build_command.html#target-syntax

So for example, stack build myproject1:exe:webserver myproject2:exe:http-client should build two target executables. The targets syntax is something that I'm leaving as command line only, but could change due to demand.

[mgsloan]

Yeah, it's true that stack build --test --no-run-tests --bench --no-run-benchmarks is pretty longwinded. How about stack build --all?

[silky]

yes, i like stack build --all as an alias for that former command.

[lukehoersten]

I could be misunderstanding the issue I'm having. Basically ghc-mod running on an exec section doesn't see modules from the library section until I run stack test/build. My assumption was that that was causing the exec sections to fully compile, which does indeed fix the ghc-mod issue.

[mgsloan]

Another option:

• Have --no-run be an alias for --no-run-tests and --no-run-benchmarks. Then it becomes stack build --test --bench --no-run.
• After that, I'm not sure if it's necessary, but --all could be an alias for --test and --bench, resulting in stack build --all --no-run. Is it worth adding an alias for this? Afterall, it's more to learn. Also, the aliases for --test and --bench have been commands sofar. Maybe something like stack all --no-run? Looks a little odd

[lukehoersten]

My expectation would be that the stack build command only built things (exec, lib, bench, test, etc) and running would be a separate command/option. It's almost a misnomer that, for example, stack build --test actually builds and runs the tests.

[mgsloan]

I can see that perspective, but I don't think that it's worth changing now. I think the way to read stack build --test is "build and test". Here's my reasoning on why the current default is better:

Tests and benchmarks aren't really very useful unless they get run (they don't get installed anywhere). So, this means that compiling them but not running them is only useful to ensure there aren't compile errors. The common case of building tests and benchmarks is when you also want to run them.

[lukehoersten]

I'd agree with that. Remember this all stems from the apparent failure to build exec sections and be seen by ghc-mod/flycheck-stack checker until I run stack bulid --test --no-run-tests. So I still think something is wrong with exec building. Sorry we derailed the discussion about CLI design a bit ;-)

[mgsloan]

Ahh, yeah, getting into flycheck and ghc-mod is a whole other business. Ideally, ghc-mod would know that it needs to do a build with these arguments, but I guess it doesn't know to do that yet. I've put down a note to look into fixing this, but not sure when it'll happen.

flycheck may be trickier. There's some in progress work here getting flycheck working better with stack: #1737

[lukehoersten]

I'm not sure if #1737 is the same issue. Mine is an issue of a cabal file with a library section and multiple executable sections. The executable section code doesn't recognize the librarysection code even though it's included as a build-dependency. I have to run stack test first and then it seems to work. And even then sometimes it doesn't. So apparently some artifacts are built with stack test that are not built with stack build for the executable sections.

[mgsloan]

Yes, when you run stack test, it gets configured with tests enabled. ghc-mod runs cabal-helper, which reads configuration info out of the dist directory. I believe ghc-mod has some code that runs stack build, but I don't think it knows to enable benchmarks and tests. It's hard to say in general whether tests and benchmarks should be enabled (they incur additional dependencies). I think it would be a very sensible default.

[lukehoersten]

The flycheck/ghc-mod issue is with executable sections of the cabal file, not the test section weirdly. Building the tests just fixes the issue. I'm sorry I know I'm doing a terrible job describing this issue. You're doing a great job of helping me nail down exactly my symptoms though. Thanks for your patience.

[mgsloan]

Oh, sorry! I read your prior comments too quickly. That is some strange behavior, I don't know what's going on there.

There are some ongoing discussions of ghc-mod + stack, hopefully things will work better after things are polished up a bit DanielG/ghc-mod#751

[DanielG]

@lukehoersten mind opening an issue on the ghc-mod issue tracker for that? We still have some really weird behavior lurking in the stack support. For example DanielG/ghc-mod#640 and DanielG/ghc-mod#699 are also about test suite weirdness.

[lukehoersten]

This is an issue with ghc-mod and flycheck (which uses stack directly). I actually use flycheck not ghc-mod but the issue is present with both.

[DanielG]

Ah, nevermind then.