build-tools for executable in same package incorrect

[brandonchinn178]

hpack 0.34.4

Currently on this commit: https://github.com/brandonchinn178/tasty-autocollect/tree/1f45427cc4784c94679a14d0ceda72024491ae10

package.yaml contains

build-tools:
  - tasty-autocollect:tasty-autocollect

The cabal file contains

build-tools:
    tasty-autocollect

Doing a stack build from scratch shows the message:

No packages found in snapshot which provide a "tasty-autocollect" executable, which is a build-tool dependency of tasty-autocollect

But doing a manual

verbatim:
  build-tool-depends: tasty-autocollect:tasty-autocollect

then building from scratch (stack purge && stack build) no longer shows the message

[sol]

@brandonchinn178 sorry for getting back late. This is likely an issue with stack.

Can you reproduce the issue by running hpack && cabal build?

[mpilgrem]

@brandonchinn178, that's odd. When I try a test package with the following in its package.yaml:

build-tools:
- tasty-autocollect:tasty-autocollect

the Cabal file generated by Hpack 0.35.5 (I am using Stack 2.13.0.1) contains the following in its stanzas:

build-tool-depends:
      tasty-autocollect:tasty-autocollect

(EDIT: I get the same with Stack 2.11.1 and its Hpack 0.35.2)

[sol]

@mpilgrem the issue only occurs with build-tools from the same package. The whole build tools story is somewhat convoluted.

Hpack desugars build tools from the same package to build-tools instead of build-tool-depends as the resulting .cabal files work with Cabal 1.x. This is legal according to the Cabal spec and works with cabal. My suspicion is that stack somehow doesn't handle this correctly.

You could try to construct a reproducer for stack with a plain .cabal file that references a local executable as build-tool.

However, taking a step back, producing .cabal files that ever specify anything < 2 as cabal-version is probably not crucial anymore. So we could drop 1.x support, simplify Hpack and at the same time always produce something that works for stack. Before doing so I still think it's a good idea to pin down the exact nature of the bug, e.g. by constructing said reproducer.

[mpilgrem]

@sol, thanks for the explanation. I'll raise an issue on Stack's repository and see if I can work out what is happening Stack-wise.

[mpilgrem]

I am going to close this Hpack issue, as it is indeed a Stack issue.

[sol]

I'm reopening this for the time being. build-tools is deprecated and can't be used with cabal-version: 3.0. For that reason I'm inclined to desugar local build tools to build-tool-depends.

@mpilgrem you can still fix this in stack of course, but it won't be required for projects that use Hpack.

[mpilgrem]

@sol, build-tool-depends was introduced to Cabal files in Cabal 2.0. However, I've noticed that Hpack currently produces Cabal files with both (a) cabal-version: 1.12 and (b) build-tool-depends. I think that is an Hpack bug. EDIT: But Cabal tolerates the inconsistency , which I think is a Cabal bug.

Stack, currently, does not seek to drop support for Cabal < 2.0 before 2024-12-19, due to a 'seven year' approach to backwards compatibility. (The origin of 'seven years' was a discussion in the Slack channel; some people wanted a longer period than, say, the Cabal project's 'five years' - largely because, at that time, they were still using 'beloved' GHC 7.10.3, released 8 December 2015.)

If Hpack does drop Cabal's build-tools, I think that means Stack will need to accelerate that date, because Stack also seeks to incorporate the latest version of Hpack. If Stack accelerates the date to 'today', that reduces the backwards compatibility to a few months short of 'six years'. Perhaps that is tolerable, now that GHC 7.10.3 has been dropped.

[sol]

There is an other complication, Hackage does not support build-tool-depends yet, as it still uses v1-build. haskell/hackage-server#821

For that reason, I am not eager to change anything here.

EDIT: But Cabal tolerates the inconsistency , which I think is a Cabal bug.

Yes, exactly, I assume you already read the relevant parts of the cabal documentation, so I'm not reiterating them here. But the bottom line is that to produce cabal files that behave sane with cabal v1- and cabal v2- requires surgical precision. Hpack tries hard to produce output that is as compatible as possible. The rule set for build-tools is too complicated to fully internalize it. So, instead of requiring the user to go back and forth between the cabal documentation, their package, Hackage build logs, etc., Hpack tries to do the right thing.

This does not perfectly align with one of our design goals, "give the user 100% control when needed". A user who wants 100% control over when to use build-tools vs build-tool-depends has to resort to verbatim.

[sol]

@mpilgrem if you are still going to fix this in stack, I'm happy to do a review.

EDIT: Just realize that you already fixed it, great work 👍