Project dependencies that call getDataFileName inside Template Haskell splices return paths that don’t exist

[patrickt]

Describe the bug
This is a bit of a doozy. Credit to @rewinfrey for finding this out.

The https://github.com/github/semantic project depends on a package called https://github.com/tclem/lingo-haskell/, which provides autogenerated data structures that encapsulate the language information supported by https://github.com/github/linguist. This information is extracted from a YAML file of considerable length (just under 7,000 lines). Previously, we used a custom Setup.hs file to generate code with string concatenation; recently, because we wanted to support both cabal and bazel, we switched away from this rather ad-hoc approach, instead electing to read the file inside a Template Haskell splice and generate it appropriately, as per tclem/lingo-haskell#9. In cabal builds, we get the file’s location by invoking getDataFileName from the autogenerated Paths_ library; under bazel we provide the path directly via a preprocessor directive. We published this revision as lingo 0.4.0.0; it builds fine both on Hackage CI and in local development within the lingo-haskell repository.

However, any packages that depend on this new revision fail to build. The error reported looks like so:

Building library for lingo-0.4.0.0..
[1 of 3] Compiling Paths_lingo      ( dist/build/autogen/Paths_lingo.hs, dist/build/Paths_lingo.o, dist/build/Paths_lingo.dyn_o )
[2 of 3] Compiling Data.Languages.Templates ( src/Data/Languages/Templates.hs, dist/build/Data/Languages/Templates.o, dist/build/Data/Languages/Templates.dyn_o )
[3 of 3] Compiling Data.Languages   ( src/Data/Languages.hs, dist/build/Data/Languages.o, dist/build/Data/Languages.dyn_o )

src/Data/Languages.hs:1:1: error:
    Exception when trying to run compile-time code:
      InvalidYaml (Just (YamlException "Yaml file not found: /Users/patrickt/.cabal/store/ghc-8.10.2/lng-0.4.0.0-bea0d472/share/languages.yml"))
    Code: generateLanguageMap
  |
1 | {-# LANGUAGE TemplateHaskell #-}

The store directory in question does not contain a lng-0.4.0.0 directory, but getDataFilePath is reporting it as the correct path. For ease of reproducibility, I’ve attached test-lingo.zip, which is a bare project with a single lingo dependency.

To Reproduce
Steps to reproduce the behavior:

• download the file
• cd
• cabal v2-build

Expected behavior
Expected to build this dependency.

System information

• Darwin
• ghc 8.10.2
• cabal 3.2.0.0 (tested with 3.4.0.0-rc1 too)

Additional context
It’s possible that we may be doing something wrong w/r/t Hackage packaging of the lingo file; please let us know if this is the case. This is definitely an edge case, but if possible we’d like to keep these Template Haskell splices without having to generate .hs files ahead of time. We can work around this in client apps with some CPP directives, but it seems like a bug that cabal builds the lingo project itself correctly but not when it’s a dependency.

[patrickt]

See also tweag/rules_haskell#1337 for an analogous problem.

[phadej]

getDataFileName will never work in TH. The location in dependency builds points to the installation directory, but at compile time there is nothing installed yet.

If you want to read extra sources during compilation time lookup how file-embed does that. (If I understand right, that you want to read data at compile time, and generate code based on it).

[patrickt]

file-embed got me a little closer, though it too can’t find the YAML file when it’s embedded inside a dependency’s archive. I’ve worked around by embedding a copy of said YAML file in the projects. In the future, we're planning to just generate the code manually and embed it in the project.