foreign-library source files are not tracked in file-watch

[cblp]

General summary

If cabal file contains foreign-library section, its source files are not tracked with stack build --file-watch. Stack doesn't trigger rebuild when files change.

Steps to reproduce

foreign-library example
  build-depends:
      base
  hs-source-dirs: foreign
  other-modules: Example
  type: native-shared

stack build --file-watch

Expected

Rebuild when foreign/Example.hs is changed.

Actual

Nothing.

Stack version

$ stack --version
Version 1.6.5 x86_64 hpack-0.20.0

Method of installation

brew

[karulont]

Just to let people know, the issue is also present with:

$ stack --version
Version 1.9.1, Git revision f9d0042c141660e1d38f797e1d426be4a99b2a3c (6168 commits) x86_64 hpack-0.31.0

[dbaynard]

I've taken a quick look through the FileWatch.hs code. There have been a couple of issues, so it may be quite straightforward to fix.

[isovector]

I can confirm this in 2.1.1

[brandonchinn178]

Related, Stack doesn't seem to rebuild foreign-library components when source files change, even without --file-watch. i.e. stack build my-lib after making a code change doesn't rebuild. This is happening on Stack 2.9.3

Logs just say

2023-06-16 17:19:29.787612: [debug] Checking if we are going to build multiple executables with the same name
2023-06-16 17:19:29.787713: [debug] Executing the build plan

[mpilgrem]

Notes for self:

1. Stack.Build.Source.loadLocalPackage has nonLibComponents = toComponents exes tests benches, and

2.   componentFiles <- memoizeRefWith $
       fst <$> getPackageFilesForTargets pkg (ppCabalFP pp) nonLibComponents

   and

3. (for the lpComponentFiles field of the pure LocalPackage) lpComponentFiles = componentFiles.

4. Stack.Types.Package.lpFiles is:

   lpFiles :: HasEnvConfig env => LocalPackage -> RIO env (Set.Set (Path Abs File))
   lpFiles = runMemoizedWith . fmap (Set.unions . M.elems) . lpComponentFiles

5. Stack.Build.build has:

       for_ msetLocalFiles $ \setLocalFiles -> do
         files <-
           if boptsCLIWatchAll boptsCli
             then sequence [lpFiles lp | lp <- allLocals]

So, it looks to me that any component that is not exes tests or benches does not get included in the scope of the files that can be watched.

EDIT: But that can't be correct, because stack build --file-watch --watch-all lists watched files even if there is only a library stanza ...

EDIT: The answer is Stack.Build.Source.getPackageFilesForTargets:

-- | Gets list of Paths for files relevant to a set of components in a package.
-- Note that the library component, if any, is always automatically added to the
-- set of components.
getPackageFilesForTargets ::
     HasEnvConfig env
  => Package
  -> Path Abs File
  -> Set NamedComponent
  -> RIO env (Map NamedComponent (Set (Path Abs File)), [PackageWarning])
getPackageFilesForTargets pkg cabalFP nonLibComponents = do
  (components',compFiles,otherFiles,warnings) <-
    getPackageFiles (packageFiles pkg) cabalFP
  let necessaryComponents =
        Set.insert CLib $ Set.filter isCInternalLib (M.keysSet components')
      components = necessaryComponents `Set.union` nonLibComponents
      componentsFiles = M.map
        (\files ->
           Set.union otherFiles (Set.map dotCabalGetPath $ Set.fromList files)
        )
        $ M.filterWithKey (\component _ -> component `elem` components) compFiles
  pure (componentsFiles, warnings)

6. Working further back, there is Stack.Package.packageFromPackageDescription and pkgFiles = GetPackageFiles $ getPackageFile pkg, leading to Stack.PackageFile.getPackageFile and to Stack.PackageFile.packageDescModulesAndFiles ...

7. packageDescModulesAndFiles (-- | Get all files referenced by the package.) makes use of the following fields (only) of pkg :: PackageDescription: library, subLibraries, executables, testSuites, benchmarks, extraSrcFiles, dataFiles.

8. This is where support for foreign-libraries was first introduced: Support for Cabal 2.0 sub and foreign libraries #3430