Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stack ghci/repl with internal sublibraries only works after a build #4148

Open
mihaimaruseac opened this issue Jul 11, 2018 · 25 comments
Open

Comments

@mihaimaruseac
Copy link
Contributor

General summary/comments (optional)

When running stack ghci on a fresh clone of a project having internal libraries, the attempt fails with

<command line>: cannot satisfy -package z-files-z-lib
    (use -v for more information)

However, if one first stack builds then the subsequent stack ghci succeeds.

Found while working on #4142 and will probably work on it in the coming weeks.

Steps to reproduce

  1. Go to the test code in test/integration/tests/3926-ghci-with-sublibraries/files
  2. Run rm -rf .stack-work to start from a clean state
  3. Run stack ghci and see that it fails
  4. Run step 2 again
  5. Run stack build and then stack ghci and it succeeds

Expected

Both step 3 and 5 should succeed.

Actual

Only step 3 succeeded

Stack version

[files] λ stack --version
Version 1.8.0, Git revision bcbfb1e90d1470ed6b9748b79e1655c52fc65b88 (6034 commits) x86_64 hpack-0.28.2

Method of installation

Git clone

@dbaynard
Copy link
Contributor

@mihaimaruseac it would be good to get this working or add a note to the docs, before the next release.

@mihaimaruseac
Copy link
Contributor Author

I'm travelling at the moment, so won't be able to get it working, most likely. Will come back to this on Friday and do one or the other

@dbaynard
Copy link
Contributor

Related: #2790 (possibly a duplicate)

mihaimaruseac added a commit that referenced this issue Jul 28, 2018
This documents the #2790 and #4148 bugs as we decided in #4177 to only
document this behavior for 1.8.1 release and solve them later.
mihaimaruseac added a commit that referenced this issue Jul 28, 2018
This documents the #2790 and #4148 bugs as we decided in #4177 to only
document this behavior for 1.8.1 release and solve them later.
@mihaimaruseac
Copy link
Contributor Author

Added documentation for this, will try solving it later when I get some time to dwell deep into the code.

@adamschoenemann
Copy link

I have this problem on 1.9.1 also (just fyi)

@dbaynard
Copy link
Contributor

It hasn't been fixed — just documented, for now.

@adamschoenemann
Copy link

Yeah, just wanted to bump this issue. Any idea why it's happening? I'd love to help but I'd need some pointers to have a chance (aside: why are there 'z's interspersed with the internal library name?).

@mihaimaruseac
Copy link
Contributor Author

I'm sorry, didn't get a chance to look into this :( Probably I'll get a chance next week or the one after, after HCAR is released.

The z's are just a name mangling scheme.

@TD5
Copy link

TD5 commented Jan 7, 2019

I get this issue with stack ghci even after nuking .stack-work and then kicking off a stack build.

λ stack --version
Version 1.9.1, Git revision f9d0042c141660e1d38f797e1d426be4a99b2a3c (6168 commits) x86_64 hpack-0.31.0

@mihaimaruseac
Copy link
Contributor Author

Seems there has been a regression and now #4564 has the up to date status of this issue

cimbul added a commit to cimbul/hasklog that referenced this issue Jun 1, 2019
Using an internal library led to an error with "stack ghci" that also affected by IDE. It seems to be related to this stack issue, but wasn't fixed by running "stack build" first:

commercialhaskell/stack#4148
@austinvhuang
Copy link

Is anyone else still running into this issue? I think I'm running into something related with stack Version 2.1.3, Git revision 636e3a759d51127df2b62f90772def126cdf6d1f (7735 commits) x86_64 hpack-0.31.2 LTS 14.7. May open up a separate issue.

@ProofOfKeags
Copy link

@austinvhuang I just ran into this today on a stack version 4 commits after what you have.

@aryairani
Copy link

aryairani commented Apr 17, 2020

@austinvhuang I'm running into this today too; Stack Version 2.1.3 x86_64. Glad I was able to find this ticket, because I thought I was going to have to choose between ghcid and internal libraries; didn't know about the "successful stack build" workaround.

@istathar
Copy link
Contributor

istathar commented Nov 8, 2020

I tried a stack build before stack ghci but am still running into the problem:

<command line>: cannot satisfy -package z-technique-z-technique-internal
    (use -v for more information)

@mihaimaruseac
Copy link
Contributor Author

Reopening in this case.

@mihaimaruseac mihaimaruseac reopened this Nov 10, 2020
@mpilgrem mpilgrem changed the title GHCi with internal libraries only works after a build stack ghci/repl with internal sublibraries only works after a build Jun 17, 2023
@andreasabel
Copy link

I am running into this in the hasktorch project:

@andreasabel
Copy link

@mihaimaruseac @mpilgrem : This issue has a milestone P0: blocking release from 2018. Maybe this milestone needs update, or is it intended that this issue will be fixed for the next release? (Would be nice.)

@mpilgrem mpilgrem removed this from the P0: Blocking release milestone Nov 3, 2023
@mpilgrem
Copy link
Member

mpilgrem commented Nov 3, 2023

@andreasabel, I've cleared the 'milestone' label as it was misleading. I'll likely find a different way of labelling the most important issues. I consider this issue to be one of the most important (in the top 3) because it interferes with the mechanism that HLS uses to get information from Stack.

@mihaimaruseac
Copy link
Contributor Author

Regarding implementation, I thought it was something easy back in 2018, but I realized I needed much more time to understand Stack's codebase. Then had a job change and didn't get a chance to look much into it, but hoping to be able to pick it again in the enar future.

@mpilgrem
Copy link
Member

mpilgrem commented Nov 5, 2023

To confirm: this still affects the master branch version of Stack (Version 2.14.0, Git revision 8c02ca86caacc1c30f16df91f5cc475375bed667 (10100 commits) PRE-RELEASE x86_64). Tested with a simple one-package project ghciTest, with package.yaml:

spec-version: 0.36.0
name: ghciTest
version: 0.1.0.0

dependencies:
- base >= 4.7 && < 5

library:
  source-dirs: src
  dependencies:
  - sub

internal-libraries:
  sub:
   source-dirs: sub
> stack purge
❯ stack ghci
ghciTest> configure (lib + sub-lib)
Configuring ghciTest-0.1.0.0...
ghciTest> initial-build-steps (lib + sub-lib)
Configuring GHCi with the following packages: ghciTest.
GHCi, version 9.4.7: https://www.haskell.org/ghc/  :? for help
<command line>: cannot satisfy -package z-ghciTest-z-sub
    (use -v for more information)
❯ stack build
ghciTest> build (lib + sub-lib)
Preprocessing library 'sub' for ghciTest-0.1.0.0..
Building library 'sub' for ghciTest-0.1.0.0..
[1 of 1] Compiling SubLib
Preprocessing library for ghciTest-0.1.0.0..
Building library for ghciTest-0.1.0.0..
[1 of 1] Compiling Lib
ghciTest> copy/register
Installing internal library sub in D:\Users\mike\Code\Haskell\ghciTest\.stack-work\install\d537d1c2\lib\x86_64-windows-ghc-9.4.7\ghciTest-0.1.0.0-Lgr8UKNJkvT1kMCZcjHaWw-sub
Installing library in D:\Users\mike\Code\Haskell\ghciTest\.stack-work\install\d537d1c2\lib\x86_64-windows-ghc-9.4.7\ghciTest-0.1.0.0-AgYgWsPNk3w95cAA2YGFHT
Registering library 'sub' for ghciTest-0.1.0.0..
Registering library for ghciTest-0.1.0.0..
❯ stack ghci
Configuring GHCi with the following packages: ghciTest.
GHCi, version 9.4.7: https://www.haskell.org/ghc/  :? for help
[1 of 2] Compiling SubLib           ( D:\Users\mike\Code\Haskell\ghciTest\sub\SubLib.hs, interpreted )
[2 of 2] Compiling Lib              ( D:\Users\mike\Code\Haskell\ghciTest\src\Lib.hs, interpreted )
Ok, two modules loaded.
Loaded GHCi configuration from C:\Users\mikep\AppData\Local\stack\ghci-script\8af0ba7e\ghci-script
ghci>

@mpilgrem
Copy link
Member

mpilgrem commented Nov 5, 2023

The command that fails is this (reformatted):

Run process within D:\Users\mike\Code\Haskell\ghciTest\:

D:\sr\programs\x86_64-windows\ghc-9.4.7\bin\ghc-9.4.7.exe
--interactive 
-i 
-odir=D:\Users\mike\Code\Haskell\ghciTest\.stack-work\odir 
-hidir=D:\Users\mike\Code\Haskell\ghciTest\.stack-work\odir 
-hide-all-packages 
-XHaskell2010 
-iD:\Users\mike\Code\Haskell\ghciTest\.stack-work\dist\74a2d300\build 
-iD:\Users\mike\Code\Haskell\ghciTest\src 
-iD:\Users\mike\Code\Haskell\ghciTest\.stack-work\dist\74a2d300\build\autogen 
-iD:\Users\mike\Code\Haskell\ghciTest\.stack-work\dist\74a2d300\build\global-autogen 
-stubdir=D:\Users\mike\Code\Haskell\ghciTest\.stack-work\dist\74a2d300\build 
-ID:\sr\programs\x86_64-windows\msys2-20230526\mingw64\include 
-LD:\sr\programs\x86_64-windows\msys2-20230526\mingw64\lib 
-LD:\sr\programs\x86_64-windows\msys2-20230526\mingw64\bin 
-package=z-ghciTest-z-sub    <<<< The problematic option <<<<
-package-id=base-4.17.2.0 
-iD:\Users\mike\Code\Haskell\ghciTest\.stack-work\dist\74a2d300\build\sub 
-iD:\Users\mike\Code\Haskell\ghciTest\sub 
-iD:\Users\mike\Code\Haskell\ghciTest\.stack-work\dist\74a2d300\build\sub\autogen 
-iD:\Users\mike\Code\Haskell\ghciTest\.stack-work\dist\74a2d300\build\sub\sub-tmp 
-haddock 
-optP-include -optPD:\Users\mike\Code\Haskell\ghciTest\.stack-work\ghci\bca0e71a\cabal_macros.h 
-ghci-script=C:\Users\mikep\AppData\Local\stack\ghci-script\8af0ba7e\ghci-script

The command that succeeds is this (reformatted), the difference is the -package= is replaced by -package-id=:

Run process within D:\Users\mike\Code\Haskell\ghciTest\: 

D:\sr\programs\x86_64-windows\ghc-9.4.7\bin\ghc-9.4.7.exe 
--interactive 
-i 
-odir=D:\Users\mike\Code\Haskell\ghciTest\.stack-work\odir 
-hidir=D:\Users\mike\Code\Haskell\ghciTest\.stack-work\odir 
-hide-all-packages 
-XHaskell2010 
-iD:\Users\mike\Code\Haskell\ghciTest\.stack-work\dist\74a2d300\build 
-iD:\Users\mike\Code\Haskell\ghciTest\src 
-iD:\Users\mike\Code\Haskell\ghciTest\.stack-work\dist\74a2d300\build\autogen 
-iD:\Users\mike\Code\Haskell\ghciTest\.stack-work\dist\74a2d300\build\global-autogen 
-stubdir=D:\Users\mike\Code\Haskell\ghciTest\.stack-work\dist\74a2d300\build 
-ID:\sr\programs\x86_64-windows\msys2-20230526\mingw64\include 
-LD:\sr\programs\x86_64-windows\msys2-20230526\mingw64\lib 
-LD:\sr\programs\x86_64-windows\msys2-20230526\mingw64\bin 
-package-id=ghciTest-0.1.0.0-Lgr8UKNJkvT1kMCZcjHaWw-sub   <<<< the installed id is known <<<<
-package-id=base-4.17.2.0 
-iD:\Users\mike\Code\Haskell\ghciTest\.stack-work\dist\74a2d300\build\sub 
-iD:\Users\mike\Code\Haskell\ghciTest\sub 
-iD:\Users\mike\Code\Haskell\ghciTest\.stack-work\dist\74a2d300\build\sub\autogen 
-iD:\Users\mike\Code\Haskell\ghciTest\.stack-work\dist\74a2d300\build\sub\sub-tmp 
-haddock 
-optP-include -optPD:\Users\mike\Code\Haskell\ghciTest\.stack-work\ghci\bca0e71a\cabal_macros.h 
-ghci-script=C:\Users\mikep\AppData\Local\stack\ghci-script\8af0ba7e\ghci-script

@ulidtko
Copy link

ulidtko commented Apr 15, 2024

Also hitting this, with HLS... bump + subscribe.

Severity: DiagnosticSeverity_Error
Message:
Failed to parse result of calling stack

Error: [S-6948]
Stack failed to parse the target(s).

While parsing, Stack encountered the error:

Directory not found: my-acme-package:lib:my-internal-sublib.

@mpilgrem
Copy link
Member

@ulidtko, my-acme-package:lib:my-internal-sublib is not a valid Stack target and I would not expect HLS to be using it. Are you using a hie.yaml 'cradle' and, if so, how is it configured?

@ulidtko
Copy link

ulidtko commented Apr 17, 2024

@mpilgrem yes, using hand-written cradle:

cradle:
  stack:
    - path: ./app/Apiserver.hs
      component: my-acme-package:exe:apiserver

    - path: ./test
      component: my-acme-package:test:test

    - path: ./src
      #component: my-acme-package:lib:my-internal-sublib
      #-- Can't specify the correct component name; it also fails to autodetect,
      #-- because of https://github.com/commercialhaskell/stack/issues/4148
      #-- Point HLS to "this directory" instead, and hope for the best.
      component: .

Something I failed to figure out and gave up on: Cabal docs, logs & source seem to suggest that these "internal sublibs" are also cabal package components.

Aren't they? stack ide targets doesn't list them, the paths mangle into the funny structure z-my-acme-package-z-my-internal-sublib, HLS (hie-bios?..) doesn't autodetect them, and apparently I can't name them in hie.yaml either.

However all that looks like just stack lacking proper support for named/internal/sub- libs.

For example here, Cabal defines ComponentName as:

-- Libraries live in a separate namespace, so must distinguish
data ComponentName
  = CLibName LibraryName
  | CNotLibName NotLibComponentName

and LibraryName is either the single MainLib, or a named SubLib:

data LibraryName
  = LMainLibName
  | LSubLibName UnqualComponentName

So naturally, we'd think about named sublibs as cabal package components — and therefore, valid stack targets — but it's guessable that actually they're heavily specialcased in stack, and supported very differently than other types of components.

@mpilgrem
Copy link
Member

mpilgrem commented Apr 17, 2024

@ulidtko, currently, Stack's target syntax my-package:lib targets all of the library components of package my-package. 'Currently' because #6042 is on my radar.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests