HIE depends on hlint.yaml and cabal-helper-wrapper in its build dir #400

k0aaa · 2017-12-10T03:51:40Z

Following the installing instructions, the hie ends up using libraries in the project direcotory(i.e. <hie_project_dir>/.stack_work), supposingly they should be installed to the global stack project(i.e %STACK_ROOT%/snapshots)
This causes duplicating libraries in both locations, which is space ineffecient

nponeccop · 2017-12-10T03:58:12Z

Hie is linked statically. That is, the libraries aren't used at runtime, everything is in .local/bin/hie.

Many libraries are installed at .stack_work, because they don't belong to snapshots. But %STACK_ROOT%/snapshots should still contain some 350+ libs.

For 32-bit 8.0.2 check for example snapshots/i386-linux-nopie/lts-9.14/8.0.2/lib/i386-linux-ghc-8.0.2/ folder. It contains some 300 libs.

k0aaa · 2017-12-10T03:59:06Z

When i renamed the hie project directory, hie ended up not functioning
How is that supposed to be?

k0aaa · 2017-12-10T04:04:23Z

btw are you really sure?
cuz i got only 190 libs in the snapshot

nponeccop · 2017-12-10T04:09:48Z

I think 190 libs are normal. I may have libs there from other projects as well on my system. For most functions of hie to work you need only hie.exe on your %PATH%.

What exactly stopped working?

k0aaa · 2017-12-10T04:15:10Z

So im using the VS code integration.
when i rename the dir,
Linting stops working, ghc-mod doesnt respond, formatting doesnt work, go to def doesnt work even in the same file
basically the server doesnt respond anything and nothing works
I can check more if you could tell me where the log lies or how to work woth HIE under command line

k0aaa · 2017-12-10T04:16:37Z

Is it normal that with all libs statically linked, hie.exe is still only 119MB large?

nponeccop · 2017-12-10T04:23:46Z

Yes, the static libs due to various historical and performance reasons use a very inefficient format, so a very little fraction of them is code - the rest are section headers and so on.

119 is normal. 32-bit stack on linux is 44MBs.

You can check dynamic dependencies of a windows exe using this tool: http://dependencywalker.com/.
On linux it's ldd.

nponeccop · 2017-12-10T04:27:18Z

The log is in %temp%\hie.log on Windows.

To work with command line run hie.exe --lsp and see if it crashes or not. In this configuration it doesn't create any logs but VS code plugin passes extra arguments.

ghc-mod must work with only hie.exe, so let's focus on it.

Delete the log, open vscode and attempt to produce a ghc-mod error, and paste the log here. Also check that you don't have more than one hie.exe and your path doesn't contain subfolders of hie build folder.

k0aaa · 2017-12-10T04:35:47Z

The DepVerifier shows some dlls in c:\windows and some not found api-ms-....dll and ext-ms-....dlls

k0aaa · 2017-12-10T04:39:09Z

It doesnt crash, and when run in command line, the output didnt change
The only difference is that it doesnt respond to the addons

k0aaa · 2017-12-10T05:01:18Z

https://gist.github.com/soylens/30096602d210e73793810fab1e117602
the 2 logs and the problematic code

k0aaa · 2017-12-10T05:17:03Z

@nponeccop

k0aaa · 2017-12-10T06:25:37Z

Is there an operation to "merge" the project to the global project?

nponeccop · 2017-12-10T06:25:51Z

It's 2 am here. Talk to you tomorrow :)

Both of your logs contain bad uris:

2017-12-10 12:56:59.0872056 [ThreadId 3] - ---> {"jsonrpc":"2.0","method":"textDocument/didChange","params":{"textDocument":{"uri":"file:///f%3A/TEMP/DEBUG/tROUBLE-orig.hs","version":3},"contentChanges":[{"range":{"start":{"line":18,"character":12},"end":{"line":18,"character":12}},"rangeLength":0,"text":")"}]}}

There is f%3A instead of correct F:. This is a manifestation of a bug in haskell-lsp. The bug was fixed in haskell/lsp#63 but it hasn't been merged yet. @alanz?

The issue you experiences is a duplicate of #329. We'll update everything soon.

k0aaa · 2017-12-10T06:28:00Z

Are you sure this is the cause?

nponeccop · 2017-12-10T06:31:04Z

I'm sure that %3a causes big problems. But I'm not sure that this %3a bug is the only problem you have. Let's fix it and see

k0aaa · 2017-12-10T06:40:32Z

So do I have to do stack clean or whatever before building the later HIE upon update?

nponeccop · 2017-12-10T06:41:46Z

git pull and stack install is enough

k0aaa · 2017-12-10T06:43:41Z

Thx have a good sleep

k0aaa · 2017-12-10T08:07:04Z

I just noticed that the code range of the diagnostics were L20C0 to L20C100000, this might be the cause for another of my issue

alanz · 2017-12-10T08:35:50Z

HIE installs a whole lot of stuff into the project 'stack-work directory because it is using git-based dependencies for some things, and these go in the local directory.

Once all the dependencies are in a stackage snapshot, this will cease to be a problem.

k0aaa · 2017-12-10T09:58:50Z

Is it planned to be uploaded to the hackage tree any sooner?

alanz · 2017-12-10T10:06:52Z

That's why I am currently focusing on cabal-helper. It is blocking a ghc-mod update, which is blocking HaRe and HIE

k0aaa · 2017-12-10T11:18:03Z

So I just checked and all packages pulled by git during stack install are in the hackage tree. So i suppose what you mean is that due to compatability issues, hie uses outdated or dev versions of those packages?

k0aaa · 2017-12-10T11:34:16Z

@alanz BTW is Cabal-helper still alive? It stopped updating into Hackage since Jan 2017 and seems to be failing the builds for quite a time.

alanz · 2017-12-10T12:28:32Z

yes, see https://gitlab.com/dxld/cabal-helper/pipelines and https://github.com/DanielG/cabal-helper/network

k0aaa · 2017-12-10T12:41:09Z

Thx, I'll keep myself updated to this

nponeccop · 2017-12-16T21:38:47Z

@soylens Can you reproduce the issue and recollect the logs with updated hie?

Note that it only makes sense to use the 8.2.2 version if your project is 8.2.2 too - ghc versions must match

k0aaa · 2017-12-17T13:37:10Z

wait I saw this as the wrong issue XD

k0aaa · 2017-12-17T13:45:17Z

Nope, it still doesnt work unless the build dir remains in place.
All of brittany, ghc-mod, Hare and hlint still gets installed in .stack-work

nponeccop · 2017-12-19T03:11:08Z

okay. I ran strace on HIE and found that hie actually uses 2 files from its build directory:

cabal-helper-wrapper from $builddir/.stack-work/install/i386-linux-nopie/lts-9.14/8.0.2/libexec/i386-linux-ghc-8.0.2/cabal-helper-0.8.0.0/cabal-helper-wrapper
hlint.yaml from $builddir/.stack-work/install/i386-linux-nopie/lts-9.14/8.0.2/share/i386-linux-ghc-8.0.2/hlint-2.0.11/hlint.yaml

Having these ones in the snapshot doesn't help us as snapshots are disposable caches of build products. They take lots of space and are subject to purges. We should take the route of normal per-user installation instead:

cabal-helper-wrapper and hie go to ~/.local/bin - we only need to change build instructions
hlint.yaml goes to ~/.local/share/hlint/hlint.yaml - we need to figure out how to configure hlint with --datadir option, as it uses getDataDir call here

@alanz any thoughts?

k0aaa · 2018-02-03T02:42:51Z

Has this problem been resolved? @alanz

alanz · 2018-02-03T08:58:36Z

We switched to using hlint from hackage, so the hlint.yaml should be the global one.

We put out a new version of cabal-helper (to hackage) but there was a silly bug, now fixed in master.

So until that is published to hackage, it will use the one from the build dir

marmistrz · 2018-03-29T09:42:12Z

For me under Arch HIE wants:

hlint.yaml from ~/.stack/snapshots/x86_64-linux-tinfo6-nopie/nightly-2018-02-26/8.2.2/share/x86_64-linux-ghc-8.2.2/hlint-2.1/hlint.yaml
the ~/.stack/programs directory:

hie: /home/marcin/.stack/programs/x86_64-linux/ghc-tinfo6-nopie-8.2.2/lib/ghc-8.2.2/settings: openFile: does not exist (No such file or directory)

After removing the build directory (~/.cache/pacaur/haskell-ide-engine-git) there are no visible issues. Using the AUR package.

samuelpilz · 2019-04-05T14:09:07Z

This is still an issue!

Hie complains that it does not find the cabal-helper-wrapper executable it requires, which it expects to be in /home/vagrant/haskell-ide-engine/.stack-work/install/x86_64-linux-tinfo6/nightly-2019-03-07/8.6.4/libexec/x86_64-linux-ghc-8.6.4/cabal-helper-0.9.0.0/cabal-helper-wrapper. I built hie from /home/vagrant/haskell-ide-engine using as much isolation as possible.

@alanz I thought cabal-helper-wrapper has been replaced by something else. Is this not the case?

This currently blocks my efforts to make binary-packages for linux distributions. @mpickering Does your work for the hie-bios solve these issues?

mpickering · 2019-04-05T14:20:43Z

Yes my branch doesn't use cabal-helper-wrapper so it is probably easier to package.

samuelpilz · 2019-04-05T14:23:51Z

which one is it? I would like to give it a try. Is it mpickering/haskell-ide-engine:master?

samuelpilz · 2019-04-05T15:16:50Z

Another issue related to cabal-helper-wrapper: It is possible to set the cabal-helper-libexecdir environment variable and copy the built cabal-helper-wrapper binary into that directory. However, then the settings file of the ghc that compiled the cabal-helper-wrapper needs to be present... sigh.

I really hope that the issues related to cabal-helper-wrapper can be resolved fairly quickly. @mpickering, I appreciate your effort.

alanz · 2019-04-05T15:18:19Z

Ping @DanielG

samuelpilz · 2019-04-05T15:21:27Z

It seems that cabal-helper has some nostalgia-issues. It does not work without finding the compiler that was compiling it 😉

shinzui · 2019-04-24T16:30:18Z

This is still an issue on macOS. I built the binaries in a tmp dir then deleted the folder which broke hie. I discovered the problem by looking at neovim's language server logs.

Language server logs

2019-04-24 09:09:22.422921 [ThreadId 4] - run entered for hie-wrapper(hie-wrapper) Version 0.8.0.0 (2573 commits) x86_64 ghc-8.6.4
2019-04-24 09:09:22.424822 [ThreadId 4] - Current directory:/Users/shinzui/Gyehoeg/asobiba/haskell/
2019-04-24 09:09:22.703527 [ThreadId 4] - Cradle directory:/Users/shinzui/Gyehoeg/asobiba/haskell
2019-04-24 09:09:22.704841 [ThreadId 4] - Using stack GHC version
2019-04-24 09:09:22.883989 [ThreadId 4] - Project GHC version:8.6.4
2019-04-24 09:09:22.884628 [ThreadId 4] - hie exe candidates :["hie-8.6.4","hie-8.6","hie"]
2019-04-24 09:09:22.885694 [ThreadId 4] - found hie exe at:/Users/shinzui/.local/bin/hie-8.6.4
2019-04-24 09:09:22.886608 [ThreadId 4] - args:[]
2019-04-24 09:09:22.887044 [ThreadId 4] - launching ....



2019-04-24 09:09:22.899466 [ThreadId 4] - Using stack GHC version
2019-04-24 09:09:23.080625 [ThreadId 4] - Run entered for HIE(hie-8.6.4) Version 0.8.0.0 (2573 commits) x86_64 ghc-8.6.4
2019-04-24 09:09:23.081832 [ThreadId 4] - Current directory:/Users/shinzui/Gyehoeg/asobiba/haskell/
2019-04-24 09:09:23.113792 [ThreadId 10] - Using stack GHC version
hie-8.6.4: Could not find $libexecdir/cabal-helper-wrapper

If you are a cabal-helper developer you can set the environment variable
`cabal_helper_libexecdir' to override $libexecdir[1]. The following will
work in the cabal-helper source tree:

    `$ export cabal_helper_libexecdir=$PWD/dist/build/cabal-helper-wrapper`

[1]: /private/tmp/haskell-ide-engine/.stack-work/install/x86_64-osx/lts-13.15/8.6.4/libexec/x86_64-osx-ghc-8.6.4/cabal-helper-0.9.0.0
/private/tmp/haskell-ide-engine/.stack-work/install/x86_64-osx/lts-13.15/8.6.4/libexec/x86_64-osx-ghc-8.6.4/cabal-helper-0.9.0.0/cabal-helper-wrapper
junk/build/cabal-helper-wrapper/cabal-helper-wrapper
/private/tmp/haskell-ide-engine/.stack-work/install/x86_64-osx/lts-13.15/8.6.4/bin/cabal-helper-wrapper

If you don't know what I'm talking about something went wrong with your
installation. Please report this problem here:

    https://github.com/DanielG/cabal-helper/issues
2019-04-24 09:12:29.29119 [ThreadId 4] - run entered for hie-wrapper(hie-wrapper) Version 0.8.0.0 (2573 commits) x86_64 ghc-8.6.4
2019-04-24 09:12:29.293075 [ThreadId 4] - Current directory:/Users/shinzui/Gyehoeg/asobiba/haskell
2019-04-24 09:12:29.48922 [ThreadId 4] - Cradle directory:/Users/shinzui/Gyehoeg/asobiba/haskell
2019-04-24 09:12:29.490459 [ThreadId 4] - Using stack GHC version
2019-04-24 09:12:29.677505 [ThreadId 4] - Project GHC version:8.6.4
2019-04-24 09:12:29.678143 [ThreadId 4] - hie exe candidates :["hie-8.6.4","hie-8.6","hie"]
2019-04-24 09:12:29.679187 [ThreadId 4] - found hie exe at:/Users/shinzui/.local/bin/hie-8.6.4
2019-04-24 09:12:29.680017 [ThreadId 4] - args:[]
2019-04-24 09:12:29.680446 [ThreadId 4] - launching ....

2019-04-24 09:12:29.716384 [ThreadId 4] - Using stack GHC version
2019-04-24 09:12:29.885745 [ThreadId 4] - Run entered for HIE(hie-8.6.4) Version 0.8.0.0 (2573 commits) x86_64 ghc-8.6.4
2019-04-24 09:12:29.887035 [ThreadId 4] - Current directory:/Users/shinzui/Gyehoeg/asobiba/haskell/
2019-04-24 09:12:29.919378 [ThreadId 10] - Using stack GHC version

samuelpilz · 2019-04-25T08:36:52Z

@shinzui, this is still a known issue and some effort is being made towards fixing this

jneira · 2019-10-01T07:40:39Z

This is related with those issues:

Package hie binaries #1068 Package hie binaries
Rethink/rework the HIE BIOS #1053 / Implement the HIE Bios #1126 Work to replace cabal-helper with hie-bios
Use new cabal-helper #1245 Pr to upgrade cabal-helper that removes cabal-helper-wrapper

fendor · 2019-12-20T22:11:02Z

With the merge of #1126, now only the hlint.yaml files are a problem. These can be fixed quite easily from now on.

jneira · 2020-01-24T21:09:19Z

And with #1588 hie uses the hlint version that include hlint.yaml within the executable so... closing! 🎉

k0aaa changed the title ~~HIE doesnt install into snapshot~~ HIE doesnt install into stack snapshot Dec 10, 2017

k0aaa changed the title ~~HIE doesnt install into stack snapshot~~ HIE depends on libs/bins in its build dir Dec 10, 2017

sniperrifle2004 mentioned this issue Dec 13, 2017

Complete unresponsiveness Tehnix/ide-haskell-hie#5

Closed

nponeccop changed the title ~~HIE depends on libs/bins in its build dir~~ HIE depends on hlint.yaml and cabal-helper-wrapper in its build dir Dec 19, 2017

This was referenced Dec 19, 2017

HLint and neovim errors #390

Closed

How do I figure out if it's working? Tehnix/ide-haskell-hie#8

Closed

k0aaa mentioned this issue Dec 20, 2017

VsCode under OSX - hie not working haskell/vscode-haskell#36

Closed

marmistrz mentioned this issue Mar 23, 2018

HIE fails when run from within VSCode #483

Closed

alanz added this to the Some time milestone Dec 24, 2018

nponeccop mentioned this issue Mar 21, 2019

Hlint data-files can not be easily distributed #1143

Closed

nponeccop mentioned this issue Oct 1, 2019

No output/effect in neo-vim, no obvious errors #1090

Open

jneira closed this as completed Jan 24, 2020

HIE depends on hlint.yaml and cabal-helper-wrapper in its build dir #400

HIE depends on hlint.yaml and cabal-helper-wrapper in its build dir #400

Comments

k0aaa commented Dec 10, 2017 • edited Loading

nponeccop commented Dec 10, 2017

k0aaa commented Dec 10, 2017

k0aaa commented Dec 10, 2017 • edited Loading

nponeccop commented Dec 10, 2017

k0aaa commented Dec 10, 2017 • edited Loading

k0aaa commented Dec 10, 2017

nponeccop commented Dec 10, 2017 • edited Loading

nponeccop commented Dec 10, 2017 • edited Loading

k0aaa commented Dec 10, 2017

k0aaa commented Dec 10, 2017

k0aaa commented Dec 10, 2017 • edited Loading

k0aaa commented Dec 10, 2017

k0aaa commented Dec 10, 2017

nponeccop commented Dec 10, 2017

k0aaa commented Dec 10, 2017

nponeccop commented Dec 10, 2017

k0aaa commented Dec 10, 2017

nponeccop commented Dec 10, 2017

k0aaa commented Dec 10, 2017

k0aaa commented Dec 10, 2017

alanz commented Dec 10, 2017

k0aaa commented Dec 10, 2017

alanz commented Dec 10, 2017

k0aaa commented Dec 10, 2017 • edited Loading

k0aaa commented Dec 10, 2017

alanz commented Dec 10, 2017

k0aaa commented Dec 10, 2017

nponeccop commented Dec 16, 2017

k0aaa commented Dec 17, 2017 • edited Loading

k0aaa commented Dec 17, 2017 • edited Loading

nponeccop commented Dec 19, 2017 • edited Loading

k0aaa commented Feb 3, 2018

alanz commented Feb 3, 2018

marmistrz commented Mar 29, 2018

samuelpilz commented Apr 5, 2019 • edited Loading

mpickering commented Apr 5, 2019

samuelpilz commented Apr 5, 2019 • edited Loading

samuelpilz commented Apr 5, 2019

alanz commented Apr 5, 2019

samuelpilz commented Apr 5, 2019

shinzui commented Apr 24, 2019

samuelpilz commented Apr 25, 2019

jneira commented Oct 1, 2019

fendor commented Dec 20, 2019

jneira commented Jan 24, 2020

k0aaa commented Dec 10, 2017 •

edited

Loading

k0aaa commented Dec 10, 2017 •

edited

Loading

k0aaa commented Dec 10, 2017 •

edited

Loading

nponeccop commented Dec 10, 2017 •

edited

Loading

nponeccop commented Dec 10, 2017 •

edited

Loading

k0aaa commented Dec 10, 2017 •

edited

Loading

k0aaa commented Dec 10, 2017 •

edited

Loading

k0aaa commented Dec 17, 2017 •

edited

Loading

k0aaa commented Dec 17, 2017 •

edited

Loading

nponeccop commented Dec 19, 2017 •

edited

Loading

samuelpilz commented Apr 5, 2019 •

edited

Loading

samuelpilz commented Apr 5, 2019 •

edited

Loading