Distributable binaries #165

lukel97 · 2020-06-18T20:02:06Z

Ignore the commits for now, they should be squashed when merged!

This PR contains a lot of stuff, a lot of it coming from upstream components so it might not be visible in the diff:

Update ghcide to obtain the GHC lib dir at runtime, rather than at compile time with ghc-paths. This means that the binaries can be moved about since the lib dir is obtained on the fly
Share the exe/main.hs logic between ghcide and hls: the session setup logic which previously took up most of exe/main.hs now resides inside the ghcide library, and is used by both ghcide and hls's executables
Add a --project-ghc-version option to the wrapper which spits out the project's ghc version to stdout. This is useful for the vscode extension which can then use it to download the corresponding version of binary that the wrapper would have otherwise attempted to launch
Make the wrapper check to see if the correct tool is installed beforehand. For example, if it detects a stack project but stack isn't on the path, it will report an error then and there, rather than having hls/ghcide confusingly fail later on. The vscode extension uses this new error message as well to provide a pop up message linking the user to a website to install the missing tool
Remove cabal-helper from the wrapper, so that the implicit cradle logic is the same between ghcide/hls/hls-wrapper
And of course, add a GitHub action workflow that runs whenever a release is created on GitHub that builds static binaries on Linux, and distributable enough binaries on macOS and windows. This is documented a bit more in docs/releases.md

I hate that I have to do this

fendor · 2020-06-18T20:48:48Z

Figure out how to implement runGhcLibDir for the cabal helper cradles

Practically, CabalHelper is only used in HLS-wrapper. In my opinion, we may remove it if it is not really used.

Also, there is the implementation of https://github.com/haskell/haskell-language-server/blob/master/src/Ide/Cradle.hs#L158 which obtained the libdir in Haskell IDE Engine. That should work, right?

lukel97 · 2020-07-16T17:07:49Z

The stuff that needs to go into ghcide has PRs open here, but has been cherry-picked into the ghcide submodule for now haskell/ghcide#697 haskell/ghcide#696

alanz

I am ok provided @wz1000 and co are happy

alanz · 2020-07-16T17:33:17Z

.gitmodules

@@ -13,4 +13,5 @@
 	# url = https://github.com/digital-asset/ghcide.git
 	# url = https://github.com/alanz/ghcide.git
 	# url = https://github.com/wz1000/ghcide.git
-	url = https://github.com/fendor/ghcide.git
+	# url = https://github.com/fendor/ghcide.git
+	url = https://github.com/bubba/ghcide.git


Is this the correct upstream repo?

Until those other PRs and @wz1000s PRs get merged into ghcide, I think so. This checkout is just @fendor's previous submodule + my patches cherry-picked on top

alanz · 2020-07-16T17:33:44Z

docs/releases.md

@@ -0,0 +1,72 @@
+# Releases and distributable binaries
+
+Starting with 0.X.0.0 haskell-language-server provides pre-built binaries on


I imagine 0.3.0.0, if we merge soon.

use split sections

The cabal-helper cradle was only used by the wrapper for detecting the project GHC version in the absence of an explicit hie.yaml file, whilst ghcide itself used the hie-bios implicit cradle logic. This brings the two in sync so the wrapper should behave more predictably now.

…age-server into github-action-builds

lukel97 · 2020-07-17T12:57:38Z

Now that the tests appear to be passing, review/test away @fendor @jneira etc.

jneira · 2020-07-17T13:00:34Z

I hope we will not miss cabal-helper in the future (although for missing something you need remembering it 😝 )

fendor · 2020-07-17T13:13:05Z

src/Ide/Cradle.hs

@@ -1,862 +0,0 @@
-{-# LANGUAGE ScopedTypeVariables #-}


Here rests my biggest contribution to HIE.
RIP

The most beautifully documented module ⚰️

fendor · 2020-07-17T13:13:57Z

exe/Wrapper.hs

+-- | Find the cradle that the given File belongs to.
+--
+-- First looks for a "hie.yaml" file in the directory of the file
+-- or one of its parents. If this file is found, the cradle
+-- is read from the config. If this config does not comply to the "hie.yaml"
+-- specification, an error is raised.
+--
+-- If no "hie.yaml" can be found, the implicit config is used.
+-- The implicit config uses different heuristics to determine the type
+-- of the project that may or may not be accurate.
+findLocalCradle :: FilePath -> IO (Cradle Void)
+findLocalCradle fp = do
+  cradleConf <- findCradle fp
+  crdl       <- case cradleConf of
+    Just yaml -> do
+      hPutStrLn stderr $ "Found \"" ++ yaml ++ "\" for \"" ++ fp ++ "\""
+      loadCradle yaml
+    Nothing -> loadImplicitCradle fp
+  hPutStrLn stderr $ "Module \"" ++ fp ++ "\" is loaded by Cradle: " ++ show crdl
+  return crdl


Dont we need something similar for exe/Main.hs? So, should be put into the library, also for writing tests against it.

The corresponding line in Ghcide's exe/Main.hs (src/Development/Ide/Session.hs now) is

cradle <- maybe (loadImplicitCradle $ addTrailingPathSeparator dir) loadCradle hieYaml

but it looks like the hieYaml is cached in some memo thingy

cradleLoc <- liftIO $ memoIO $ \v -> do res <- findCradle v -- Sometimes we get C:, sometimes we get c:, and sometimes we get a relative path -- try and normalise that -- e.g. see https://github.com/digital-asset/ghcide/issues/126 res' <- traverse makeAbsolute res return $ normalise <$> res'

I would have liked to have shared that bit but I'm not 100% sure how to abstract it

Is that still used? I can imagine that we do not need to cache this.

@fendor After looking at it a bit closer I think the memoisation part does make sense, since I presume that shake action can get called/executed (not sure what the terminology is) multiple times on an individual file. And since findCradle does a lot of directory traversing we probably don't want to do that every single time

fendor

One small question, other than that, just awesome!

fendor · 2020-07-20T11:14:26Z

exe/Wrapper.hs

 import HIE.Bios
-import           Ide.Cradle (findLocalCradle)
-import           Ide.Logger (logm)


Why is the logger gone here?

The logger never worked in the first place from what I could tell, since the wrapper didn't call the haskell-lsp setup function! So all those logm/debugm calls seemed to just go down the drain.

fendor · 2020-07-20T11:16:13Z

docs/releases.md

@@ -0,0 +1,72 @@
+# Releases and distributable binaries


Awesome Documentation!

jneira · 2020-07-20T20:54:20Z

@bubba sorry, due to windows build issues i had no time to test it again. 😟

lukel97 added 29 commits June 12, 2020 18:17

WIP

69b4f07

WIP 2

0e30291

WIP 3

43c8076

WIP 4

8ba974b

WIP 5

cde25d7

WIP 6

c328c80

WIP 7

2a5b400

WIP 8

ffe3d4e

WIP 9 Use patched hie-bios to get libdir dynamically

7c7c207

Try building the wrapper

bf59f29

Try to fix build_wrapper env variable not being picked up

0ace84b

Try again

32c5937

Give up on the env var idea

1a66953

Try out static optimised builds?

5d3fed7

Try squashing the working dir

191272a

Woops

a3b309c

Try squashing the builddir

a626ffc

Try going into the parent directory

4219d51

Radical approach - don't use such a long name for the wrapper

dc99a50

Use dist-binary flag

277f20e

Debug why floskell fails to build on windows

4feadad

haskell-language-server => hls on CI

f4a7fb8

I hate that I have to do this

Employ extreme path saving measures

a91ae03

sed time :(

d53fb6e

Try making sed command portable

7e0402b

Compress artefacts

505ee0a

Tidy up wrapper logging

304f4c3

Use version checking logic in hie-bios

0c48be8

Add documentation on the releases process

65eb6c4

lukel97 force-pushed the github-action-builds branch from 409ca9c to 773ec86 Compare July 16, 2020 16:30

lukel97 added 2 commits July 16, 2020 17:37

Try caching

bcfa908

Tidy up and switch back to cabal helper implicit cradle

53c3676

lukel97 force-pushed the github-action-builds branch from 43fb062 to 53c3676 Compare July 16, 2020 16:48

lukel97 marked this pull request as ready for review July 16, 2020 17:04

lukel97 requested review from alanz, jneira, fendor and Ailrun and removed request for alanz and jneira July 16, 2020 17:05

alanz approved these changes Jul 16, 2020

View reviewed changes

amesgen and others added 4 commits July 16, 2020 19:55

use split sections

54ce0ec

Merge pull request #1 from amesgen/use-split-sections

c162fca

use split sections

Merge branch 'github-action-builds' of github.com:bubba/haskell-langu…

72abb71

…age-server into github-action-builds

lukel97 changed the title ~~GitHub Action static binaries and runtime libdir~~ Distributable binaries Jul 17, 2020

lukel97 added 3 commits July 17, 2020 13:46

Merge remote-tracking branch 'origin/master' into github-action-builds

1cd5016

Undo agpl common stanza change

9238ced

Add release number

ebe5ef1

fendor reviewed Jul 17, 2020

View reviewed changes

fendor approved these changes Jul 20, 2020

View reviewed changes

Merge remote-tracking branch 'origin/master' into github-action-builds

3e2ac6f

lukel97 merged commit 0c99ce0 into haskell:master Jul 20, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Distributable binaries #165

Distributable binaries #165

lukel97 commented Jun 18, 2020 •

edited

Loading

fendor commented Jun 18, 2020 •

edited

Loading

lukel97 commented Jul 16, 2020

alanz left a comment

alanz Jul 16, 2020

lukel97 Jul 16, 2020

alanz Jul 16, 2020

lukel97 commented Jul 17, 2020

jneira commented Jul 17, 2020

fendor Jul 17, 2020

lukel97 Jul 17, 2020

fendor Jul 17, 2020 •

edited

Loading

lukel97 Jul 17, 2020

fendor Jul 17, 2020

lukel97 Jul 20, 2020

fendor left a comment

fendor Jul 20, 2020

lukel97 Jul 20, 2020

fendor Jul 20, 2020

jneira commented Jul 20, 2020

		@@ -0,0 +1,72 @@
		# Releases and distributable binaries

		Starting with 0.X.0.0 haskell-language-server provides pre-built binaries on

Distributable binaries #165

Distributable binaries #165

Conversation

lukel97 commented Jun 18, 2020 • edited Loading

fendor commented Jun 18, 2020 • edited Loading

lukel97 commented Jul 16, 2020

alanz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lukel97 commented Jul 17, 2020

jneira commented Jul 17, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fendor Jul 17, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fendor left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jneira commented Jul 20, 2020

lukel97 commented Jun 18, 2020 •

edited

Loading

fendor commented Jun 18, 2020 •

edited

Loading

fendor Jul 17, 2020 •

edited

Loading