Skip to content

Commit

Permalink
Distributable binaries (#165)
Browse files Browse the repository at this point in the history
Update ghcide to obtain the GHC lib dir at runtime, rather than at compile time with ghc-paths. This means that the binaries can be moved about since the lib dir is obtained on the fly
Share the exe/main.hs logic between ghcide and hls: the session setup logic which previously took up most of exe/main.hs now resides inside the ghcide library, and is used by both ghcide and hls's executables
Add a --project-ghc-version option to the wrapper which spits out the project's ghc version to stdout. This is useful for the vscode extension which can then use it to download the corresponding version of binary that the wrapper would have otherwise attempted to launch
Make the wrapper check to see if the correct tool is installed beforehand. For example, if it detects a stack project but stack isn't on the path, it will report an error then and there, rather than having hls/ghcide confusingly fail later on. The vscode extension uses this new error message as well to provide a pop up message linking the user to a website to install the missing tool
Remove cabal-helper from the wrapper, so that the implicit cradle logic is the same between ghcide/hls/hls-wrapper
And of course, add a GitHub action workflow that runs whenever a release is created on GitHub that builds static binaries on Linux, and distributable enough binaries on macOS and windows. This is documented a bit more in docs/releases.md

* WIP

* WIP 2

* WIP 3

* WIP 4

* WIP 5

* WIP 6

* WIP 7

* WIP 8

* WIP 9 Use patched hie-bios to get libdir dynamically

* Try building the wrapper

* Try to fix build_wrapper env variable not being picked up

* Try again

* Give up on the env var idea

* Try out static optimised builds?

* Try squashing the working dir

* Woops

* Try squashing the builddir

* Try going into the parent directory

* Radical approach - don't use such a long name for the wrapper

* Use dist-binary flag

* Debug why floskell fails to build on windows

* haskell-language-server => hls on CI

I hate that I have to do this

* Employ extreme path saving measures

* sed time :(

* Try making sed command portable

* Compress artefacts

* Tidy up wrapper logging

* Use version checking logic in hie-bios

* Add documentation on the releases process

* Remove unused code

* Append .exe to windows binaries

* Try out building remaining supported ghc configurations

* Add wrapper tests and update hie-bios

* Use index timestamp that exists on hackage

Fixes warning

* Update hie-bios

* Update hie-bios

* Try building windows jobs on -j1

* Skip windows 8.8.2

* Update ghc-check to use hie-bios runtime ghc libdir

* Upload binaries as an artifact too

* Try flicking on enable-executable-static

I don't expect this to work, puffnfresh has already tried this and had
to fork ghcup

* Fix artifact upload

* Update to latest ghcide and reuse loadSession

* Check if the tool is installed in --project-ghc-version in the wrapper

* Fix wrapper tests by copying to temporary directory

* Try caching

* Tidy up and switch back to cabal helper implicit cradle

* use split sections

* Remove cabal-helper and replace it with hie-bios implicit logic

The cabal-helper cradle was only used by the wrapper for detecting the
project GHC version in the absence of an explicit hie.yaml file, whilst
ghcide itself used the hie-bios implicit cradle logic. This brings the
two in sync so the wrapper should behave more predictably now.

* Undo agpl common stanza change

* Add release number

Co-authored-by: amesgen <[email protected]>
  • Loading branch information
lukel97 and amesgen authored Jul 20, 2020
1 parent acaba05 commit 0c99ce0
Show file tree
Hide file tree
Showing 28 changed files with 426 additions and 1,644 deletions.
116 changes: 116 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
name: Builds

on:
release:
types: [created]
jobs:

build:
runs-on: ${{ matrix.os }}

strategy:
fail-fast: false
matrix:
ghc: ['8.10.1', '8.8.3', '8.8.2', '8.6.5', '8.6.4']
os: [ubuntu-latest, macOS-latest, windows-latest]
exclude:
- os: windows-latest
ghc: '8.8.3' # fails due to segfault
- os: windows-latest
ghc: '8.8.2' # fails due to error with Cabal

steps:
- uses: actions/checkout@v2
with:
submodules: true
- uses: actions/[email protected]
with:
ghc-version: ${{ matrix.ghc }}
cabal-version: '3.2'

- name: Cache Cabal
uses: actions/[email protected]
with:
path: ~/.cabal
key: ${{ runner.OS }}-${{ matrix.ghc }}-${{ hashFiles('**/*.cabal') }}

- name: Shorten binary names
shell: bash
run: |
sed -i.bak -e 's/haskell-language-server/hls/g' \
-e 's/haskell_language_server/hls/g' \
haskell-language-server.cabal
sed -i.bak -e 's/Paths_haskell_language_server/Paths_hls/g' \
src/**/*.hs exe/*.hs
- name: Set some window specific things
if: matrix.os == 'windows-latest'
shell: bash
run: |
echo '::set-env name=EXE_EXT::.exe'
- name: Set some linux specific things
if: matrix.os == 'ubuntu-latest'
run: |
echo '::set-env name=LINUX_CABAL_ARGS::--enable-executable-static --ghc-options=-split-sections'
- name: Build Server
shell: bash
# Try building it twice in case of flakey builds on Windows
run: |
cabal build exe:hls -O2 --disable-documentation $LINUX_CABAL_ARGS || \
cabal build exe:hls -O2 --disable-documentation $LINUX_CABAL_ARGS
- name: Find Server Binary
id: find_server_binary
shell: bash
run: |
HLS=$(find dist-newstyle \( -name 'hls' -o -name 'hls.exe' \) -type f)
gzip --best $HLS
echo ::set-output name=hls_binary::$HLS.gz
- name: Upload Server Binary
uses: actions/[email protected]
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ github.event.release.upload_url }}
asset_path: ${{ steps.find_server_binary.outputs.hls_binary }}
asset_name: haskell-language-server-${{ runner.OS }}-${{ matrix.ghc }}${{env.EXE_EXT}}.gz
asset_content_type: application/gzip

- uses: actions/upload-artifact@v2
with:
name: haskell-language-server-${{ runner.OS }}-${{ matrix.ghc }}${{env.EXE_EXT}}.gz
path: ${{ steps.find_server_binary.outputs.hls_binary }}

- name: Build Wrapper
if: matrix.ghc == '8.10.1'
run: cabal build exe:hls-wrapper -O2 --disable-documentation $WIN_CABAL_ARGS $LINUX_CABAL_ARGS

- name: Find Wrapper Binary
if: matrix.ghc == '8.10.1'
id: find_wrapper_binary
shell: bash
run: |
HLS_WRAPPER=$(find dist-newstyle \( -name 'hls-wrapper' -o -name 'hls-wrapper.exe' \) -type f)
gzip --best $HLS_WRAPPER
echo ::set-output name=hls_wrapper_binary::$HLS_WRAPPER.gz
- name: Upload Wrapper
if: matrix.ghc == '8.10.1'
uses: actions/[email protected]
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ github.event.release.upload_url }}
asset_path: ${{ steps.find_wrapper_binary.outputs.hls_wrapper_binary }}
asset_name: haskell-language-server-wrapper-${{ runner.OS }}${{env.EXE_EXT}}.gz
asset_content_type: application/gzip

- uses: actions/upload-artifact@v2
if: matrix.ghc == '8.10.1'
with:
name: haskell-language-server-wrapper-${{ runner.OS }}${{env.EXE_EXT}}.gz
path: ${{ steps.find_wrapper_binary.outputs.hls_wrapper_binary }}

3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,8 @@ shake.yaml.lock
stack*.yaml.lock
shake.yaml.lock

# ignore hie.yaml's for testdata
test/testdata/**/hie.yaml

# metadata files on macOS
.DS_Store
3 changes: 2 additions & 1 deletion .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,5 @@
# url = https://github.com/digital-asset/ghcide.git
# url = https://github.com/alanz/ghcide.git
# url = https://github.com/wz1000/ghcide.git
url = https://github.com/fendor/ghcide.git
# url = https://github.com/fendor/ghcide.git
url = https://github.com/bubba/ghcide.git
72 changes: 72 additions & 0 deletions docs/releases.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# Releases and distributable binaries

Starting with 0.3.0.0 haskell-language-server provides pre-built binaries on
each [GitHub
release](https://github.com/haskell/haskell-language-server/releases). These
binaries are used by the [vscode-hie-server
extension](https://github.com/alanz/vscode-hie-server) to provide automatic
installation for users on VS Code, but they can also be installed manually
when added to the path.

## Making a new release of haskell-language-server

Go to the [GitHub releases
page](https://github.com/haskell/haskell-language-server/releases) for
haskell-language-server and start to create a new release. Choose or create a
tag, fill out the release notes etc., but before you create it
**make sure to check the pre-release checkbox**. This will prevent VS Code
*extension
users from attempting to install this version before the binaries are
created.

Once the release is created the [GitHub Actions
workflow](https://github.com/haskell/haskell-language-server/actions) will be
kicked off and will start creating binaries. They will be gzipped and
uploaded to the release.

It creates a `haskell-language-server-OS-GHC` binary for each platform
(Linux, macOS, Windows) and each GHC version that we currently support, as well
as a `haskell-language-server-wrapper-OS` binary for each platform. Note that
only one wrapper binary is created per platform, and it should be built with the
most recent GHC version.

Once all these binaries are present

## Distributable binaries
In order to compile a hls binary on one machine and have it run on another, you
need to make sure there are **no hardcoded paths or data-files**.

### ghc libdir
One noteable thing which cannot be hardcoded is the **GHC libdir** – this is
a path to `/usr/local/lib/ghc` or something like that, which was previously
baked in at compile-time with ghc-paths. Note that with static binaries we
can no longer use this because the GHC libdir of the GitHub Actions machine
will most almost certainly not exist on the end user's machine.
Therefore, hie-bios provides `getGhcRuntimeLibDir` to obtain this path on the fly
by consulting the cradle.

### Static binaries
We use the word "distributable" here because technically only the Linux builds
are static. They are built by passing `--enable-executable-static` to cabal.
Static binaries don't really exist on macOS, and there are issues with
proprietary code being linked in on Windows. However, the `.dylib`s linked on
macOS are all already provided by the system:

```
$ objdump -macho --dylibs-used haskell-language-server
haskell-language-server:
/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
/usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)
/usr/lib/libcharset.1.dylib (compatibility version 2.0.0, current version 2.0.0)
```

## The GitHub Actions workflow
It just kicks off a matrix of jobs varying across GHC versions and OSs, building
the binaries with Cabal and extracting them from the dist-newstyle directory.
The binaries are built with -O2.

One caveat is that we need to rename the binaries from
haskell-language-server/haskell-language-server-wrapper to hls/hls-wrapper due to
path length limitations on windows. But whenever we upload them to the release,
we make sure to upload them as their full name variant.
18 changes: 7 additions & 11 deletions exe/Arguments.hs
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,11 @@
module Arguments
( Arguments(..)
, getArguments
, ghcideVersion
, getLibdir
, haskellLanguageServerVersion
) where

import Data.Maybe
import Data.Version
import Development.GitRev
import qualified GHC.Paths
import Options.Applicative
import Paths_haskell_language_server
import System.Environment
Expand All @@ -37,6 +34,7 @@ data Arguments = Arguments
, argsDebugOn :: Bool
, argsLogFile :: Maybe String
, argsThreads :: Int
, argsProjectGhcVersion :: Bool
} deriving Show

getArguments :: String -> IO Arguments
Expand Down Expand Up @@ -80,21 +78,19 @@ arguments exeName = Arguments
<> value 0
<> showDefault
)
<*> switch (long "project-ghc-version"
<> help "Work out the project GHC version and print it")

-- ---------------------------------------------------------------------
-- Set the GHC libdir to the nix libdir if it's present.
getLibdir :: IO FilePath
getLibdir = fromMaybe GHC.Paths.libdir <$> lookupEnv "NIX_GHC_LIBDIR"

ghcideVersion :: IO String
ghcideVersion = do
haskellLanguageServerVersion :: IO String
haskellLanguageServerVersion = do
path <- getExecutablePath
let gitHashSection = case $(gitHash) of
x | x == "UNKNOWN" -> ""
x -> " (GIT hash: " <> x <> ")"
return $ "ghcide version: " <> showVersion version
return $ "haskell-language-server version: " <> showVersion version
<> " (GHC: " <> VERSION_ghc
<> ") (PATH: " <> path <> ")"
<> gitHashSection

-- ---------------------------------------------------------------------
Loading

0 comments on commit 0c99ce0

Please sign in to comment.