Failing ghcide tests in windows

[jneira]

I've run the test suite locally and in azure ci and i've got those failing tests:

 cpp
  cpp-error:                                        FAIL (4.16s)
diagnostics
  add missing module (non workspace):               FAIL
watched files
  non workspace file:                               FAIL

Full test log of failing tests here: https://gist.github.com/jneira/afe77151a540c5a27943d7661eefe3fa

[jneira]

Locally (windows 7) i've got an additional one:

  plugins:                                            FAIL (0.45s)
    test/src\Development\IDE\Test.hs:35:
    Could not find (DsError,(8,14),"Variable not in scope: c") in List [Diagnostic {_range =
Range {_start = Position {_line = 0, _character = 0}, _end = Position {_line = 100000, _character = 0}}, _severity = Just DsError, _code = Nothing, _source = Just "typecheck", _message = "Unexpected usage error\nCould not find module \8216GHC.TypeLits.KnownNat.Solver\8217\nUse -v to see a list of the files searched for.\n", _relatedInformation = Nothing}]

[ndmitchell]

One of the CPP ones is actually a GHC bug which is being fixed as part of https://gitlab.haskell.org/ghc/ghc/issues/17786

[jneira]

I am thinking in creating a helper function knowBrokenInWindows, use it in the actual failing tests for windows.
At least the test suite will be executed.

[jneira]

Local failing tests:

cpp
    cpp-error:                                                                                            FAIL (3.30s)
      test/src\Development\IDE\Test.hs:41:
      Could not find (DsError,(2,1),"error: unterminated") in List [Diagnostic {_range = Range {_start = Position {_line = 0, _character = 0}, _end = Position {_line = 1, _character = 0}}, _severity = Just DsError, _code = Nothing, _source = Just "compiler", _message = "`gcc.exe' failed in phase `C pre-processor'. (Exit code: 1)", _tags = Nothing, _relatedInformation = Nothing}]

add missing module (non workspace):                                                                   FAIL
      Exception: Maybe.fromJust: Nothing
      CallStack (from HasCallStack):
        error, called at libraries\base\Data\Maybe.hs:148:21 in base:Data.Maybe
        fromJust, called at test/src\Development\IDE\Test.hs:108:59 in main:Development.IDE.Test

Haddock html links:                                                                                 FAIL (3.37s)
        test\\exe\\Main.hs:2132:
        failed to find: `[Documentation](file:///` in hover message:
        ```haskell
        Data.Text.Internal.Text
        :: *
        ```
        *       *       *
        *Defined in ‘Data.Text.Internal’*
        *       *       *


        A space efficient, packed, unboxed Unicode text type.*  *       *

plugins:                                                                                                FAIL (0.78s)
    test/src\Development\IDE\Test.hs:41:
    Could not find (DsError,(8,14),"Variable not in scope: c") in List [Diagnostic {_range = Range {_start = Position {_line = 0, _character = 0}, _end = Position {_line = 1, _character = 0}}, _severity = Just DsError, _code = Nothing, _source = Just "typecheck", _message = "Unexpected usage error\nCould not find module \8216GHC.TypeLits.KnownNat.Solver\8217\nUse -v (or `:set -v` in ghci) to see a list of the files searched for.\n", _tags = Nothing, _relatedInformation = Nothing}]

non workspace file:                                                                                   FAIL (0.60s)
      test\\exe\\Main.hs:576:
      expected: 3
       but got: 5

[jneira]

After haskell/ghcide#821 the ignored tests are:

• cpp/cpp-error: @ndmitchell noted that it would be related with a ghc issue. It was merged 6 months ago but we dont have a reliable ghc version for windows yet
• plugin tests: those tests are problematic in the ci env for ghc > 8.6.5, sometimes fails, other times the test passes (the second one) but they throws weird exceptions frequently or even do not terminate
  • simple plugin: it fails consistently but sometimes throws exceptions: Exception: fd:3: hPutBuf: resource vanished (Broken pipe)
    • when the test check itself fail is with:

Bad interface file: C:\\sr\\snapshots\\5530c14b\\lib\\x86_64-windows-ghc-8.10.1\\ghc-typelits-knownnat-0.7.2-2xmxeOP6eE9BjPxla9bOqw\\GHC\\TypeLits\\KnownNat.hi\n

• parsedResultAction plugin: also throws Broken pipe exceptions or lsp session timeout but it fails too in ghc-8.10.1 with a unexpected diagnostic:

Not in scope:
  type constructor or class GHC.Records.Extra.HasField
Perhaps you meant GHC.Records.Extra.HasField (imported from GHC.Records.Extra)
Module GHC.Records.Extra does not export HasField.

• but knownBroken also fails! weird

[jneira]

Frequent transient errors:

    ghcide restarts shake session on config changes:                                                      FAIL
      Exception: Timed out waiting to receive a message from the server.
      Last message received:
      {
          "tag": "NotWorkDoneProgressEnd",
          "contents": {
              "jsonrpc": "2.0",
              "params": {
                  "value": {
                      "kind": "end"
                  },
                  "token": "1"
              },
              "method": "$/progress"
          }
      }

Last message varies, f.e. finish: kick (took 0.00s)

[jneira]

haskell/ghcide#836 has introduced another test ignored in windows 😟 so the list is

• cpp/cpp-error for all ghc versions
  https://github.com/haskell/ghcide/blob/d874a32cdffe5db34d034208a17eb865984575c6/test/exe/Main.hs#L2338
• plugin tests (2) for ghc >= 8.8
  https://github.com/haskell/ghcide/blob/d874a32cdffe5db34d034208a17eb865984575c6/test/exe/Main.hs#L2299
  https://github.com/haskell/ghcide/blob/d874a32cdffe5db34d034208a17eb865984575c6/test/exe/Main.hs#L2322
• add missing module (non workspace) for all ghc version
  https://github.com/haskell/ghcide/blob/d874a32cdffe5db34d034208a17eb865984575c6/test/exe/Main.hs#L287

Transient errors (all terminate with lsp session timeouts):

• ghcide restarts shake session on config changes for ghc-8.10
• TemplateHaskell/reloading-th-test for ghc-8.8
• addDependentFile/file-changed for ghc-8.8