Compiling ouroboros-consensus on Windows

[coot]

This might be just specific to my setup (I am using git-bash command line on Windows10)

Failed to build cryptonite-openssl-0.7. The failure occurred during the
configure step.
Build log (
C:\Users\mszam\AppData\Roaming\cabal\logs\ghc-8.6.5\cryptonite-op_-0.7-b4e2df2f3fb6950220cf7ba4da569
0ca0ffa9962.log
):
Configuring library for cryptonite-openssl-0.7..
cabal.exe: Missing dependencies on foreign libraries:
* Missing (or bad) C libraries: eay32, ssl32
This problem can usually be solved by installing the system packages that
provide these libraries (you may need the "-dev" versions). If the libraries
are already installed but in a non-standard location then you can use the
flags --extra-include-dirs= and --extra-lib-dirs= to specify where they are.If
the library files do exist, it may contain errors that are caught by the C
compiler at the preprocessing stage. In this case you can re-run configure
with the verbosity flag -v3 to see the error messages.

cabal.exe: Failed to build cryptonite-openssl-0.7 (which is required by
test:test-consensus from ouroboros-consensus-0.1.0.0, exe:byron-db-converter
from ouroboros-consensus-0.1.0.0 and others). See the build log above for
details.

[kderme]

@coot do you have OpenSSL for Windows? I downloaded mine from here http://slproweb.com/products/Win32OpenSSL.html.
(I use msys2 and I can build locally).

[mrBliss]

@dcoutts did we get rid of the dependency on cryptonite-openssl-0.7 already? I vaguely remember that you had problems with it when trying to compile on Raspberry Pi.

[mrBliss]

See #285.

1.1.0 because the exchanges are not using Windows.

[mrBliss]

The scrypt dependency in cardano-ledger was recently removed: IntersectMBO/cardano-ledger#675

@kderme @coot Can you check again whether ouroboros-consensus compiles on Windows? (Not mingw32, but real Windows)

[mrBliss]

@kderme @coot What's the status on this?

[coot]

On my machine it fails with:

Failed to build cryptonite-openssl-0.7. The failure occurred during the
configure step.
Build log (
C:\Users\IEUser\AppData\Roaming\cabal\logs\ghc-8.6.5\cryptonite-op_-0.7-a1533bf39b55912fd598a886105fb971078cd843.log
):
Configuring library for cryptonite-openssl-0.7..
cabal.exe: Missing dependencies on foreign libraries:
* Missing (or bad) C libraries: eay32, ssl32

Let me check if I can get them with chocolatey.

[coot]

@vincenthz building cryptonite-openssl fails on Windows. It seems that openssl-1.1 does not come with this libraries any more. There is also this old thread which mentions that eay32 and ssl32 where removed / renamed as openssl. Do you plan to update cryptonite-openssl or should we install older version of openssl?

[coot]

The agreed way forward is to remove dependency on openssl: IntersectMBO/cardano-crypto#62

[kderme]

The agreed way forward is to remove dependency on openssl: input-output-hk/cardano-crypto#62

That's good news, especially since a cabal issue haskell/cabal#2997 didn't allow to easily include the external openssl libraries.

Appart from that, I get an error because in windows a very old unix-compat version is picked.

System\PosixCompat\Files.hsc:86:1: error:
    Could not find module `GHC.Handle'
    Perhaps you meant GHC.IO.Handle (from base-4.12.0.0)
    Use -v to see a list of the files searched for.
   |
86 | import GHC.Handle (fdToHandle)
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
cabal.exe: Failed to build unix-compat-0.1.2.1 (which is required by
exe:byron-db-converter from ouroboros-consensus-0.1.0.0). See the build log
above for details.

Adding a constraint unix-compat >= 0.2.0 in cabal.project fixes it. But I'm not quite sure if this is a general issue or there is something wrong with my setup. Do you also get this error?

EDIT: The above is actually not a real issue.

[mrBliss]

Thanks @vhulchenko-iohk! @kderme and/or @coot can you propagate this change?

[coot]

That's great, I can take care of towards end of the week. After propagating it and merging #1499 whole ouroboros-network will compile on windows, hopefully we will be able to test cardano-node on windows.

[mrBliss]

That's great, I can take care of towards end of the week. After propagating it and merging #1499 whole ouroboros-network will compile on windows, hopefully we will be able to test cardano-node on windows.

@coot What's the status of this?

[coot]

I did had time yet to look do it. We first need to finish the byron tickets and then I'll return to work Windows issues.

[kderme]

cardano-ledger has another dependency to openssl https://github.com/input-output-hk/cardano-ledger/blob/master/crypto/cardano-crypto-wrapper.cabal. I'll see if we can drop it.

[mrBliss]

@coot When do you think we can close this issue? After #1774 or is there more to do?

[coot]

ouroboros-netowrk compiles on Windows, so it's only to fix ouroboros-consensus issues.

There is this in cardano-node which will help us track and resolve all the windows compilation issues.

[dcoutts]

Lets also make windows required for the CI in this repo, just like in IntersectMBO/cardano-node#669.

[kderme]

ouroboros-netowrk compiles on Windows, so it's only to fix ouroboros-consensus issues.

@coot are there any known issues for ouroboros-consensus (apart from stack related)? Are there still plans to use Win32-2.6.1.0?

[mrBliss]

When #1808 is merged, we should try to get consensus included in there too.

[dcoutts]

So we know it works overall, but we need CI to stop regressions. This should be enough once it passes and is merged.

#1814

[oneEdoubleD]

I tried to build cardano-node on Windows 10 and got the following error:

 [ 21 of 120] Compiling Ouroboros.Consensus.Storage.IO ( src-win32\Ouroboros\Consensus\Storage\IO.hs, .stack-work\dist\e626a42b\build\Ouroboros\Consensus\Storage\IO.o )
    src-win32\Ouroboros\Consensus\Storage\IO.hs:47:30: error:
        Variable not in scope:
          setFilePointerEx :: HANDLE -> Integer -> FilePtrDirection -> IO a4
       |
    47 |       AppendMode{} -> void $ setFilePointerEx h 0 fILE_END
       |                              ^^^^^^^^^^^^^^^^
    src-win32\Ouroboros\Consensus\Storage\IO.hs:65:3: error:
        Variable not in scope:
          setFilePointerEx :: Ptr () -> Int64 -> FilePtrDirection -> IO a3
       |
    65 |   setFilePointerEx h size (fromSeekMode seekMode)
       |   ^^^^^^^^^^^^^^^^
    src-win32\Ouroboros\Consensus\Storage\IO.hs:78:26: error:
        Variable not in scope:
          setFilePointerEx
            :: HANDLE -> Integer -> FilePtrDirection -> IO Int64
       |
    78 | getCurrentFileOffset h = setFilePointerEx h 0 fILE_CURRENT
       |                          ^^^^^^^^^^^^^^^^
    src-win32\Ouroboros\Consensus\Storage\IO.hs:84:10: error:
        Variable not in scope:
          setFilePointerEx :: HANDLE -> Integer -> FilePtrDirection -> IO a0
       |
    84 |     _ <- setFilePointerEx h (fromIntegral pos) fILE_BEGIN
       |          ^^^^^^^^^^^^^^^^
    src-win32\Ouroboros\Consensus\Storage\IO.hs:86:10: error:
        Variable not in scope:
          setFilePointerEx :: HANDLE -> Int64 -> FilePtrDirection -> IO a1
       |
    86 |     _ <- setFilePointerEx h initialOffset fILE_BEGIN
       |          ^^^^^^^^^^^^^^^^
    src-win32\Ouroboros\Consensus\Storage\IO.hs:93:8: error:
        Variable not in scope:
          setFilePointerEx :: HANDLE -> Integer -> FilePtrDirection -> IO a2
       |
    93 |   _ <- setFilePointerEx h (fromIntegral size) (fromSeekMode AbsoluteSeek)
       |

[mrBliss]

Related thread on Slack: https://input-output-rnd.slack.com/archives/C819S481Y/p1584555182036500

[mrBliss]

The problem @oneEdoubleD encountered is only a problem in cardano-node and has to be fixed there. It's caused by Stack's allow-newer: true setting. See IntersectMBO/cardano-node#706 (which is closed, but a proper fix is on the way).

Moreover, as of #1835, CI builds and tests on Windows using Wine (mingw32), which gives us some confidence consensus works on Windows. cardano-node, which uses consensus, has been reported to run successfully on Windows.

I think the only thing left to do is: let CI build and test on Windows (real Windows, not using Wine). @coot has a PR open for this (#1808), when that's merged, we can simply add the consensus tests to it. At that point we'll notice any remaining Windows problems and it will also make sure we don't regress.

[dcoutts]

Can we move discussion of this additional requirement "test on real windows as part of CI" into a new ticket please.

This ticket is "Compiling ouroboros-consensus on Windows". We are already doing that. As of PR #1835 it is also now a required build target in CI. We even run most tests on wine (a bonus).

[mrBliss]

Closing, remaining CI work moved to IntersectMBO/ouroboros-consensus#697.

[dcoutts]

"test on real windows as part of CI" is fixed by #1808 now right?

[mrBliss]

"test on real windows as part of CI" is fixed by #1808 now right?

No, @coot only enabled the network tests, not the consensus tests. I'll try adding them to the workflow today.