travis-script.sh: test bootstrapping

[ghost]

No description provided.

[ghost]

Works as expected, meaning fails right now (see #3344).

[23Skidoo]

I'm a bit worried about how this will affect compile times on Travis.

[23Skidoo]

@Tuncer Can you rebase this against master and do push -f? I want to evaluate the impact of this PR on compile times before merging.

[ezyang]

@23Skidoo If we replace the code which uses cabal-install to build cabal-install, with code that uses the bootstrap script, I think that the time to build should be comparable.

[ghost]

Rebased, which also incorporates the fixed bootstrap.sh, so it should work now.

[ghost]

@ezyang, I've thought about replacing bits in there as well, but I wasn't sure which parts are okay to remove. Is it a better idea for you to make those changes in a follow-up commit?

[ezyang]

Yes, assuming @23Skidoo is OK with the Travis time we can defer it.

[ghost]

Extra commit that moves cabal update and optional cabal install parsec to a later stage, so that we don't run update twice and also use the freshly bootstrapped cabal-install.

[23Skidoo]

This is now blocked by #3345.

[23Skidoo]

It looks like #3356 needs to be merged to make the build bot error go away.

[23Skidoo]

So the bootstrap script also needs to check that a recent version of bytestring is installed (and install it if it's not).

[ghost]

Added bytestring, and while at it, fixed a tiny whitespace issue separately.

[23Skidoo]

Still fails, bytestring has to be installed before binary.

[ghost]

Fixed order.

[ghost]

Adjusted bytestring regex to require >=0.10.2.

[ghost]

Changed bytestring version that gets fetched to 0.10.6.0, in order to relax its base vsn bound. The previous error was that bytestring >=0.10.2.0 is needed, and 0.10.6.0 is the last bytestring release before the lower bound is bumped.

[ghost]

First, binary-0.8.3.0 complained that it needs bytestring >=0.10.2:

Configuring binary-0.8.3.0...
Setup: Encountered missing dependencies:
bytestring >=0.10.2

To fix that, I've tried giving it 0.10.8.0, but that failed because GHC 7.6.3's base is too old. Therefore, I've changed it to 0.10.6.0, since that's the last version before the base lower bound was bumped, and we cannot go older than 0.10.2.0 anyway.

Now we have a new error:

Configuring cabal-install-1.25.0.0...
Warning: This package indirectly depends on multiple versions of the same
package. This is highly likely to cause a compile failure.
package Cabal-1.25.0.0 requires binary-0.5.1.1
package text-1.2.2.1 requires binary-0.8.3.0
package cabal-install-1.25.0.0 requires binary-0.8.3.0
package unix-2.6.0.1 requires bytestring-0.10.0.2
package binary-0.5.1.1 requires bytestring-0.10.0.2
package Cabal-1.25.0.0 requires bytestring-0.10.0.2
package zlib-0.6.1.1 requires bytestring-0.10.6.0
package text-1.2.2.1 requires bytestring-0.10.6.0
package tar-0.5.0.3 requires bytestring-0.10.6.0
package parsec-3.1.9 requires bytestring-0.10.6.0
package network-2.6.2.1 requires bytestring-0.10.6.0
package hashable-1.2.4.0 requires bytestring-0.10.6.0
package hackage-security-0.5.1.0 requires bytestring-0.10.6.0
package ed25519-0.0.5.0 requires bytestring-0.10.6.0
package cryptohash-sha256-0.11.7.1 requires bytestring-0.10.6.0
package cabal-install-1.25.0.0 requires bytestring-0.10.6.0
package binary-0.8.3.0 requires bytestring-0.10.6.0
package base64-bytestring-1.0.0.1 requires bytestring-0.10.6.0
package base16-bytestring-0.1.1.6 requires bytestring-0.10.6.0
package HTTP-4000.3.3 requires bytestring-0.10.6.0
Building cabal-install-1.25.0.0...
[...]
Distribution/Client/World.hs:103:33:
    Couldn't match type `B.ByteString'
                  with `bytestring-0.10.0.2:Data.ByteString.Lazy.Internal.ByteString'
    Expected type: [Char]
                   -> bytestring-0.10.0.2:Data.ByteString.Lazy.Internal.ByteString
      Actual type: [Char] -> B.ByteString
    In the second argument of `(.)', namely `B.pack'
    In the expression: writeFileAtomic world . B.pack
    In a stmt of a 'do' block:
      writeFileAtomic world . B.pack
      $ unlines [(display pkg) | pkg <- pkgsNewWorld]

What should we do?

[hvr]

@Tuncer fwiw, bytestring-0.10.8.1 will support GHC 7.6 again

[ghost]

@hvr, thanks.

@23Skidoo, what happened to the new hackage-security release? Is it ready, and would it help us avoid adding bytestring and also allow us to remove the other temporary packages?

[23Skidoo]

I think what we can try doing to fix the error is check the version of bytestring installed and then install an older version of binary if bytestring is < 0.10.2 (binary-0.8.2.1 should work).

[ghost]

@hvr, now fetching bytestring 0.10.8.1.

[ghost]

@23Skidoo, can you file a ticket to remove the extra packages once the new hackage-security release can be integrated in bootstrap.sh?

[23Skidoo]

I'm probably missing something. The failure on 7.6 and 7.4 is due to us having installed two versions of bytestring, so I'm not sure how using a newer hackage-security will help here. I believe that the way to fix this is to use the bytestring that comes with GHC instead (which means using binary-0.8.2.1 on 7.4 and 7.6).

[ghost]

Some of the deps were meant to be intermittent until the new hackage-security package is out, so assuming we have a new hackage-security, I'd prefer to first trim deps and then fix any issues left.

[23Skidoo]

Which ones? If you're referring to the dependency on cryptohash, it has already been removed in favour of cryptohash-sha256. That happened in hackage-security 0.5.1.0

[ghost]

That explains why I haven't been able to find it.

[ghost]

I'll modify bootstrap.sh accordingly.

[23Skidoo]

I think we can remove all mentions of bytestring from the bootstrap script if we add a similar conditional for binary instead. So GHC <= 7.6 will use binary-0.8.2.1 (which depends on bytestring >= 0.9 and therefore should work with the versions of bytestring that come with GHC 7.6 and 7.4) and GHC > 7.6 will use binary-0.8.3.0.

[ghost]

Added a commit, please let me know if that's what you had in mind.

[23Skidoo]

If you remove this conditional and all mentions of bytestring from the script, I think that just conditionally defining BINARY_VER and BINARY_VER_REGEXP (like you do here for bytestring) will work.

[ghost]

Ah, right, now I got what you're saying.

[23Skidoo]

And now it fails because tar requires bytestring-builder when compiled against bytesting < 0.10. So we'll also need to conditionally install bytestring-builder depending on the version bytestring.

[ghost]

Okay, added another commit.

[23Skidoo]

Merged: 2645970.

[23Skidoo]

@Tuncer Thanks for your patience!

[23Skidoo]

One other thing we can do is to patch binary to support bytestring-0.9 via bytestring-builder (like tar does, see haskell/tar#14). That'd allow us to always install the latest version of binary on all supported versions of GHC.