git-annex: force-enable "webapp" flag #47950

csstaub · 2016-01-11T05:23:17Z

To address issue #47346 (git-annex-webapp missing), we explicitly enable the "webapp" flag for git-annex. This should prevent git-annex from being built without the webapp and make the build fail if there is e.g. a dependency issue.

In order to set the build flags to git-annex correctly, we must (1) pass them to cabal install to resolve/install all the necessary dependencies correctly, and (2) pass them to cabal configure to make sure git-annex compiles with the flags we want.

I originally wanted to enable S3 here too, but as noted in my comment on #47737 this would currently break the build because of a dependency/version conflict with aws 0.13.0. After a new version of aws is released on Hackage, it should be possible to add "S3" to :flags to re-enable and avoid accidentally dropping S3 support in the future.

csstaub · 2016-01-11T05:27:55Z

Checking flags and running brew test after build:

~ $ /usr/local/Cellar/git-annex/5.20151218/bin/git-annex version
git-annex version: 5.20151218
build flags: Assistant Webapp Webapp-secure Pairing Testsuite WebDAV FsEvents XMPP ConcurrentOutput DNS Feeds Quvi TDFA TorrentParser Database
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
remote types: git gcrypt bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
~ $ brew test git-annex
Testing git-annex
==> Using the sandbox
==> git annex test
~ $ echo $?
0

MikeMcQuaid · 2016-01-11T08:04:20Z

Library/Formula/git-annex.rb

@@ -26,7 +26,7 @@ class GitAnnex < Formula
  depends_on "quvi"

  def install
-    install_cabal_package :using => ["alex", "happy", "c2hs"] do
+    install_cabal_package({ :using => ["alex", "happy", "c2hs"], :flags => ["Webapp"] }) do


Are the ({ definitely needed? No worries if so.

csstaub · 2016-01-12T04:46:44Z

@MikeMcQuaid made changes based on your feedback, please take a look.

MikeMcQuaid · 2016-01-17T21:48:18Z

Library/Homebrew/language/haskell.rb

+            flags = "--flags='#{options[:flags].join(" ")}'"
+          end
+
+          args_and_flags = [*args, flags]


*args needs to be last on Ruby 1.8. You could build the array in two steps instead.

MikeMcQuaid · 2016-01-17T21:48:34Z

👍 to the concept once this is green.

MikeMcQuaid · 2016-02-02T09:23:53Z

@BrewTestBot test this please

DomT4 · 2016-02-13T03:08:06Z

@BrewTestBot test this please

DomT4 · 2016-02-13T03:42:52Z

Failing to build on all three platforms I'm afraid.

csstaub · 2016-02-13T04:05:43Z

Looks like it kills the build due to taking too much time? Haskell is rather slow to compile.

DomT4 · 2016-02-13T04:16:34Z

Jenkins rather than Travis: http://bot.brew.sh/job/Homebrew%20Pull%20Requests/41763/version=el_capitan/console

==> cabal install --jobs=4 --only-dependencies --flags='Webapp'
clang: warning: -Wl,-headerpad_max_install_names: 'linker' input unused
clang: warning: argument unused during compilation: '-L/usr/local/lib'
clang: warning: argument unused during compilation: '-L/System/Library/Frameworks/OpenGL.framework/Versions/Current/Libraries'
Resolving dependencies...
cabal: Could not resolve dependencies:
next goal: git-annex (user goal)
rejecting: git-annex-6.20160211 (global constraint requires ==6.20160126)
trying: git-annex-6.20160126
trying: base-4.8.2.0/installed-bfc... (dependency of git-annex-6.20160126)
next goal: persistent (dependency of git-annex-6.20160126)
rejecting: persistent-2.2.4, 2.2.3, 2.2.2.1, 2.2.2, 2.2.1, 2.2, 2.1.6, 2.1.5,
2.1.4, 2.1.3, 2.1.2, 2.1.1.7, 2.1.1.6, 2.1.1.5, 2.1.1.4, 2.1.1.3, 2.1.1.2,
2.1.1.1, 2.1.1, 2.1.0.2, 2.1.0.1, 2.1 (conflict: persistent-template =>
persistent>=1.3 && <1.4)
rejecting: persistent-1.3.3, 1.3.2, 1.3.1.1, 1.3.1, 1.3.0.6, 1.3.0.5, 1.3.0.4
(conflict: base==4.8.2.0/installed-bfc..., persistent => base>=4.5 && <4.8)
trying: persistent-1.3.0.3
next goal: esqueleto (dependency of git-annex-6.20160126)
rejecting: esqueleto-2.4.3, 2.4.2, 2.4.1, 2.4.0, 2.3.0, 2.2.12, 2.2.11,
2.2.10, 2.2.9, 2.2.8, 2.2.7, 2.2.6, 2.2.5 (conflict: persistent==1.3.0.3,
esqueleto => persistent>=2.1.1.7 && <2.3)
rejecting: esqueleto-2.2.4, 2.2.3, 2.2.2, 2.2.1, 2.2, 2.1.3 (conflict:
persistent==1.3.0.3, esqueleto => persistent>=2.1.1.7 && <2.2)
rejecting: esqueleto-2.1.2.3 (conflict: persistent==1.3.0.3, esqueleto =>
persistent>=2.1 && <2.2)
rejecting: esqueleto-2.1.2.2, 2.1.2.1, 2.1.2, 2.1.1, 2.1.0, 2.0.3, 2.0.2,
2.0.1, 2.0.0, 1.4.4, 1.4.3, 1.4.2, 1.4.1.3, 1.4.1.2, 1.4.1, 1.4, 1.3.12,
1.3.11, 1.3.10, 1.3.9, 1.3.8, 1.3.7, 1.3.6, 1.3.5, 1.3.4.6 (conflict:
base==4.8.2.0/installed-bfc..., esqueleto => base>=4.5 && <4.8)
rejecting: esqueleto-1.3.4.5, 1.3.4.4, 1.3.4.3, 1.3.4.2, 1.3.4, 1.3.3, 1.3.2,
1.3.1, 1.3, 1.2.4, 1.2.3, 1.2.2.1, 1.2.2, 1.2.1, 1.2, 1.0.8, 1.0.7, 1.0.6,
1.0.5, 1.0.3, 1.0.2, 1.0.1, 1.0, 0.2.9, 0.2.8, 0.2.7 (conflict:
base==4.8.2.0/installed-bfc..., esqueleto => base>=4.5 && <4.7)
rejecting: esqueleto-0.2.6, 0.2.5, 0.2.4, 0.2.3, 0.2.2, 0.2.1, 0.2, 0.1
(conflict: base==4.8.2.0/installed-bfc..., esqueleto => base==4.5.*)
rejecting: esqueleto-1.4.1.1 (conflict: base==4.8.2.0/installed-bfc...,
esqueleto => base>=4.5 && <4.8)
Backjump limit reached (change with --max-backjumps).

Note: when using a sandbox, all packages are required to have consistent
dependencies. Try reinstalling/unregistering the offending packages or
recreating the sandbox.

csstaub · 2016-02-13T05:19:38Z

Ugh, more version conflicts. This overly strict versioning is killing me 😐

MikeMcQuaid · 2016-02-13T14:09:56Z

I'm wondering if we shouldn't remove this from Homebrew. It has serious stability issues.

ilovezfs · 2016-02-14T07:05:24Z

@MikeMcQuaid Anything in particular you're thinking of? The interminable test suite passed with flying colors last time I ran it on 10.11.2 and 10.11.3:
https://gist.github.com/ilovezfs/0cee6dc016b6b624c734
https://gist.github.com/ilovezfs/88a2657f996ae0ba9beb

MikeMcQuaid · 2016-02-14T08:36:13Z

There's pretty regular build failures.

ilovezfs · 2016-02-14T09:09:16Z

@MikeMcQuaid I think it's only failing CI regularly due to timeouts where the build servers are deciding it's taking too long.

ilovezfs · 2016-02-14T09:36:39Z

@MikeMcQuaid The current build failure in this PR is due to a problem with Library/Homebrew/language/haskell.rb. Setting max-backjumps=-1 as described here http://stackoverflow.com/questions/29939644/backjump-limit-reached-change-with-max-backjumps fixed the issue:

==> cabal install --jobs=5 --max-backjumps=-1 --only-dependencies
clang: warning: -Wl,-headerpad_max_install_names: 'linker' input unused
clang: warning: argument unused during compilation: '-L/usr/local/lib'
clang: warning: argument unused during compilation: '-L/System/Library/Frameworks/OpenGL.framework/Versions/Current/Libraries'
Resolving dependencies...
Notice: installing into a sandbox located at
/private/tmp/git-annex20160214-6733-17xs10o/git-annex-6.20160126/.cabal-sandbox
Downloading HUnit-1.3.1.1...
Downloading ansi-terminal-0.6.2.3...
Configuring dlist-0.7.1.2...
Configuring mtl-2.2.1...
Configuring primitive-0.6.1.0...
Configuring random-1.1...
Configuring syb-0.6...
Downloading appar-0.1.4...
Downloading auto-update-0.1.3...
etc.

Brew probably either needs to allow max-backjumps to be set by the formula (adding :max-backjumps to the options hash that currently only has the :using key) or needs to set max-backjumps=-1 always. This is the diff that fixed it:

diff --git a/Library/Homebrew/language/haskell.rb b/Library/Homebrew/language/haskell.rb
index dd9bc5b..adf2f47 100644
--- a/Library/Homebrew/language/haskell.rb
+++ b/Library/Homebrew/language/haskell.rb
@@ -45,7 +45,7 @@ module Languag
       end

       def cabal_install(*args)
-        system "cabal", "install", "--jobs=#{ENV.make_jobs}", *args
+        system "cabal", "install", "--jobs=#{ENV.make_jobs}", "--max-backjumps=-1", *args
       end

       def cabal_install_tools(*tools)
Josephs-MacBook-Pro:local joe$

Update: And it finished.

==> Summary
🍺  /usr/local/Cellar/git-annex/6.20160126: 101 files, 80.3M, built in 28 minutes 42 seconds

Full log:
https://gist.github.com/ilovezfs/e25ac2488a0ddd4e095c

Note that the resolve dependencies step that caused the build failure took about 3.5 minutes to succeed with max-backjumps=-1.

And here's a PR with that change: #49158

And build succeeded on 10.11.2 with #47950 merged too: https://gist.github.com/ilovezfs/38f49e20ff41ae892658

The default max-backjumps can be too low, especially since Language::Haskell::Caball doesn't guarantee an LTS config. In particular, this fixes a git-annex build failure in Homebrew#47950

The default max-backjumps can be too low, especially since Language::Haskell::Cabal doesn't guarantee an LTS config. In particular, this fixes a git-annex build failure in Homebrew#47950

The default max-backjumps can be too low, especially since Language::Haskell::Cabal doesn't guarantee an LTS config. In particular, this fixes a git-annex build failure in #47950 Closes #49158. Signed-off-by: Mike McQuaid <[email protected]>

ilovezfs · 2016-02-21T19:03:52Z

@csstaub FYI, #49158 is now merged, so you should be able to build again, and continue with this PR where you left off, after running brew update of course.

To address issue #47346 (git-annex-webapp missing), we explicitly enable the "webapp" flag for git-annex. This should prevent git-annex from being built without the webapp and make the build fail if there is e.g. a dependency issue.

csstaub · 2016-02-21T19:09:17Z

@ilovezfs awesome, thanks! rebased on latest master, let's see if this works.

ilovezfs · 2016-02-21T19:11:08Z

@csstaub It should now build locally for you, but the build servers will probably still fail it because it takes longer to build than the configured max.

csstaub · 2016-02-21T19:12:40Z

Yeah, it will time out on travis-ci for sure, but afaik there's also the jenkins test bot where it should be allowed to take more time.

DomT4 · 2016-02-21T19:41:55Z

If you run into the current Jenkins timeout I'll eat my hat 😃.

csstaub · 2016-02-21T22:32:45Z

@MikeMcQuaid @DomT4 looks like it's green now (at least on jenkins)

DomT4 · 2016-02-22T01:36:09Z

Merged in 9e3ee3e. Thank you for your contribution to Homebrew @csstaub; we appreciate it! 😺

Fixes a regression introduced by Homebrew#47950 in 9e3ee3e causing build failures with the error cabal: The file does not exist ''. This will occur whenever the optional :flags key isn't in the options hash passed to install_cabal_package.

Fixes a regression introduced by #47950 in 9e3ee3e causing build failures with the error cabal: The file does not exist ''. This will occur whenever the optional :flags key isn't in the options hash passed to install_cabal_package. Closes #49425. Signed-off-by: Mike McQuaid <[email protected]>

MikeMcQuaid reviewed Jan 11, 2016
View reviewed changes

MikeMcQuaid reviewed Jan 17, 2016
View reviewed changes

git-annex: force-enable "webapp" flag

7d69f5b

To address issue #47346 (git-annex-webapp missing), we explicitly enable the "webapp" flag for git-annex. This should prevent git-annex from being built without the webapp and make the build fail if there is e.g. a dependency issue.

DomT4 closed this in 9e3ee3e Feb 22, 2016

ilovezfs mentioned this pull request Feb 22, 2016

haskell: fix empty flags build failures #49425

Closed

ilovezfs mentioned this pull request Apr 18, 2016

git-annex 6.20160412 Homebrew/homebrew-core#329

Closed

Homebrew locked and limited conversation to collaborators Jul 10, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

git-annex: force-enable "webapp" flag #47950

git-annex: force-enable "webapp" flag #47950

csstaub commented Jan 11, 2016

csstaub commented Jan 11, 2016

MikeMcQuaid Jan 11, 2016

csstaub commented Jan 12, 2016

MikeMcQuaid Jan 17, 2016

MikeMcQuaid commented Jan 17, 2016

MikeMcQuaid commented Feb 2, 2016

DomT4 commented Feb 13, 2016

DomT4 commented Feb 13, 2016

csstaub commented Feb 13, 2016

DomT4 commented Feb 13, 2016

csstaub commented Feb 13, 2016

MikeMcQuaid commented Feb 13, 2016

ilovezfs commented Feb 14, 2016

MikeMcQuaid commented Feb 14, 2016

ilovezfs commented Feb 14, 2016

ilovezfs commented Feb 14, 2016

ilovezfs commented Feb 21, 2016

csstaub commented Feb 21, 2016

ilovezfs commented Feb 21, 2016

csstaub commented Feb 21, 2016

DomT4 commented Feb 21, 2016

csstaub commented Feb 21, 2016

DomT4 commented Feb 22, 2016

git-annex: force-enable "webapp" flag #47950

git-annex: force-enable "webapp" flag #47950

Conversation

csstaub commented Jan 11, 2016

csstaub commented Jan 11, 2016

MikeMcQuaid Jan 11, 2016

Choose a reason for hiding this comment

csstaub commented Jan 12, 2016

MikeMcQuaid Jan 17, 2016

Choose a reason for hiding this comment

MikeMcQuaid commented Jan 17, 2016

MikeMcQuaid commented Feb 2, 2016

DomT4 commented Feb 13, 2016

DomT4 commented Feb 13, 2016

csstaub commented Feb 13, 2016

DomT4 commented Feb 13, 2016

csstaub commented Feb 13, 2016

MikeMcQuaid commented Feb 13, 2016

ilovezfs commented Feb 14, 2016

MikeMcQuaid commented Feb 14, 2016

ilovezfs commented Feb 14, 2016

ilovezfs commented Feb 14, 2016

ilovezfs commented Feb 21, 2016

csstaub commented Feb 21, 2016

ilovezfs commented Feb 21, 2016

csstaub commented Feb 21, 2016

DomT4 commented Feb 21, 2016

csstaub commented Feb 21, 2016

DomT4 commented Feb 22, 2016