Caching git submodules

[staticfloat]

@samueljohn, I'm pinging you on this to get your input. I'm going through and cleaning up this formula, and I thought it would be neat to do the same trick with the submodules Julia uses (libuv, nginx, etc....) as you did with the downloaded copy of LAPACK in OpenBLAS; I want to make mini-formulae out of them and have Homebrew control the downloading of them rather than Julia, so that we don't have to re-download the entire git repo every time we update.

Do you know if this is possible/easy? If so, could you point me toward how to actually do it? Thanks! I'll incorporate it into this refresh of the formula, which should also address #15, allow us to build both 32-bit and 64-bit versions of Julia, and all her dependencies, and be able to choose Accelerate-backed Julia or OpenBLAS-backed Julia!

[samueljohn]

Hi @staticfloat!

I want to make mini-formulae out of them and have Homebrew control the downloading of them rather than Julia, so that we don't have to re-download the entire git repo every time we update.

That sounds very good. You do a great job with the Julia formula 👍

Do you know if this is possible/easy?

The most difficult part is to tell Julia's Makesystem how to use the libuv and nginx etc.
Most common is to give an flag like --with-uv=DIR to configure (or make), so that Julia uses that one. I don't know what Julia supports here. Is there a ./configure --help? One could easily install libuv into the prefix of julia in some subfolder, so it does not mess up homebrew when Julia gets linked.

If you need the downloaded tarball, then the openblas formula shows how to do that (I think for a git repo this works, too and points to the temp dir of the checked out sources).

Why can't we tell Julia to use the libuv and nginx from main homebrew? I'd love to support you on that one and I will certainly try merge openblas and openblas64 into one formula (where the 64bit interface are named differently).
What do we need to change in the libuv and nginx formulae to make Julia happy? The homebrew way would really be to make use of those dependencies already there. At some point, I hope, we can get a julia formula in homebrew-science :-)

[staticfloat]

For reference, it seems that I will be able to address this with Homebrew/legacy-homebrew#17783

[jacknagel]

How much granularity do you need here? Is it all-or-nothing or are there some submodules that you want checked out?

[staticfloat]

All or nothing is what I was planning on. I would disable auto-submodules
in Homebrew, then manually patch .gitmodules and call git submodule init
after that.

More granularity would be difficult, I think.

-E
On Feb 13, 2013 12:30 PM, "Jack Nagel" [email protected] wrote:

How much granularity do you need here? Is it all-or-nothing or are there
some submodules that you want copied?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/20#issuecomment-13516377.

[jacknagel]

Okay, good. Yeah, anything more granular is hard to do, the submodule stuff isn't that flexible.

[jacknagel]

I merged my patches.

This diff against julia.rb will disable submodules:

diff --git a/julia.rb b/julia.rb
index 3b8de96..81bdf03 100755
--- a/julia.rb
+++ b/julia.rb
@@ -1,5 +1,8 @@
 require 'formula'

+class GitNoSubmoduleDownloadStrategy < GitDownloadStrategy
+  def submodules?; false; end
+end

 # Avoid Julia downloading these tools ondemand
 # We don't have full formulae for them, as julia makes very specific use of these formulae
@@ -14,7 +17,7 @@ class JuliaDSFMT < Formula

 class Julia < Formula
   homepage 'http://julialang.org'
-  head 'https://github.com/JuliaLang/julia.git'
+  head 'https://github.com/JuliaLang/julia.git', :using => GitNoSubmoduleDownloadStrategy

   depends_on "readline"
   depends_on "pcre"

[staticfloat]

@samueljohn upon further thought, I'm not sure that caching submodules is really what I want to do here. Caching submodules wouldn't achieve much as two of the submodules (libuv and juliadoc) are rather quick-moving, and furthermore Julia always expects the latest, so downloading it on demand is really what we want. Finally, since brew keeps the old Julia git repo around in-between installs, removing and reinstalling Julia is pretty much the same as just doing a git pull, a git submodule update and recompiling. I think I'm going to leave this formula as-is for now, if you think otherwise, comment here!