fetchGit and submodules

[roberth]

I love the support for 'normal' GitHub repo's, but niv in its current implementation can't include the submodules, which makes it useless in some cases.
This can be solved by adding support for fetchGit.

[nmattia]

Definitely. Whenever we merge #35 (cough cough Soon ™) I'll most likely deprecate the url_template fetching for github repos and use fetchGit/fetchFromGitHub instead.

[PierreR]

@nmattia I guess the resolution of this issue would solve my issue:

unpacking source archive /build/v3.7.2.tar.gz
error: zip file must contain a single file or directory.
hint: Pass stripRoot=false; to fetchzip to assume flat list of files.

I am trying to replace a call to pkgs.fetchgit with niv. Here is my source.json:

    "puppet-unit-test": {
        "sha256": "0hyvh2niq939rxhs7wx6szmvrs7s6nrn3sxf1rma5xl41aw3b068",
        "type": "tarball",
        "url": "http://bitbucket.cirb.lan/CICD/puppet-unit-test/archive/v3.7.2.tar.gz",
        "url_template": "http://bitbucket.cirb.lan/CICD/puppet-unit-test/archive/v3.7.2.tar.gz",
        "version": "3.7.2"

[nmattia]

@PierreR in this case you can also use type = "file" and do the unpacking yourself. But yes, having fetchgit and the bitbucket git@... uri should fix this.

[nmattia]

Forgot to update this! There's experimental support for fetchGit:

$ niv add git [email protected]:nmattia/niv -a rev=deadbeef

[andir]

How would I use fetchSubmodules = true with this?

[Nadrieril]

This is not directly supported, but you can make it work as follows.
First you need to use an unstable version of nix, so that fetchGit accepts a submodules argument (it's not called fetchSubmodules anymore). Then change sources.nix to pass the submodules argument to fetchGit:

diff --git a/nix/sources.nix b/nix/sources.nix                                                                                                     
index 8a725cb..800b0f8 100644                                                                                                                      
--- a/nix/sources.nix                                                                                                                              
+++ b/nix/sources.nix                                                                                                                              
@@ -19,7 +19,7 @@ let                                                                                                                              
       pkgs.fetchzip { inherit (spec) url sha256; };                                                                                               
                                                                                                                                                   
   fetch_git = spec:                                                                                                                               
-    builtins.fetchGit { url = spec.repo; inherit (spec) rev ref; };                                                                               
+    builtins.fetchGit { url = spec.repo; inherit (spec) rev ref submodules; };
  
   fetch_builtin-tarball = spec:
     builtins.trace

Finally, add "submodules": true in a sources.json entry that has "type": "git". Example:

    "dhall-haskell": {
        "ref": "master",
        "repo": "https://github.com/dhall-lang/dhall-haskell",
        "rev": "9afffaea7108a86df197cb97b64a9504fe37ecfc",
        "submodules": true,
        "type": "git"
    },

[shajra]

@nmattia I just realized how long-standing this issue has been. I've personally run into a second time now (submodules aren't so rare to never see them... but they aren't the common case). What are your latest thoughts on how best to solve it? Is there anything I can do to help push it over the finish line?

[nmattia]

hey @shajra , I wrote up some thoughts in here: #229 (comment)

I unfortunately don't have as much time as I used to these days, but one way to contribute would be to share the above and get feedback on it; then we can actually go forward (implementation shouldn't take long).

Also I didn't see @Nadrieril 's comment above; that's definitely something we could add to sources.nix right now! PRs welcome.

[tfc]

Hey i think this can be closed now with the latest PR being merged, right? #352

[nmattia]

Yes, thanks!