recipe.download_file: implement shallow git cloning #2682
Conversation
When a recipe uses a `git+...` url, and has a `version` specified, only do a shallow git clone.
This saves disk space and bandwidth.
Tested with a custom qt5 recipe.
Without this patch, the git clone on disk was 8.5 GB, now it is 5.0 GB.
```
class Qt5Recipe(BootstrapNDKRecipe):
url = 'git+https://code.qt.io/qt/qt5.git'
#version = '5.15.2'
version = '9b43a43ee96198674060c6b9591e515e2d27c28f'
```
| # if 'version' is specified, do a shallow clone | ||
| if self.version: | ||
| shprint(sh.mkdir, '-p', target) | ||
| with current_directory(target): | ||
| shprint(sh.git, 'checkout', self.version) | ||
| shprint(sh.git, 'submodule', 'update', '--recursive') | ||
| shprint(sh.git, 'init') | ||
| shprint(sh.git, 'remote', 'add', 'origin', url) |
There was a problem hiding this comment.
have we thought about directly cloning that branch from here and skipping the other code path?
Something like:
shprint(sh.git, 'clone', '--branch', self.version, '--single-branch', url)I haven't tested so I don't know if the rest of the code apply something we would need still
There was a problem hiding this comment.
Which code path do you mean?
Do you mean you would do this instead of
shprint(sh.git, 'fetch', '--depth', '1', 'origin', self.version)
shprint(sh.git, 'checkout', self.version)but note that version is not necessarily a branch -- I for example always set it to a commit hash, for reproducibility. In that case your command errors.
Also, btw, I could simplify this logic if we raised on version not being set. I am not sure in what scenario it makes sense to set a git repo but not even specify a branch.
The combinatorial matrix of possibilities is surprisingly large here btw,
{isdir(target)}x{version in (not_set, branch, commit)} gives 6 options, and that's not even counting whether there are submodules.
When a recipe uses a "git+..."
url, and has aversionspecified, only do a shallow git clone. This saves disk space and bandwidth.Tested with a custom qt5 recipe.
Without this patch, the git clone on disk was 8.5 GB, now it is 5.0 GB.