Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Huge performance regression when evaluating clean flake #5280

Closed
Kha opened this issue Sep 21, 2021 · 5 comments
Closed

Huge performance regression when evaluating clean flake #5280

Kha opened this issue Sep 21, 2021 · 5 comments

Comments

@Kha
Copy link
Contributor

Kha commented Sep 21, 2021

Describe the bug

On a clean checkout of a sizeable repo (~20k commits), the time for the first evaluation of a commit has recently increased by more than 10x (and an unhelpful message from Git is shown):

$ nix build  # cache build

$ echo >> flake.nix; git commit -am foo
[fvis 296fea547d] foo
 1 file changed, 1 insertion(+)

$ nix shell github:NixOS/nix/90b2dd570cbd8313a8cf45b3cf66ddef2bb06e07 -c sh -c 'nix --version; time nix build'
nix (Nix) 2.4pre20210902_90b2dd5
2.10user 0.59system 0:02.60elapsed 103%CPU (0avgtext+0avgdata 287372maxresident)k
0inputs+402536outputs (0major+115786minor)pagefaults 0swaps

$ echo >> flake.nix; git commit -am foo
[fvis 07a735a0b9] foo
 1 file changed, 1 insertion(+)

$ nix shell github:NixOS/nix/6678e98411cd3bfe8143a4ba1b35d1451babce32 -c sh -c 'nix --version; time nix build'
nix (Nix) 2.4pre20210907_6678e98
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint: 	git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint: 	git branch -m <name>
94.89user 2.88system 0:35.60elapsed 274%CPU (0avgtext+0avgdata 1234872maxresident)k
264inputs+402224outputs (1major+488572minor)pagefaults 0swaps

This is due to #4922, which enables cloning submodules by default. But the submodules option is extremely inefficient, even if the repo does not even contain any submodules.

I know there has been some discussion on the PR itself after merging, but I felt that this regression deserved its own issue due to its severity.

Steps To Reproduce

git clone https://github.com/leanprover/lean4
cd lean4
nix flake prefetch

Expected behavior

A much faster store copy, as before

nix-env --version output

nix (Nix) 2.4pre20210907_6678e98
@Kha Kha added the bug label Sep 21, 2021
@Kha
Copy link
Contributor Author

Kha commented Sep 21, 2021

Also note this comment from the original PR introducing submodules:

This PR does not change any of the fetchGit behavior when submodules are not used (the default), so only people using this feature are paying these costs.

@edolstra
Copy link
Member

Hm, maybe we should revert #4922 for now and revisit the submodules issue after 2.4 is released.

@edolstra edolstra added this to the nix-2.4 milestone Sep 22, 2021
@Kha
Copy link
Contributor Author

Kha commented Sep 22, 2021

Yes, that would probably be for the best. Ironically I updated Nix specifically so I could add a (non-critical) submodule to this flake, so I'm interested in seeing this resolved properly eventually, but the current state is not really tenable.

@edolstra
Copy link
Member

Closing this because #4922 was reverted.

@nixos-discourse
Copy link

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/tweag-nix-dev-update-18/15300/1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants