Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fetch-npm-deps: Reduce flaky network issues by limiting parallelism #258428

Closed

Conversation

amarshall
Copy link
Member

Description of changes

Sometimes fails flaky with errors like:

[55] Failed sending data to the peer

that are difficult to reproduce at times, and cause a lot of frustration as the general answer is to just keep retrying.

Testing with nix-build --check -A bitwarden.npmDeps, I got these results:

  • 2 cores: 20 / 20 runs succeeded in ~35 min total
  • 4 cores: 18 / 20 runs succeeded in ~17 min total
  • 8 cores: 11 / 20 runs succeeded in ~ 8 min total

This isn’t the ideal solution as it doesn’t fix the problem, but it is at least something to temper the frequent flaky failures without sacrificing too much runtime.

See
#256873
#257805

Things done

Have only tested bitwarden.npmDeps, assuming given the context of the change that it will not cause problems for other dependents.

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 23.11 Release Notes (or backporting 23.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Sometimes fails flaky with errors like:

> [55] Failed sending data to the peer

that are difficult to reproduce at times, and cause a lot of
frustration as the general answer is to just keep retrying.

Testing with `nix-build --check -A bitwarden.npmDeps`, I got these
results:

- 2 cores: 20 / 20 runs succeeded in ~35 min total
- 4 cores: 18 / 20 runs succeeded in ~17 min total
- 8 cores: 11 / 20 runs succeeded in ~ 8 min total

This isn’t the ideal solution as it doesn’t fix the problem, but it is
at least something to temper the frequent flaky failures without
sacrificing too much runtime.

See
NixOS#256873
NixOS#257805
@lilyinstarlight
Copy link
Member

Can you see if #260011 fixes the issue?

@amarshall
Copy link
Member Author

Closing in favor of #260011

@amarshall amarshall closed this Oct 10, 2023
@amarshall amarshall deleted the npm-fetch-limit-parallelism branch December 11, 2023 17:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants