Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: prefetch ranged requests #1053

Merged
merged 10 commits into from
Feb 10, 2022
Merged

Feature: prefetch ranged requests #1053

merged 10 commits into from
Feb 10, 2022

Conversation

jim3ma
Copy link
Member

@jim3ma jim3ma commented Feb 8, 2022

Description

When download ranged requests, eg: http://example.com/file with header "Range: bytes=0-63", we can prefetch the whole file with http://example.com/file. After done these, other requests, eg: http://example.com/file with header "Range: bytes=64-127", can speed up.

  1. Add prefetch option in dfget.yaml.
  2. Support reuse parent when download ranged requests.

Related Issue

Motivation and Context

Screenshots (if appropriate):

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation Update (if none of the other choices apply)

Checklist:

  • Code compiles correctly.
  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@jim3ma jim3ma requested a review from a team as a code owner February 8, 2022 09:03
@codecov-commenter
Copy link

Codecov Report

Merging #1053 (ff532e4) into main (a855c0e) will decrease coverage by 0.48%.
The diff coverage is 23.83%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1053      +/-   ##
==========================================
- Coverage   46.24%   45.75%   -0.49%     
==========================================
  Files         129      129              
  Lines        9390     9534     +144     
==========================================
+ Hits         4342     4362      +20     
- Misses       4564     4679     +115     
- Partials      484      493       +9     
Flag Coverage Δ
Object-compatibility-e2etests ?
e2etests ?
unittests 45.75% <23.83%> (-0.49%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
client/clientutil/range.go 87.03% <0.00%> (-1.65%) ⬇️
client/config/peerhost.go 35.29% <ø> (ø)
client/daemon/daemon.go 5.41% <0.00%> (ø)
client/daemon/peer/peertask_file.go 56.69% <0.00%> (-0.91%) ⬇️
client/daemon/peer/peertask_manager.go 33.65% <0.00%> (-10.65%) ⬇️
client/daemon/storage/local_storage.go 22.54% <0.00%> (-0.86%) ⬇️
client/daemon/storage/storage_manager.go 19.20% <ø> (ø)
client/dfget/dfget.go 13.40% <0.00%> (-0.23%) ⬇️
client/daemon/transport/transport.go 51.38% <20.68%> (-6.60%) ⬇️
client/daemon/peer/peertask_reuse.go 52.02% <30.58%> (-29.23%) ⬇️
... and 8 more

@jim3ma jim3ma force-pushed the feat/prefetch-ranged-requests branch from 9e7d122 to 6d3daa1 Compare February 10, 2022 08:19
@jim3ma jim3ma merged commit fed4f56 into main Feb 10, 2022
@jim3ma jim3ma deleted the feat/prefetch-ranged-requests branch February 10, 2022 13:40
gaius-qi pushed a commit that referenced this pull request Jun 28, 2023
1. implement prefetch ranged requests
2. optimize exact http code in transport
3. simplify reuse peer task logic
4. reuse peer task for ranged request size error
5. fix data race for peer task storage

Signed-off-by: Jim Ma <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants