Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: merge ranged request storage into parent #1078

Merged
merged 21 commits into from
Feb 23, 2022

Conversation

jim3ma
Copy link
Member

@jim3ma jim3ma commented Feb 16, 2022

Signed-off-by: Jim Ma [email protected]

Description

In previous prefetch PR #1503, ranged request and parent request are split into 2 peer task and 2 storage cache.
In this PR, we merge them into only one storage cache.

With this feature, we can support nydusd to invoke daemon grpc to download partial blobs with final target path. nydusd did not need to save network data info file. The underlay file is only one copy via sharing inode in same filesystem.

Motivation and Context

Screenshots (if appropriate):

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation Update (if none of the other choices apply)

Checklist:

  • Code compiles correctly.
  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@jim3ma jim3ma requested a review from a team as a code owner February 16, 2022 09:47
@codecov-commenter
Copy link

codecov-commenter commented Feb 16, 2022

Codecov Report

Merging #1078 (390a903) into main (252a748) will decrease coverage by 0.51%.
The diff coverage is 35.20%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1078      +/-   ##
==========================================
- Coverage   45.83%   45.31%   -0.52%     
==========================================
  Files         129      130       +1     
  Lines        9543     9904     +361     
==========================================
+ Hits         4374     4488     +114     
- Misses       4675     4891     +216     
- Partials      494      525      +31     
Flag Coverage Δ
Object-compatibility-e2etests ?
e2etests ?
unittests 45.31% <35.20%> (-0.52%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
client/config/dfget.go 20.77% <ø> (ø)
client/config/peerhost.go 35.29% <ø> (ø)
client/daemon/daemon.go 5.41% <ø> (ø)
client/dfget/dfget.go 13.18% <0.00%> (-0.23%) ⬇️
pkg/source/httpprotocol/http_source_client.go 63.00% <ø> (ø)
client/daemon/storage/storage_manager.go 18.55% <26.26%> (-0.66%) ⬇️
client/daemon/storage/local_storage_subtask.go 29.26% <29.26%> (ø)
client/daemon/storage/local_storage.go 28.78% <40.62%> (+6.23%) ⬆️
client/daemon/rpcserver/rpcserver.go 31.89% <42.85%> (-1.75%) ⬇️
client/daemon/peer/peertask_stream.go 64.78% <50.00%> (+0.50%) ⬆️
... and 7 more

@jim3ma jim3ma force-pushed the feat/merge-ranged-request-storage-into-parent branch from caffc12 to 6db9418 Compare February 17, 2022 09:36
@jim3ma jim3ma requested a review from a team as a code owner February 18, 2022 08:09
Copy link
Member

@gaius-qi gaius-qi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@gaius-qi gaius-qi merged commit fe8fb6f into main Feb 23, 2022
@gaius-qi gaius-qi deleted the feat/merge-ranged-request-storage-into-parent branch February 23, 2022 10:02
@jim3ma
Copy link
Member Author

jim3ma commented Feb 25, 2022

Implement for #1050

gaius-qi pushed a commit that referenced this pull request Jun 28, 2023
* feat: merge ranged request storage into parent

Signed-off-by: Jim Ma <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants