-
Notifications
You must be signed in to change notification settings - Fork 368
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
pkg/download: close files after extracting in tar #310
Conversation
- Close file handle after extracting files on windows in extractTARGZ(). - Also ensure we close when io.Copy fails on extractZIP(). - Log the ignored error from the cleanup of DownloadPath. Signed-off-by: Ahmet Alp Balkan <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for investigating this nasty bug. I have minor questions.
@@ -131,9 +132,12 @@ func extractTARGZ(targetDir string, at io.ReaderAt, size int64) error { | |||
if err != nil { | |||
return errors.Wrapf(err, "failed to create file %q", path) | |||
} | |||
close := func() { f.Close() } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would a defer f.Close()
not be a cleaner solution?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we could end up with have thousands of files open and end up exceeding open fd limit. no need to delay.
@@ -74,15 +74,16 @@ func extractZIP(targetDir string, read io.ReaderAt, size int64) error { | |||
if err != nil { | |||
return errors.Wrap(err, "can't create file in zip destination dir") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess we should src.Close()
here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah I think this is valid.
@@ -74,15 +74,16 @@ func extractZIP(targetDir string, read io.ReaderAt, size int64) error { | |||
if err != nil { | |||
return errors.Wrap(err, "can't create file in zip destination dir") | |||
} | |||
close := func() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't quite see why defer
can't be used here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it would defer until all of the files are extracted.
Signed-off-by: Ahmet Alp Balkan <[email protected]>
Codecov Report
@@ Coverage Diff @@
## master #310 +/- ##
==========================================
+ Coverage 55.66% 55.72% +0.05%
==========================================
Files 19 19
Lines 918 926 +8
==========================================
+ Hits 511 516 +5
- Misses 355 358 +3
Partials 52 52
Continue to review full report at Codecov.
|
/lgtm |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: ahmetb, corneliusweig The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Fixes #308.