Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Close HTTP response when retrying on errors #5

Merged
merged 1 commit into from
Mar 21, 2023

Conversation

northnose
Copy link

This attempts to fix the problem described in https://forum.duplicacy.com/t/cannot-check-chunks-socket-too-many-open-files/7385 which seems to occur when the Dropbox client retries on certain errors (401, 429, 500). In these cases, the client retries with another HTTP request, but it fails to close the existing one. Eventually, this leaks sockets and the process runs out of sockets.

This attempts to fix the problem described in https://forum.duplicacy.com/t/cannot-check-chunks-socket-too-many-open-files/7385 which seems to occur when the Dropbox client retries on certain errors (401, 429, 500). In these cases, the client retries with another HTTP request, but it fails to close the existing one. Eventually, this leaks sockets and the process runs out of sockets.
client.go Show resolved Hide resolved
client.go Show resolved Hide resolved
client.go Show resolved Hide resolved
client.go Show resolved Hide resolved
@gilbertchen
Copy link
Owner

This pull request has been mentioned on Duplicacy Forum. There might be relevant details there:

https://forum.duplicacy.com/t/cannot-check-chunks-socket-too-many-open-files/7385/3

@QuaCKeReD
Copy link

Could this also be fix for the Storj ‘too many files’ issue?

@northnose
Copy link
Author

Could this also be fix for the Storj ‘too many files’ issue?

The issue may be conceptually related or even the same as this one, but it's not possible that this PR fixes it. This change is in the Dropbox Go API wrapper, and Duplicacy uses an entirely different storage implementation for Storj: https://github.com/gilbertchen/duplicacy/blob/master/src/duplicacy_storjstorage.go#L16

@northnose
Copy link
Author

@gilbertchen would you mind reviewing this PR?

@gilbertchen
Copy link
Owner

This looks good. Thank you very much for fixing it!

@gilbertchen gilbertchen reopened this Mar 21, 2023
@gilbertchen gilbertchen merged commit 087ef8d into gilbertchen:master Mar 21, 2023
@northnose northnose deleted the error-close branch March 21, 2023 03:14
@gilbertchen
Copy link
Owner

This pull request has been mentioned on Duplicacy Forum. There might be relevant details there:

https://forum.duplicacy.com/t/backup-to-dropbox-aborts-on-synology-due-to-too-many-open-files/7733/1

@gilbertchen
Copy link
Owner

This pull request has been mentioned on Duplicacy Forum. There might be relevant details there:

https://forum.duplicacy.com/t/cli-release-3-2-0-is-now-available/7952/1

@gilbertchen
Copy link
Owner

This pull request has been mentioned on Duplicacy Forum. There might be relevant details there:

https://forum.duplicacy.com/t/duplicacy-cli-3-2-2-release/8019/1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants