[downloader/http] Retry download when urlopen times out #26603
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Before submitting a pull request make sure you have:
In order to be accepted and merged into youtube-dl each piece of code must be in public domain or released under Unlicense. Check one of the following options:
What is the purpose of your pull request?
Description of your pull request and other information
When the http downloader's
urlopen
times out, it used to stop the program with the timeout error, disrespecting the user's retry count settings. This fix addresses that, by having a timeout trigger a retry.I've tested it on Python 2.7.18 and 3.8.2.
While I think the fix is a useful thing to have in general, it addresses a specific issue I have, and described in #25183. That bug, however, was closed as a duplicate without any explanation or reference to a duplicate bug, so I never found out if there was any existing discussion about it.
Many times when the
dash
downloader is used to download youtube videos, my download would time out every 1-3 fragments. This means I would have to re-run the command over and over again.With the timeout retry fix, the dash fragments that get a timeout are immediately retried, and the download eventually completes without any visible errors, or me having to restart it. (Note that for a smooth download I need to run it with a shortened socket-timeout, because the default of 20 seconds is too much for this particular use-case).