[Twitter] extractor.*.retries is not respected

[rEnr3n]

config.txt

{
    "extractor": {
        "retries": -1
    }
}

$ gallery-dl -v --ignore-config -c config.txt https://twitter.com/220cmjessica/status/1266215449588494336                                                                                                         :(
[gallery-dl][debug] Version 1.13.6
[gallery-dl][debug] Python 3.8.3 - Linux-5.6.14-arch1-1-x86_64-with-glibc2.2.5
[gallery-dl][debug] requests 2.23.0 - urllib3 1.25.9
[gallery-dl][debug] Starting DownloadJob for 'https://twitter.com/220cmjessica/status/1266215449588494336'
[twitter][debug] Using TwitterTweetExtractor for 'https://twitter.com/220cmjessica/status/1266215449588494336'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): twitter.com:443
[urllib3.connectionpool][debug] https://twitter.com:443 "GET /i/web/status/1266215449588494336 HTTP/1.1" 200 59693
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): api.twitter.com:443
[urllib3.connectionpool][debug] https://api.twitter.com:443 "POST /1.1/guest/activate.json HTTP/1.1" 200 63
[urllib3.connectionpool][debug] https://api.twitter.com:443 "GET /1.1/videos/tweet/config/1266215449588494336.json HTTP/1.1" 200 1208
[downloader.ytdl][debug] [generic] g2FGHOgeEYMhs_KO: Requesting header
[downloader.ytdl][warning] Could not send HEAD request to https://video.twimg.com/ext_tw_video/1266215394768850948/pu/pl/g2FGHOgeEYMhs_KO.m3u8?tag=10: <urlopen error [Errno -3] Temporary failure in name resolution>
[downloader.ytdl][debug] [generic] g2FGHOgeEYMhs_KO: Downloading webpage
[downloader.ytdl][error] ERROR: Unable to download webpage: <urlopen error [Errno -3] Temporary failure in name resolution> (caused by URLError(gaierror(-3, 'Temporary failure in name resolution')))
[download][error] Failed to download 1266215449588494336_1.mp4

It doesn't seem to use the default retries of 4 either.

[mikf]

This is the result of a missing feature in youtube-dl. It has a retries option, but it only applies for file downloads. Network errors before that stage don't get retried by it.

The new Twitter code (#806) won't need youtube-dl for video downloads anymore, so at least there it shouldn't be an issue anymore

[ntqr]

@mikf
Wait so does that mean extractor.twitter.videos will work differently?
Will setting it to 'true' now mean that gallery-dl will handle all the video extraction without youtube-dl's help?

[mikf]

@anonieee
Exactly. The new version won't use youtube-dl to download videos from Twitter anymore, unless you explicitly tell it to by setting extractor.twitter.videos to "ytdl"