Can't download video from Twitter, JSON metadata download returns 404

[deanylev]

Checklist

• I'm reporting a broken site support
• I've verified that I'm running youtube-dl version 2019.06.08
• I've checked that all provided URLs are alive and playable in a browser
• I've checked that all URLs and arguments with special characters are properly quoted or escaped
• I've searched the bugtracker for similar issues including closed ones

Verbose log

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--verbose', u'https://twitter.com/ViviEducation/status/1136534865145286656']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.06.08
[debug] Python version 2.7.10 (CPython) - Darwin-18.6.0-x86_64-i386-64bit
[debug] exe versions: none
[debug] Proxy map: {}
[twitter] 1136534865145286656: Downloading webpage
[twitter:card] 1136534865145286656: Downloading webpage
[twitter:card] 1136534865145286656: Downloading webpage
[twitter:card] 1136534865145286656: Downloading guest token
[twitter:card] 1136534865145286656: Downloading JSON metadata
ERROR: Unable to download JSON metadata: HTTP Error 404: Not Found (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 627, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2227, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 437, in open
    response = meth(req, response)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 550, in http_response
    'http', request, response, code, msg, hdrs)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 475, in error
    return self._call_chain(*args)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 558, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)

Description

I'm trying to download a video from Twitter, but the point where youtube-dl tries to download the JSON metadata results in a 404. This is the link https://twitter.com/ViviEducation/status/1136534865145286656

[RingoTheDog]

Another example: https://twitter.com/NiagRegPolice/status/1179380128993992704

youtube-dl.py -v "https://twitter.com/NiagRegPolice/status/117
9380128993992704"
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://twitter.com/NiagRegPolice/status/1179
380128993992704']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252
[debug] youtube-dl version 2019.09.28
[debug] Python version 3.6.3 (CPython) - Windows-7-6.1.7601-SP1
[debug] exe versions: ffmpeg N-71727-g46778ab, rtmpdump 2.4
[debug] Proxy map: {}
[twitter] 1179380128993992704: Downloading webpage
[twitter:card] 1179380128993992704: Downloading webpage
[twitter:card] 1179380128993992704: Downloading webpage
[twitter:card] 1179380128993992704: Downloading guest token
[twitter:card] 1179380128993992704: Downloading JSON metadata
ERROR: Unable to download JSON metadata: HTTP Error 404: Not Found (caused by <H
TTPError 404: 'Not Found'>); please report this issue on https://yt-dl.org/bug .
 Make sure you are using the latest version; type  youtube-dl -U  to update. Be
sure to call youtube-dl with the --verbose flag and include its complete output.

  File "C:\Transmogrifier\youtube-dl.py\youtube_dl\extractor\common.py", line 62
7, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "C:\Transmogrifier\youtube-dl.py\youtube_dl\YoutubeDL.py", line 2237, in
urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "C:\Program Files\Python36\lib\urllib\request.py", line 532, in open
    response = meth(req, response)
  File "C:\Program Files\Python36\lib\urllib\request.py", line 642, in http_resp
onse
    'http', request, response, code, msg, hdrs)
  File "C:\Program Files\Python36\lib\urllib\request.py", line 570, in error
    return self._call_chain(*args)
  File "C:\Program Files\Python36\lib\urllib\request.py", line 504, in _call_cha
in
    result = func(*args)
  File "C:\Program Files\Python36\lib\urllib\request.py", line 650, in http_erro
r_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)


```

Thanks
Ringo

[andywalden]

Another example:

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', 'https://twitter.com/VeritasSola/status/1182712384407908352']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.09.28
[debug] Python version 3.7.4 (CPython) - Linux-4.15.0-60-generic-x86_64-with-Ubuntu-18.04-bionic
[debug] exe versions: ffmpeg 3.4.6, ffprobe 3.4.6
[debug] Proxy map: {}
[twitter] 1182712384407908352: Downloading webpage
[twitter:card] 1182712384407908352: Downloading webpage
[twitter:card] 1182712384407908352: Downloading webpage
[twitter:card] 1182712384407908352: Downloading guest token
[twitter:card] 1182712384407908352: Downloading JSON metadata
ERROR: Unable to download JSON metadata: HTTP Error 404: Not Found (caused by <HTTPError 404: 'Not Found'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
File "/home/aleister/code/prod/redbot/env_redbot/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 627, in _request_webpage
return self._downloader.urlopen(url_or_request)
File "/home/aleister/code/prod/redbot/env_redbot/lib/python3.7/site-packages/youtube_dl/YoutubeDL.py", line 2237, in urlopen
return self._opener.open(req, timeout=self._socket_timeout)
File "/usr/lib/python3.7/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.7/urllib/request.py", line 641, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python3.7/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "/usr/lib/python3.7/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)