RoosterTeeth.com support is broken following site update #16094
Comments
|
Nevermind, the API provides it directly. Example: https://svod-be.roosterteeth.com/api/v1/episodes/rooster-teeth-podcast-2018-486/videos From input url "https://roosterteeth.com/episode/rooster-teeth-podcast-2018-486", grab the last item and call the API endpoint. This doesn't work for FIRST videos, as they likely want authentication tokens passed along. |
|
I opened a pull request to use the new API, so downloading public videos works now, but anything that requires you to be logged in fails. I'm trying to find out how to authenticate with the API, but I'm not making much headway. |
|
@ddmgy I Might be able to help the AchieveLand MineCraft Lets Play is currently FIRST and while I haven't had time to determine precisely why, I am currently able to download it. The steps I used were log in to my account in Safari (obviously I'm using macOS) then once the video loads I right clicked and copied the video URL which is a standard right click option for HTML5 video in safari if I remember correctly. Then I gave that URL without a username or password to youtube-dl and it started downloading using hls , applehttp. If you need any more information then I can provide that although I will be somewhat busy |
|
I can confirm this, the m3u8 urls don't seem to require authentication to access. However we still face the issue of getting the initial m3u8 file. After a bit of network analysis, I've found what I believe should allow a username/password combo to access FIRST content. A POST request to 'https://auth.roosterteeth.com/oauth/token' with the below JSON file returns some parameters. Of these parameters, "access_token" seems to be most valuable. By calling the api url with the 'authorization' header set to "Bearer <<access_token>>" returns the m3u8 file. I have no idea what client_id is. It might be related to their various apps (Xbox, Apple TV, iOS, Browser?) I'm sharing it as I presume it isn't unique to my username, as it was set prior to the login. |
|
@Candunc Yeah, that client_id is not unique to your account, I see it in my request headers. I found that authorization URL earlier and I'm able to log in through youtube-dl now. I'm trying to get the m3u8 URL now, to see if downloading FIRST videos will work. Thanks for the tip about the Authorization header; I haven't done anything with OAuth in years, so it didn't occur to me how to use the access_token. Edit: Downloading FIRST-only videos now works. I'll clean it up and push my changes, and hopefully someone will take a look at merging my pull request soon. |
|
Wait, is it still broken for free videos? I don't understand how the OP (and second post) obtained any of those URLs, but I know that now that Camp Camp is not being uploaded to YouTube anymore, this is happening to me on the newest version (2018.06.14): |
If you put the episode you want to download in to this url:
eg: becomes:
and then from that webpage copy the .m3u8 link, you can feed it in to youtube-dl, and it'll download OK. |
|
For now, a quick and dirty fix could be to run a python equivalent of this, which gets exactly what we want: |
|
Has there been an update on this? Even free videos no longer work. |
|
@DuendeInexistente this didn't work for me on a free video: walkman@WalkmanPC:~/RWBY$ curl https://svod-be.roosterteeth.com/api/v1/episodes/rwby-volume-6-1/videos | sed 's/"/\n/g' | grep .m3u8
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 993 0 993 0 0 516 0 --:--:-- 0:00:01 --:--:-- 516
https://rtv3-video.roosterteeth.com/store/777dcdfb41acc2cebc48a69a94ab4460-f6785b63/ts/1540590815_index.m3u8
walkman@WalkmanPC:~/RWBY$ youtube-dl --verbose "https://rtv3-video.roosterteeth.com/store/777dcdfb41acc2cebc48a69a94ab4460-f6785b63/ts/1540590815_index.m3u8"
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', 'https://rtv3-video.roosterteeth.com/store/777dcdfb41acc2cebc48a69a94ab4460-f6785b63/ts/1540590815_index.m3u8']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2018.11.03
[debug] Python version 3.5.2 (CPython) - Linux-4.4.0-17134-Microsoft-x86_64-with-Ubuntu-16.04-xenial
[debug] exe versions: avconv 2.8.14-0ubuntu0.16.04.1, avprobe 2.8.14-0ubuntu0.16.04.1, ffmpeg 2.8.14-0ubuntu0.16.04.1, ffprobe 2.8.14-0ubuntu0.16.04.1
[debug] Proxy map: {}
[generic] 1540590815_index: Requesting header
WARNING: Could not send HEAD request to https://rtv3-video.roosterteeth.com/store/777dcdfb41acc2cebc48a69a94ab4460-f6785b63/ts/1540590815_index.m3u8: <urlopen error [Errno -3] Temporary failure in name resolution>
[generic] 1540590815_index: Downloading webpage
ERROR: Unable to download webpage: <urlopen error [Errno -3] Temporary failure in name resolution> (caused by URLError(gaierror(-3, 'Temporary failure in name resolution'),))
File "/usr/bin/youtube-dl/youtube_dl/extractor/common.py", line 605, in _request_webpage
return self._downloader.urlopen(url_or_request)
File "/usr/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2211, in urlopen
return self._opener.open(req, timeout=self._socket_timeout)
File "/usr/lib/python3.5/urllib/request.py", line 466, in open
response = self._open(req, data)
File "/usr/lib/python3.5/urllib/request.py", line 484, in _open
'_open', req)
File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
result = func(*args)
File "/usr/bin/youtube-dl/youtube_dl/utils.py", line 1139, in https_open
req, **kwargs)
File "/usr/lib/python3.5/urllib/request.py", line 1256, in do_open
raise URLError(err)I get the same error when doing it in one line (so no copy-and-paste): curl https://svod-be.roosterteeth.com/api/v1/episodes/rwby-volume-6-1/videos | sed 's/"/\n/g' | grep .m3u8 | youtube-dl -a - |
|
@Walkman100 that URL to m3u8 file works OK for me here. Unfortunately, it can't fetch the subs, no errors in the output but: video:795292kB audio:21132kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.072632% I'm not on the last youtube-dl version, so I will update ASAP and report back about the subs. EDIT: confirmed subs are also not fetched with 2018.11.03 |
|
weirdly enough that url works fine in a browser, I can download the m3u8 file and the file it references with aria2c, but even on another system it doesn't work with youtube-dl... |
|
In case this helps, the way I automated it still works for me with Camp Camp, when given a direct link to a video such as https://roosterteeth.com/episode/camp-camp-season-3-halloween-special (that I assigned to $URL): |
|
Thanks for trying to help, but that gives the same playlist url as the one-liner above (although yours can be given the full page url) which gives the same errors WARNING: Could not send HEAD request to https://rtv3-video.roosterteeth.com/store/777dcdfb41acc2cebc48a69a94ab4460-f6785b63/ts/1540590815_index.m3u8: <urlopen error [Errno -3] Temporary failure in name resolution>
ERROR: Unable to download webpage: <urlopen error [Errno -3] Temporary failure in name resolution> (caused by URLError(gaierror(-3, 'Temporary failure in name resolution'),))I noted above this doesn't work on another system (which is my Raspberry Pi), I tried this on Termux on my Android phone, and it works (same WiFi connection)… so the one-liner above not working on my system has nothing to do with youtube-dl or the script, it's an issue with DNS on my two systems on command-line somehow... |
|
About subtitles download: The m3u8 file lists the different subtitles as this: An inside each of those files, there is entries like this:
So, i.e., you can fetch the spanish subtitles if you use the URL of the spanish subs m3u8 file and change the filename for the name inside of the subs m3u8 file: https://rtv3-video.roosterteeth.com/caption_store/3ba18c5d-1f54-4f1d-b2fa-5e1c5e97eaca/vtt_es.webvtt |
so I am trying to download: https://roosterteeth.com/episode/rwby-volume-6-1 I have to input this: https://rtv3-video.roosterteeth.com/store/777dcdfb41acc2cebc48a69a94ab4460-f6785b63/ts/1540590815_index.m3u8 But it gives me an error: So I have to download one of them and the problem is fix? |
Yes, install ffmpeg to your path and you can download the videos. |
|
Can confirm, using the api trick and slapping the m3u8 link on to youtube-dl does indeed download Rooster Teeth videos. (I don't have a subscription so I can't say anything for those exclusive ones.) Any update on this problem? It's been almost three months since last comment for both the pull request and issue thread. |
|
When using the API trick I now get a "Missing Key-Pair-Id query parameter or cookie value" when putting the m3u8 url i nthe browser. And with yotube-dl I straight up get a 403. |
|
@sasjafor Try clearing cache and cookies. If that doesn't work, wait a day. It happened to me once and that's what I had to do to make it work again. |
|
I got it to work now. So for anyone having the same problem.
I suspect that the crypto parameters might expire, so you probably can't wait too long between doing the API request and then downloading with ffmpeg. I could not get it to work with youtube-dl as I kept getting a 403 error, but this is a solution until youtube-dl works. |
|
@sasjafor Thanks for this workaround! I noticed this is only necessary when the link starts with When it starts with |
|
Has there been any work to upstream a fix? |
|
The steps described by @sasjafor works for me. I wrote a Python 3 script to automate downloading of RWBY Volume 6 videos: import urllib.request, json
import os
episode = input("Episode: ")
link = "https://svod-be.roosterteeth.com/api/v1/episodes/rwby-volume-6-%s/videos" % episode
with urllib.request.urlopen(link) as url:
data = json.loads(url.read().decode())
dl = data['data'][0]['attributes']['url']
cmd = 'ffmpeg -protocol_whitelist file,http,https,tcp,tls,crypto -i "%s" -c copy rwby-volume-6-%s.mp4' % (dl, episode)
os.system(cmd) |
|
Forgive my lack of programming knowledge, but I could not get either @sasjafor or @jarrettyeo 's solutions to work. For @sasjafor , I followed the steps, and typed into command prompt: "C:\Program Files\Youtube-DL\ffmpeg-20190621-01d8c72-win64-static\bin\ffmpeg.exe" -protocol_whitelist file,http,https,tcp,tls,crypto -i "{"data":[{"_index":"videos-production-en_20190622090728596","type":"video","id":"113692","score":98.830475,"id":113692,"type":"video","uuid":"4cde6796-0a42-4b57-bb3c-dc520042b539","attributes":{"url":"https://rtv3-roosterteeth.akamaized.net/store/777dcdfb41acc2cebc48a69a94ab4460-f6785b63/ts/1540590815_index.m3u8?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly9ydHYzLXZpZGVvLnJvb3N0ZXJ0ZWV0aC5jb20vc3RvcmUvNzc3ZGNkZmI0MWFjYzJjZWJjNDhhNjlhOTRhYjQ0NjAtZjY3ODViNjMvdHMvMTU0MDU5MDgxNV9pbmRleC5tM3U4IiwiQ29uZGl0aW9uIjp7IkRhdGVMZXNzVGhhbiI6eyJBV1M6RXBvY2hUaW1lIjoxNTYxMjc3NDQwfX19XX0%Signature=WURo1N6XtDjfUMBSvS7voRKevSamalW4Z-F5OkcE8eI3w8VN9BXXHZzACxDa2ViWBVEDWQ-ZPPdTGbSTGoswckpSPDtPpX98UZUhBzdOUEEj7DmTNrXXS4f All I get is: No such file or directory |
|
For @jarrettyeo I opened up Notepad and put in: import urllib.request, json episode = input("Episode: ") cmd = '"C:\Program Files\Youtube-DL\ffmpeg-20190621-01d8c72-win64-static\bin\ffmpeg.exe" -protocol_whitelist file,http,https,tcp,tls,crypto -i "%s" -c copy rwby-volume-6-%s.mp4' % (dl, episode) where "C:\Program Files\Youtube-DL\ffmpeg-20190621-01d8c72-win64-static\bin\ffmpeg.exe" is my ffmpeg location, and saved it as a .py file. Upon opening the file with Python 3.0, it seems to run initially, with a prompt for episode number. However, after typing in a number, the script closes and nothing happens. |
|
@sasjafor realized I was typing it in wrong. Now copied the url starting at the https: part. But I am getting 403 access denied error now. |
|
@Genje I got the 403 error message as well. Only got it to work via my python 3 code. Try to open a terminal and execute the following: |
|
Hey @jarrettyeo thanks for trying to help. I opened Command Prompt and ran your script like you said. It prompted for an episode number, like so: C:\Users\Ryan\Documents>RWBY.py After pressing Enter I get: Traceback (most recent call last): |
|
@thegreatestminer It looks like a change was pushed a month ago, have you tried updating your version? |
|
@ipat8 Well I'll be damned. Updated to 2019.09.01 and it works flawlessly. Thanks! |
|
Getting HTTP Error 404: Not Found (caused by HTTPError()) etc when trying to download a video that's first-exclusive, non-first videos work perfectly. Updated to latest version. Added username and password using -u and -p. Recognises the videos are first-exclusive when -u and -p aren't used. |
|
@thegreatestminer Have you tried First-exclusive shows? Still doesn't work for me. Command I'm using is - "youtube-dl -u -p https://roosterteeth.com/watch/camp-camp-season-4-15" - with username and password. |
|
First exclusive shows are broken as c72dc20 only extracts free episodes. It needs to be modified to support authentication tokens like #16105 does. @remitamine I can give you a valid first login to test with if you are willing to write a patch. |
|
I've just created a pull request with a working FIRST only video downloader. #22689 I've tested it with a few FIRST videos and it seems to work, please try it with any others and tell me if you have any problems. |
Seems to have same problem as before, FIRST videos aren't working. Updated youtube-dl to latest version. |
|
@Matt14451 Did you specifically download the Version from my pull request? The main youtube-dl doesn't have my fixes until the pull request is approved and merged. |
|
@RichardHancock does your PR support downloading subtitles too? If so, I will consider getting a subscription and trying. |
|
@RichardHancock I downloaded and built yours and I get a 404 page not found error. URL Used: https://roosterteeth.com/watch/rwby-volume-7-1 Here is the output.
|
|
Here is the output from a Ubuntu system, which seems to at least use the new extractor, but has the same issue.
|
|
@madmalkav Sorry I don't believe it does, I'll try and have a look into it. |
|
Could you provide your binary? I confirmed that I built the one from your source. Not sure on what's wrong with my build. |
|
@ipat8 I'm not building the binary just running this from the top level directory of the repo: I've always had trouble building the binary myself for youtube-dl (weird build system configuration issues) and this method is a lot quicker for testing. |
@ipat8, it looks like you are not using RichardHancock's version. I'll admit I haven't tried building an exe from source, but that log line indicates you are using the wrong version. Did you |
|
Tried again, same issue. I may just have to wait for the code merge because I can not get this to work. I've tried with and without a VPN, tried multiple different OS/ Versions. This was cloned directly from your repo.
|
|
@ipat8 That login 404 error is the problem my commit should fix. Can you manually check the youtube_dl/extractor/rooster teeth.py and make sure it matches this: https://github.com/ytdl-org/youtube-dl/blob/3e738a38fd077f4fe0e8ad4be95a3889fc6cdfa0/youtube_dl/extractor/roosterteeth.py if not just download the file and replace the one you have. Otherwise I have no idea. |
|
I have no idea why that worked, but it did. I cloned your repo directly through github desktop. Not sure what the issue was. |
|
@madmalkav I have got subtitles working, I don't want to add them to my current pull request as it will probably delay it getting merged. I plan on doing a separate pull request after this one is merged to add show and season page support and any other improvements I can make. I can provide the subtitle code some other way in the meantime if wanted, It's just a little rough and needs some polish but works on all videos I've tried so far. |
|
I think your planning is good, I will wait to that future pull request to make an account on RT and test. |
Please follow the guide below
xinto all the boxes [ ] relevant to your issue (like this:[x])Make sure you are using the latest version: run
youtube-dl --versionand ensure your version is 2018.04.03. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.Before submitting an issue make sure you have:
What is the purpose of your issue?
Single video: https://rtv3-roosterteeth.akamaized.net/store/7281d24a2d5af5cba832b5335ced69ea-1e35796e/ts/index.m3u8
Note that youtube-dl does not support sites dedicated to copyright infringement. In order for site support request to be accepted all provided example URLs should not violate any copyrights.
Description of your issue, suggested solution and other information
Following the upgrade to RoosterTeeth.com, youtube-dl is no longer able to download videos, nor authenticate with the login page.
The m3u8 files have previously been hard to find, but now they are nearly impossible - it seems that the pages are dynamically generated on the client side as the tag is completely empty, aside from some required <script> tags. Reverse engineering the API may be necessary, as simply snooping on network traffic doesn't result in getting the m3u8 value.See below content, the m3u8 file is available through an API call
The text was updated successfully, but these errors were encountered: