Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is there still no way to get past "Watch this video on YouTube. Playback on other websites has been disabled by the video owner" #54

Closed
nicolaasjan opened this issue Sep 7, 2020 · 12 comments

Comments

@nicolaasjan
Copy link

Checklist

  • [x ] I'm reporting a site feature request
  • [x ] I've verified that I'm running youtube-dlc version 2020.09.06
  • [x ] I've searched the bugtracker for similar site feature requests including closed ones

Description

Like the title says.

Example:
https://www.youtube.com/watch?v=7takIh1nK0s

Verbose output:

youtube-dlc -v https://www.youtube.com/watch?v=7takIh1nK0s
[debug] System config: []
[debug] User config: ['-o', '~/Desktop/%(title)s.%(ext)s', '-f', 'bestvideo[height<=1280][ext=mp4]+bestaudio[ext=m4a]/best', '--no-mtime', '--add-metadata', '--embed-thumbnail']
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www.youtube.com/watch?v=7takIh1nK0s']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dlc version 2020.09.06
[debug] Python version 3.6.9 (CPython) - Linux-4.15.0-115-generic-x86_64-with-LinuxMint-19.3-tricia
[debug] exe versions: ffmpeg N-99019-gbc86629-Nico, ffprobe N-99019-gbc86629-Nico
[debug] Proxy map: {}
[youtube] 7takIh1nK0s: Downloading webpage
[youtube] 7takIh1nK0s: Downloading embed webpage
[youtube] 7takIh1nK0s: Refetching age-gated info webpage
ERROR: Watch this video on YouTube. Playback on other websites has been disabled by the video owner.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dlc/youtube_dlc/YoutubeDL.py", line 797, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dlc/youtube_dlc/extractor/common.py", line 530, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/bin/youtube-dlc/youtube_dlc/extractor/youtube.py", line 2239, in _real_extract
    raise ExtractorError(error_message, expected=True)
youtube_dlc.utils.ExtractorError: Watch this video on YouTube. Playback on other websites has been disabled by the video owner.
@blackjack4494
Copy link
Owner

Hm. Have you tried with cookies by any chance? This is very strange. I believe there is a way somehow to get this video. But this will require some more inspection as why won't youtube allow you to grab it.

@nicolaasjan
Copy link
Author

Hm. Have you tried with cookies by any chance? This is very strange. I believe there is a way somehow to get this video. But this will require some more inspection as why won't youtube allow you to grab it.

Just created a fresh cookies.txt file, but still no luck...:

youtube-dlc -v https://www.youtube.com/watch?v=7takIh1nK0s
[debug] System config: []
[debug] User config: ['-o', '~/Desktop/%(title)s.%(ext)s', '-f', 'bestvideo[height<=1280][ext=mp4]+bestaudio[ext=m4a]/best', '--no-mtime', '--add-metadata', '--embed-thumbnail', '--cookies', '/home/nico/.config/youtube-dl/cookies.txt']
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www.youtube.com/watch?v=7takIh1nK0s']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dlc version 2020.09.06
[debug] Python version 3.6.9 (CPython) - Linux-4.15.0-115-generic-x86_64-with-LinuxMint-19.3-tricia
[debug] exe versions: ffmpeg N-99019-gbc86629-Nico, ffprobe N-99019-gbc86629-Nico
[debug] Proxy map: {}
[youtube] 7takIh1nK0s: Downloading webpage
[youtube] 7takIh1nK0s: Downloading embed webpage
[youtube] 7takIh1nK0s: Refetching age-gated info webpage
ERROR: Watch this video on YouTube. Playback on other websites has been disabled by the video owner.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dlc/youtube_dlc/YoutubeDL.py", line 797, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dlc/youtube_dlc/extractor/common.py", line 530, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/bin/youtube-dlc/youtube_dlc/extractor/youtube.py", line 2239, in _real_extract
    raise ExtractorError(error_message, expected=True)
youtube_dlc.utils.ExtractorError: Watch this video on YouTube. Playback on other websites has been disabled by the video owner.

See also:
https://old.reddit.com/r/youtubedl/comments/io4brl/error_when_downloading_video_with_disabled/
(same video).

@blackjack4494
Copy link
Owner

blackjack4494 commented Sep 8, 2020

First off I blame it on youtube-dl

I made some progress on this. It is definitely possible to download this. However it seems the way youtube-dl work it won't work.

curl 'https://www.youtube.com/watch?v=7takIh1nK0s&pbs=1' \
  -H 'cookie: VISITOR_INFO1_LIVE=__REDACTED__; HSID=__REDACTED__; SSID=__REDACTED__; SID=__REDACTED__;' \
  --compressed

Or this

curl 'https://www.youtube.com/watch?v=7takIh1nK0s' \
  -H 'cookie: VISITOR_INFO1_LIVE=__REDACTED__;  __Secure-3PSID=__REDACTED__; LOGIN_INFO=__REDACTED__; ' \
  --compressed

This is all I needed to get the data with the itags. So I tried adding the headers using --add-header. But that didn't work out. Then I tried supplying cookies. Which didn't work either.

There may be a problem with the age gated fix.

Proof it can work:

PS D:\gitkraken\youtube-dl> python3 -m youtube_dlc 7takIh1nK0s -F -v
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['7takIh1nK0s', '-F', '-v']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252
[debug] youtube-dlc version 2020.09.06
[debug] Git HEAD: 2045de7
[debug] Python version 3.6.5 (CPython) - Windows-10-10.0.19041-SP0
[debug] exe versions: ffmpeg 4.2.1, ffprobe 4.2.1
[debug] Proxy map: {}
[youtube] 7takIh1nK0s: Downloading webpage
[info] Available formats for 7takIh1nK0s:
format code extension resolution note
249 webm audio only tiny 53k , opus @ 50k (48000Hz), 3.86MiB
250 webm audio only tiny 71k , opus @ 70k (48000Hz), 4.90MiB
140 m4a audio only tiny 130k , m4a_dash container, mp4a.40.2@128k (44100Hz), 10.01MiB
251 webm audio only tiny 137k , opus @160k (48000Hz), 9.15MiB
394 mp4 256x144 144p 81k , av01.0.00M.08, 30fps, video only, 5.34MiB
278 webm 256x144 144p 97k , webm container, vp9, 30fps, video only, 6.51MiB
160 mp4 256x144 144p 101k , avc1.4d400c, 30fps, video only, 4.32MiB
395 mp4 426x240 240p 177k , av01.0.00M.08, 30fps, video only, 9.54MiB
133 mp4 426x240 240p 200k , avc1.4d4015, 30fps, video only, 7.69MiB
242 webm 426x240 240p 207k , vp9, 30fps, video only, 9.56MiB
396 mp4 640x360 360p 318k , av01.0.01M.08, 30fps, video only, 16.89MiB
243 webm 640x360 360p 341k , vp9, 30fps, video only, 16.48MiB
134 mp4 640x360 360p 358k , avc1.4d401e, 30fps, video only, 13.92MiB
244 webm 854x480 480p 503k , vp9, 30fps, video only, 23.47MiB
397 mp4 854x480 480p 521k , av01.0.04M.08, 30fps, video only, 28.64MiB
135 mp4 854x480 480p 543k , avc1.4d401f, 30fps, video only, 19.89MiB
298 mp4 1280x720 720p60 880k , avc1.4d4020, 60fps, video only, 35.34MiB
398 mp4 1280x720 720p60 1160k , av01.0.08M.08, 60fps, video only, 70.91MiB
302 webm 1280x720 720p60 1285k , vp9, 60fps, video only, 54.62MiB
247 webm 1280x720 720p 1317k , vp9, 30fps, video only, 63.26MiB
136 mp4 1280x720 720p 1504k , avc1.4d401f, 30fps, video only, 61.72MiB
399 mp4 1920x1080 1080p60 2098k , av01.0.09M.08, 60fps, video only, 129.17MiB
248 webm 1920x1080 1080p 2326k , vp9, 30fps, video only, 117.54MiB
137 mp4 1920x1080 1080p 2722k , avc1.640028, 30fps, video only, 113.06MiB
299 mp4 1920x1080 1080p60 3376k , avc1.64002a, 60fps, video only, 129.37MiB
303 webm 1920x1080 1080p60 3662k , vp9, 60fps, video only, 178.52MiB
18 mp4 640x360 360p 441k , avc1.42001E, 30fps, mp4a.40.2@ 96k (44100Hz), 34.11MiB
22 mp4 1280x720 720p 927k , avc1.64001F, 30fps, mp4a.40.2@192k (44100Hz) (best)

How did I do that?

  1. I simply used the curl as stated above with actual parameters (replace REDACTED with your values).
  2. Copied the response (html object).
  3. Put that into a file (e.g. youtube_html.txt)
  4. and searched for the age restriction line which I then removed - otherwise the age restriction check fires up and wants to use an alternative approach / workaround but since we have all the data in the html object we don't want that.
    <meta property="og:restrictions:age" content="18+"> <- remove that in your txt file.
  5. Overwrite video_webpage (do not remove the line but add a new line with same variable name)
    video_webpage, urlh = self._download_webpage_handle(url, video_id)

    To this by defining the variable again
video_webpage, urlh = self._download_webpage_handle(url, video_id)
video_webpage = open("FULL PATH TO YOUR SAVED FILE", "r", encoding="utf-8").read()
  1. python3 -m youtube_dlc 7takIh1nK0s -F -v

Alternative to Step 1 - Ctrl + U (show source) and copy that. It should do the same but haven't tested it.

Keep in mind that you cannot download any other youtube video then. This is just a POC (proof of concept). So someone may be able or want to fix that issue.
Another solution would be to add a feature to use files when declared instead of actual web pages. That way you can feed data by another tool. That should also circumvent when videos are geo restricted and someone is providing that html object. But that is just theory and would need to be tested.

I believe this video is a special case where it requires you to be logged in to view it and in addition it's age restricted.

@nicolaasjan
Copy link
Author

Hmm...
Step1 gave an error:

curl 'https://www.youtube.com/watch?v=7takIh1nK0s'-H 'cookie: VISITOR_INFO1_LIVE=REDACTED; HSID=REDACTED; SSID=REDACTED; SID=REDACTED;' --compressed
curl: (3) Port number ended with ' '

So I put the source into a html file.

Then I cloned the source tree of youtube-dlc and looked for:
<meta property="og:restrictions:age" content="18+"> in youtube.py
Looks different (line 1846):
if self._html_search_meta('og:restrictions:age', video_webpage, default=None) == "18+":
So I'm not sure which line(s) I should remove (I'm not a coder).

@blackjack4494
Copy link
Owner

blackjack4494 commented Sep 8, 2020

Hmm...
Step1 gave an error:

curl 'https://www.youtube.com/watch?v=7takIh1nK0s'-H 'cookie: VISITOR_INFO1_LIVE=REDACTED; HSID=REDACTED; SSID=REDACTED; SID=REDACTED;' --compressed
curl: (3) Port number ended with ' '

So I put the source into a html file.

Then I cloned the source tree of youtube-dlc and looked for:
<meta property="og:restrictions:age" content="18+"> in youtube.py
Looks different (line 1846):
if self._html_search_meta('og:restrictions:age', video_webpage, default=None) == "18+":
So I'm not sure which line(s) I should remove (I'm not a coder).

I updated the instructions a bit. I did not notice that the quote mode actually remove \ and interprets it as a new line character.
The meta tag is an html tag so not the python file but the source you put in a file. That file doesn't need the html type. Could be simple txt.

Also you need to replace the REDACTED with actual values. Those can be found if you are using chrome and open developer tools. Easiest to would be to go to the tab Application and then on the left side Storage -> Cookies -> youtube. Then look out for the names and replace REDACTED with the values.

@nicolaasjan
Copy link
Author

nicolaasjan commented Sep 8, 2020

Yeah, I had already replaced the REDACTED with actual values ;)
I got the values by hitting F12 in my Pale Moon browser.

Now the curl command completed successfully and I put the source in the youtube_html.txt file.
There I removed <meta property="og:restrictions:age" content="18+">
Then I inserted:
video_webpage = open("/home/nico/Desktop/youtube_html.txt", "r", encoding="utf-8").read()
after line 1800 in youtube.py.
Then:

python3 -m youtube_dlc 7takIh1nK0s -F -v
[debug] System config: []
[debug] User config: ['-o', '~/Desktop/%(title)s.%(ext)s', '-f', 'bestvideo[height<=1280][ext=mp4]+bestaudio[ext=m4a]/best', '--no-mtime', '--add-metadata', '--embed-thumbnail']
[debug] Custom config: []
[debug] Command-line args: ['7takIh1nK0s', '-F', '-v']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dlc version 2020.09.06
[debug] Git HEAD: 2045de7c3
[debug] Python version 3.6.9 (CPython) - Linux-4.15.0-115-generic-x86_64-with-LinuxMint-19.3-tricia
[debug] exe versions: ffmpeg N-99019-gbc86629-Nico, ffprobe N-99019-gbc86629-Nico
[debug] Proxy map: {}
[youtube] 7takIh1nK0s: Downloading webpage
[info] Available formats for 7takIh1nK0s:
format code  extension  resolution note
249          webm       audio only tiny   53k , opus @ 50k (48000Hz), 3.86MiB
250          webm       audio only tiny   71k , opus @ 70k (48000Hz), 4.90MiB
140          m4a        audio only tiny  130k , m4a_dash container, mp4a.40.2@128k (44100Hz), 10.01MiB
251          webm       audio only tiny  137k , opus @160k (48000Hz), 9.15MiB
394          mp4        256x144    144p   81k , av01.0.00M.08, 30fps, video only, 5.34MiB
278          webm       256x144    144p   97k , webm container, vp9, 30fps, video only, 6.51MiB
160          mp4        256x144    144p  101k , avc1.4d400c, 30fps, video only, 4.32MiB
395          mp4        426x240    240p  177k , av01.0.00M.08, 30fps, video only, 9.54MiB
133          mp4        426x240    240p  200k , avc1.4d4015, 30fps, video only, 7.69MiB
242          webm       426x240    240p  207k , vp9, 30fps, video only, 9.56MiB
396          mp4        640x360    360p  318k , av01.0.01M.08, 30fps, video only, 16.89MiB
243          webm       640x360    360p  341k , vp9, 30fps, video only, 16.48MiB
134          mp4        640x360    360p  358k , avc1.4d401e, 30fps, video only, 13.92MiB
244          webm       854x480    480p  503k , vp9, 30fps, video only, 23.47MiB
397          mp4        854x480    480p  521k , av01.0.04M.08, 30fps, video only, 28.64MiB
135          mp4        854x480    480p  543k , avc1.4d401f, 30fps, video only, 19.89MiB
298          mp4        1280x720   720p60  880k , avc1.4d4020, 60fps, video only, 35.34MiB
398          mp4        1280x720   720p60 1160k , av01.0.08M.08, 60fps, video only, 70.91MiB
302          webm       1280x720   720p60 1285k , vp9, 60fps, video only, 54.62MiB
247          webm       1280x720   720p 1317k , vp9, 30fps, video only, 63.26MiB
136          mp4        1280x720   720p 1504k , avc1.4d401f, 30fps, video only, 61.72MiB
399          mp4        1920x1080  1080p60 2098k , av01.0.09M.08, 60fps, video only, 129.17MiB
248          webm       1920x1080  1080p 2326k , vp9, 30fps, video only, 117.54MiB
137          mp4        1920x1080  1080p 2722k , avc1.640028, 30fps, video only, 113.06MiB
299          mp4        1920x1080  1080p60 3376k , avc1.64002a, 60fps, video only, 129.37MiB
303          webm       1920x1080  1080p60 3662k , vp9, 60fps, video only, 178.52MiB
18           mp4        640x360    360p  441k , avc1.42001E, 30fps, mp4a.40.2@ 96k (44100Hz), 34.11MiB
22           mp4        1280x720   720p  927k , avc1.64001F, 30fps, mp4a.40.2@192k (44100Hz) (best)

Profit!
Downloading works as well:
screenshot1

But I think we can't expect users to do all this ;)

[Edit]

video comment:

When $10,000,000 is on the line, Mark and Ethan will stop at nothing(?) to find and capture America's Most Wanted Criminals. / ᴍᴇʀᴄʜ ➤ http://www.unusannus.com / ᴜɴᴜs ➣ http://www.youtube.com/crankgameplays / ᴀɴɴᴜs ➢ http://www.youtube.com/markiplier / / ᴛᴡɪᴛᴛᴇʀ ► http://www.twitter.com/unusannus / ɪɴsᴛᴀɢʀᴀᴍ ► http://www.instagram.com/unusannus / ʀᴇᴅᴅɪᴛ ► http://www.reddit.com/r/unusannus / ᴛᴜᴍʙʟʀ ► http://www.unusannus.tumblr.com / / Edited by ► https://bit.ly/2Mv5aTM / / This channel, along with every video that has or will ever be uploaded on this channel, will be deleted after our year has ended. This is inevitable. Inescapable. Irreversible. / / Do not archive or re-upload anything. This is our last wish. Our parting gift. Stay true to the purpose of our final year or we shall lay down wrath upon those that attempt to escape the end. / / Memento Mori. / / Unus Annus.

This channel, along with every video that has or will ever be uploaded on this channel, will be deleted after our year has ended. This is inevitable. Inescapable. Irreversible. / / Do not archive or re-upload anything. This is our last wish. Our parting gift. Stay true to the purpose of our final year or we shall lay down wrath upon those that attempt to escape the end.

Don't know what to expect next... ;)

@blackjack4494
Copy link
Owner

To come back to this issue again. I believe a fix would be to change the age restriction logic a bit. From what I remember it simply uses the embedded site then but not sure if cookies etc are used then or if they are even valid. It could be the case that mobile cookies differs from desktop cookies.

@blackjack4494
Copy link
Owner

@leedoyle
ytdl-org#26652

This is also on my list to fix. The new age gate fix basically broke this.

@blackjack4494 blackjack4494 added Future and removed low labels Sep 23, 2020
@someziggyman
Copy link

Hello guys,
Seems like I've found a solution!

20 days ago, on Sep 6, 2020 youtube-dl main repo had a commit related to age-gated content.
ytdl-org@16ee69c

So, in youtube_dl/extractor/youtube.py
I just removed the newly added 2 lines and returned the old one:
if re.search(r'player-age-gate-content">', video_webpage) is not None:

This way we can not by-pass geo restriction, but at least it works with --cookies parameter.
Worked for links like:
https://www.youtube.com/watch?v=MxCRYXg6hLs
https://www.youtube.com/watch?v=fh2cDKyFdyU

@blackjack4494
Copy link
Owner

@someziggyman Yeah as I said the new age gate fix broke the whole thing.

@Aronian
Copy link

Aronian commented Oct 5, 2020

@blackjack4494 Will this fix be featured in an upcoming release?

@blackjack4494
Copy link
Owner

blackjack4494 commented Oct 7, 2020

@Aronian @someziggyman @nicolaasjan just finished the fix.
9d9314c
Will be released soon.
If you cannot wait to test it you can download this branch

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants