Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Youtube] Got server error HTTP Error 403: Forbidden(latest master version) #32905

Open
6 tasks done
Linux2010 opened this issue Aug 8, 2024 · 75 comments
Open
6 tasks done

Comments

@Linux2010
Copy link

Linux2010 commented Aug 8, 2024

Checklist

  • I'm reporting a broken site support issue
  • I've verified that I'm running youtube-dl version 2024.08.07(the latest master version)
  • I've checked that all provided URLs are alive and playable in a browser
  • I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • I've searched the bugtracker for similar bug reports including closed ones
  • I've read bugs section in FAQ

Verbose log

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2024.08.07.local
[debug] Python 3.8.5 (CPython x86_64 64bit) - macOS-10.16-x86_64-i386-64bit - OpenSSL 1.1.1h  22 Sep 2020
[debug] exe versions: ffmpeg 7.0.1-tessus, ffprobe 6.1.1, rtmpdump 2.4
[debug] Proxy map: {'http': 'http://127.0.0.1:58309', 'https': 'http://127.0.0.1:58309'}
Usage: youtube-dl [OPTIONS] URL [URL...]

youtube-dl: error: You must provide at least one URL.
Type youtube-dl --help to see a list of all options.

PASTE VERBOSE LOG HERE

Description

  • The download will run normally for a short time, and then a 403 error will appear
youtube-dl 'https://www.youtube.com/watch?v=lLSkbZ3-EOs'
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 2 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 3 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 4 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 5 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 6 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 7 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 8 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 9 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 41 (attempt 10 of 10)...
[download] Skipping fragment 41...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 42 (attempt 1 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 42 (attempt 2 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 42 (attempt 3 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 42 (attempt 4 of 10)...
@Linux2010 Linux2010 changed the title reoporting bug :Got server HTTP error: HTTP Error 403: Forbidden(the least version) reoporting bug :Got server HTTP error: HTTP Error 403: Forbidden(the latest version) Aug 8, 2024
@Linux2010 Linux2010 changed the title reoporting bug :Got server HTTP error: HTTP Error 403: Forbidden(the latest version) reoporting bug :Got server HTTP error: HTTP Error 403: Forbidden(the latest master version) Aug 8, 2024
@dirkf
Copy link
Contributor

dirkf commented Aug 8, 2024

Indeed this seems to be a pathological video where almost all video formats fail on the first fragment and 299 may fail later, regardless of Python 2.7/3.5/3.9 and User-Agent settings.

yt-dlp 2024.08.06 still works, apparently. It has fancy networking that we can't easily replicate: maybe punt to curl for all requests?

@dirkf dirkf changed the title reoporting bug :Got server HTTP error: HTTP Error 403: Forbidden(the latest master version) [Youtube]: Got server error HTTP Error 403: Forbidden(latest master version) Aug 8, 2024
@dirkf dirkf changed the title [Youtube]: Got server error HTTP Error 403: Forbidden(latest master version) [Youtube] Got server error HTTP Error 403: Forbidden(latest master version) Aug 8, 2024
@tansy
Copy link

tansy commented Aug 8, 2024

I have that with every single video I try. Curiously enough format '18' work all the time. Other formats that work are '136', '137', '248' and/or '160', but it depends on video - not always the case. Still, format '18' is the most reliable to work.

@dirkf
Copy link
Contributor

dirkf commented Aug 8, 2024

So

  1. poToken (I agree, this is always being detected today) randomly breaks the download rather than uniformly giving 403 as with the revised n-sig "throttling"?
  2. different clients get more or less functional links for formats with the same itag?

@GregoriusT
Copy link

Can confirm that a lot of the video-only formats are just being 403-ed in the middle with their downloads, resulting in me getting files that stop after about 10-20 minutes into the video, but still have full sized audio.

By now I have written something into my scripts to just pick format 18 as long as a flag is set, because i foresee this issue happening again in the future once it is eventually fixed... >.>

@dirkf
Copy link
Contributor

dirkf commented Aug 8, 2024

So has anyone tried fetching fragments in fragments of <1MB? We already had a work-around to download in fragments to avoid throttling IIRC.

Otherwise:

  1. actually ignore the data with poToken and use the existing "punt to API" logic with a selected unafflicted client.
  2. extend to user-specified other, or selection of, clients similar to the yt-dlp extractor logic.

@iactix
Copy link

iactix commented Aug 8, 2024

Apparently the latest fix worked for not even a day, that doesn't bode well. Personally I keep getting "giving up after 0 fragment retries" in my python stuff.

From what I read in yesterdays thread, it seems like this will just not work out with fake JS interpretation if they try to combat this in the slightest. Like, that almost doesn't deserve the name attack vector, that's an attack landscape.

@tansy
Copy link

tansy commented Aug 9, 2024

This change is significant. I checked old, pre quantum Firefox and videos don't work any more, when 3 days ago they did.

@dirkf
Copy link
Contributor

dirkf commented Aug 9, 2024

Maybe the new player JS uses some G JS syntax extension (aka ECMA2021+) that hadn't been contemplated in those FF versions. Is there an error in the JS console?

@tansy
Copy link

tansy commented Aug 9, 2024

It used to work as embedded or as mobile (when used mobile user agent). Now all of them display all saying error:

An error occurred. Please try again later. (Playback ID: j-bZsC_YehYVyZZ8)
Learn More (https://support.google.com/youtube/?p=player_error1&hl=en)

Loading any video at https://www.youtube.com/embed/1234567890a:

Content Security Policy: Couldn't process unknown directive 'require-trusted-types-for' <unknown>
mutating the [[Prototype]] of an object will cause your code to run very slowly; instead create the object with the correct initial [[Prototype]] value using Object.create www-embed-player.js:26:77
InvalidStateError www-embed-player.js:1128:42

Then pressing play:

Error: WebGL: Error during native OpenGL init. base.js:11283:169
Error: WebGL: WebGL creation failed. base.js:11283:169

ED.

If it's of any help, despite what was said before, there are some videos that work.

First - this one doesn't, and gives following console log

Content Security Policy: Couldn't process unknown directive 'require-trusted-types-for' <unknown>
mutating the [[Prototype]] of an object will cause your code to run very slowly; instead create the object with the correct initial [[Prototype]] value using Object.create www-embed-player.js:26:77
InvalidStateError www-embed-player.js:1128:42
Empty string passed to getElementById(). zVhcVoOEv7o line 2 > eval:795:28944

Error: WebGL: getParameter: parameter: invalid enum value <enum 0x9246> base.js:11283:254

This one does and, with this log:

Content Security Policy: Couldn't process unknown directive 'require-trusted-types-for' <unknown>
mutating the [[Prototype]] of an object will cause your code to run very slowly; instead create the object with the correct initial [[Prototype]] value using Object.create www-embed-player.js:26:77
InvalidStateError www-embed-player.js:1128:42
Empty string passed to getElementById(). KAR4fAX5T7Y line 2 > eval:4650:33869

So, after clicking 'play' it gives this error: Error: WebGL: getParameter: parameter: invalid enum value <enum 0x9246> base.js:11283:254.

@TheHolyRoger
Copy link

TheHolyRoger commented Aug 9, 2024

No matter what chunk size I use I'm seeing hard 403 errors at 1Meg as others have reported - I'm able to download as many fragments as I want up to 1Meg and then get a 403.

Have experimented with generating cpns (nonces) and adding them to the format fragment URLs without any luck as well as using rn (request numbers) in the URL query instead of byte ranges. Have tried sleeping between fragments to mimick video playback also without joy.

It feels like they've added a check somewhere which fails at the 1Meg mark but I haven't found anything yet where that might be.

Checking via the browser I can see that youtube is happily downloading /videoplayback fragment URLs above 1Meg without any issue...

@dirkf
Copy link
Contributor

dirkf commented Aug 9, 2024

But in the browser the media links have the pot parameter with its poToken challenge result, no? Which is what we can't haz.

In line with step 1 above, I'm gradually pulling stuff from the yt-dlp extractor, enough to download HLS with client ios, but plainly not yet enough to get unblocked links from tv or web_creator, eg with format 135. Should I be expecting that?

@TheHolyRoger
Copy link

TheHolyRoger commented Aug 9, 2024

But in the browser the media links have the pot parameter with its poToken challenge result, no? Which is what we can't haz.

In line with step 1 above, I'm gradually pulling stuff from the yt-dlp extractor, enough to download HLS with client ios, but plainly not yet enough to get unblocked links from tv or web_creator, eg with format 135. Should I be expecting that?

I'm not seeing a pot parameter in the query strings, I am seeing post data in the /videoplayback requests which is referred to in the source as playbackCookie

Edit: Looks like the playbackCookie / POST data is extracted from the bytes of the previous fragment response somehow

@8chanAnon
Copy link

This is the procedure that I am using in my own code.

Load https://www.youtube.com/embed/<id#> and find the base.js link. Do the usual to extract the sig and n-sig. Extract the signatureTimestamp for the next step.

Load https://www.youtube.com/youtubei/v1/player with the signatureTimestamp and TVHTML5_SIMPLY_EMBEDDED_PLAYER as the client name.

If the JSON response contains "formats" and/or "adaptiveFormats" then we're good. This covers most videos, including age-gated ones. The 403 problem occurs when we have go to the next step. We can't use "www.youtube.com". We must use "m.youtube.com" with the user agent set to something like "Mozilla/5.0 (Android 14)" which is what I'm using.

Load https://m.youtube.com/watch?v=<id#> and extract the JSON structures that you would otherwise have gotten from the previous step.

And that's it. The extra step is only required for videos that disallow embedding.

@dirkf
Copy link
Contributor

dirkf commented Aug 10, 2024

Please don't bother to supply any "me too" reports unless the log shows some novelty that may help with rectification. Just "Like", or whatever, an existing similar report.

You can see how a poToken is being sent in POST data by the browser in the Invidious code that shows how to capture the value. But I understood from yt-dlp discussions that a pot query parameter was used in the media links associated with the pot-ified session.

@8chanAnon's algorithm is what is currently done for age-gate videos, up to the last step with m.youtube.com which is new and interesting. What happens if you skip straight to that step?

Step 2 will only work if TVHTML5_SIMPLY_EMBEDDED_PLAYER is not pot-ified, and that seems to be in question.

@dirkf
Copy link
Contributor

dirkf commented Aug 10, 2024

Indeed, Android 14/FF 122 at m.youtube.com didn't list the poToken experiment IDs although yt-dlp has reported unsuccess with Android clients.

@TheHolyRoger
Copy link

TheHolyRoger commented Aug 10, 2024

I don't think it's worth trying to get around the poToken, it will eventually be required in all clients.

I keep digging into base.js when I get some time trying to understand how the token is created, it does seem to be extracted from the bytes of at least the first video fragment as far as I can tell, but not all fragments?...

There's a Uint8Array which appears to be the fragment response data?... manipulated several times and then 82/84/68 bytes of that array are stored as playbackCookie which is then sent in the POST data

@dirkf
Copy link
Contributor

dirkf commented Aug 10, 2024

At least it would be good to have a program that is not not-youtube-dl while a long term solution to the twattery is being investigated.

@PatrickJRed
Copy link

PatrickJRed commented Aug 10, 2024

@dirkf

yt-dlp 2024.08.06 still works, apparently. It has fancy networking that we can't easily replicate: maybe punt to curl for all requests?

On a lot of websites I want to download from youtube-dl and curl dont even get the correct html,
instead of the one i would get in my Browser , they receives a version that has the captcha.

Instead I have a bash-script that predownloads the non-captcha html via "https://github.com/lwthiker/curl-impersonate" (runs in docker container , i use tag: 0.5.2-ff-alpine)

AFAIK lwthiker/curl-impersonate is the only http client that completly impersonates an actual browser like firefox.
A lot of the problems I had with 403 errors in youtube-dl where captchas triggered by http-client not being exactly like an official version of firefox or chrome :) (this might even be valid for fragment-downloads)

@dirkf
Copy link
Contributor

dirkf commented Aug 10, 2024

Yes, but so far as captcha is generally understood (G/recaptcha, hacaptcha, Cloudflare challenge aka breaks the Web), that is not the problem. Even if it solved the poToken issue, a dependency running under Docker would not be an acceptable solution for the main functionality of the program, though it might be a PoC for a solution.

@TheHolyRoger
Copy link

Yeah not relevant here but I did bookmark it for other things, looks like a decent tool :)

@PatrickJRed
Copy link

I only said that i run it in docker.
Apparently it can be used as a library, see https://github.com/lwthiker/curl-impersonate?tab=readme-ov-file#Advanced-usage .
Though I have not looked into that as I run youtube-dl and almost any other apps via docker anyway :)

@seproDev
Copy link

Because someone needs to take the time and port the improvements made to yt-dlp, over to youtube-dl.
yt-dlp already switched away from the web client ~1 month ago and even before that used iOS as a secondary client.
If you require the Python 2 compatibility, you will have to wait for these improvements to be ported to youtube-dl (or even contribute something yourself).

@dirkf
Copy link
Contributor

dirkf commented Aug 25, 2024

@8chanAnon's heart is in the right place to judge from Alleycat BBS, but time spent understanding the context of an issue is rarely wasted.

Part 1 of the program above is working. That also opens the way to part 2 (multiple clients) but of course if pottery should be applied to all the InnerTube clients further invention would be needed.

@DManstrator
Copy link

The mobile UA isn't working with playlist pages and tabs.

I learned that the hard way in the last one hour investigating it. Having Mozilla/5.0 (Android 14; Mobile; rv:115.0) Gecko/115.0 Firefox/115.0 as the UA (as explained above) leads to an exception when performing a regex search.

Traceback (most recent call last):
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 879, in wrapper
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\YoutubeDL.py", line 975, in __extract_info
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\extractor\common.py", line 571, in extract
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\extractor\youtube.py", line 3472, in _real_extract
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\extractor\youtube.py", line 311, in _extract_yt_initial_data
  File "D:\a\ytdl-nightly\ytdl-nightly\youtube_dl\extractor\common.py", line 1101, in _search_regex
youtube_dl.utils.RegexNotFoundError: Unable to extract yt initial data; please [...]

However when completely removing the uppercase letters in the parenthesis (Mozilla/5.0 (ndroid 14; obile; rv:115.0) Gecko/115.0 Firefox/115.0), the playlist data (name, amount of videos, etc.) can be retrieved but each video download fails with the expected HTTP Error 403: Forbidden.

Not sure if this is the actual reason it's not supposed to work.

@dirkf dirkf mentioned this issue Sep 10, 2024
5 tasks
@TLINDEN
Copy link

TLINDEN commented Sep 11, 2024

Anything new on this?

I am using the latest snapshot and it's still not working, I'm getting the same errors as in this issue.

In addition other formats such as 140 or 18 aren't working as well. I'm getting the Unable to decode n-parameter there.

And interestingly, yt-dlp 2024.04.09 is working.

Edited to add:

This is the commandline I'm using:

youtube-dl --restrict-filenames -o "%(uploader)s-%(title)s-%(id)s-%(upload_date)s.%(ext)s" --cookies www.youtube.com_cookies.txt https://www.youtube.com/watch?v=650lXWfoPN8

I created the cookies file using the chrome extension "Get cookies.txt" right before I executed the command after visiting the uri I was about to download.

@xyzxye
Copy link

xyzxye commented Sep 23, 2024

@TLINDEN thank you, yt-dlp 2024.04.09 can download normally, but not for all videos.
some videos still fail ERROR: unable to download video data: HTTP Error 403: Forbidden

It's been a long time, I don't know if this problem can still be solved. :(

@seproDev
Copy link

@xyzxye

yt-dlp 2024.04.09

Update to 2024.08.06 or later. If the issue persists open an issue on the yt-dlp issue tracker.

@xyzxye
Copy link

xyzxye commented Sep 23, 2024

@3052 https://www.youtube.com/watch?v=VaSV4NtZCXU

I have successfully downloaded this using the latest version of yt-dlp

@xyzxye

yt-dlp 2024.04.09

Update to 2024.08.06 or later. If the issue persists open an issue on the yt-dlp issue tracker.

@dirkf
Copy link
Contributor

dirkf commented Sep 25, 2024

Please confine comments to relevant matters and questions of fact. What anyone thinks about a member's behaviour belongs elsewhere.

@dirkf
Copy link
Contributor

dirkf commented Sep 25, 2024

Not "just" asking but imputing motives, which is not relevant.

yt-dlp reports may well offer good evidence of YT site behaviour (and its YT extractor can test the known InnerTube clients). There is no adverse competition with yt-dlp since each project has its own goals and code is compatibly (un)licensed.

For avoidance of doubt, that is the end of this meta-discussion.

@3052
Copy link

3052 commented Sep 27, 2024

For avoidance of doubt, that is the end of this meta-discussion.

fair enough. lets go ahead and put some data on the table. here is a test:

https://github.com/3052/youtube/blob/v1.0.2/get_watch_test.go

it calls /youtubei/v1/get_watch (the replacement for /youtubei/v1/player) 99 times, with no po token, with one second delay between each call. I ran it just now, with all calls from the same IP. no flags, no blocks. thats what is wrong with comments like this:

Have a look at some of the pinned yt-dlp issues if you want examples of why making repeat player endpoint requests is not viable. Youtube is flagging/blocking IP addresses that aren't using PO tokens when required (among other reasons)

this isnt the YT-DLP tracker. if user want to be taken seriously, they should be including evidence HERE to back up their claims. the comment is incomplete at best, and misleading at worst. in my mind these type of comments are worse than spam, and should not be allowed in a technical discussion.

@dirkf
Copy link
Contributor

dirkf commented Sep 28, 2024

One area where there could be differences is the https implementation in use, urllib2/OpenSSL 1 (here) vs urllib3/OpenSSL (n?) or other (yt-dlp) vs whatever the Go runtime library uses (@3052)

@113Buka
Copy link

113Buka commented Sep 29, 2024

Hello, I want to ask about any updates on this issue.
I tried to download a youtube video today and I'm getting the same error.
I'm using youtube-dl nightly, updated to the latest 2024.08.07 version.

cookies were exported using EditThisCookie for Chrome

youtube-dl https://youtu.be/vGA4nXgq6PI --cookies cookies-youtube-com.txt --ffmpeg-location "ffmpeg\bin"

@dirkf
Copy link
Contributor

dirkf commented Sep 29, 2024

Like yt-dlp, yt-dl already applies 10MB chunking.

@shamilovtim
Copy link

Is there an explanation of the bug in a way that doesn't require hours of YouTube API context to understand? For me this simple example fails: youtube-dl 'https://www.youtube.com/watch?v=LbOve_UZZ54'.

@dirkf
Copy link
Contributor

dirkf commented Oct 13, 2024

A tldr; may not be out of place, even if all this can be read above.

Many programs, including this one, use an internal YT API to provide services that cannot be supported through the official YT API. The internal API has been changed such that media links extracted using the WEB client (which, for now, we use) require a further challenge (the poToken) to be solved to avoid 403 after the first ~1MB. This (with similar things) is why a YT web page is several MB instead of 10kB. Using m.youtube.com may be a work-around until after "for now".

Fixing this in line with the long established multi-client capability in yt-dlp is resulting in a long twisty set of commits that are not yet quite ready for PR.

@3052
Copy link

3052 commented Oct 13, 2024

its not just poToken/pot though.

some well known clients, such as 1/WEB, 3/ANDROID, 7/TVHTML5, 18/ANDROID_KIDS, 21/ANDROID_MUSIC are making POST /videoplayback requests with a ProtoBuf body, and getting application/vnd.yt-ump responses instead of video/webm or similar. YouTube-DL and even YT-DLP haven't touched Ump at all, which is concerning because while /videoplayback still allows GET requests, none of the 83 YouTube clients I looked at actually use a GET request any more.

https://github.com/LuanRT/googlevideo

@Chen-Alexander
Copy link

即使上面已经读完了这些内容,tldr; 也可能不会不合适。

许多程序(包括此程序)使用内部 YT API 来提供无法通过官方 YT API 支持的服务。内部 API 已更改,因此使用WEB客户端(目前我们使用)提取的媒体链接需要进一步poToken解决挑战( ),以避免在前 ~1MB 后出现 403 错误。这(以及类似的东西)就是为什么 YT 网页是几 MB 而不是 10kB。在“目前”之前,使用 m.youtube.com 可能是一种解决方法。

根据_yt-dlp_中长期建立的多客户端功能修复此问题,会导致出现一长串曲折的提交,这些提交尚未完全准备好进行 PR。

come on,baby, we need your solution, hurry, very thanks!

@nicolaasjan
Copy link

Is there an explanation of the bug in a way that doesn't require hours of YouTube API context to understand? For me this simple example fails: youtube-dl 'https://www.youtube.com/watch?v=LbOve_UZZ54'.

Try with m.youtube.com or pass:
--user-agent "Mozilla/5.0 (Android 14; Mobile; rv:128.0) Gecko/128.0 Firefox/128.0".

Works (for now...)

youtube-dl 'https://m.youtube.com/watch?v=LbOve_UZZ54'
[youtube] LbOve_UZZ54: Downloading webpage
[youtube] LbOve_UZZ54: Downloading player 2f238d39
[youtube] LbOve_UZZ54: Downloading API JSON
[youtube] LbOve_UZZ54: Downloading thumbnail ...
[youtube] LbOve_UZZ54: Writing thumbnail to: /dev/shm/test-ytd/По полям Синий трактор едет к нам - Песенка мультик для детей.jpg
[dashsegments] Total fragments: 9
[download] Destination: /dev/shm/test-ytd/По полям Синий трактор едет к нам - Песенка мультик для детей.f137.mp4
[download] 100% of 88.64MiB in 00:03
[dashsegments] Total fragments: 1
[download] Destination: /dev/shm/test-ytd/По полям Синий трактор едет к нам - Песенка мультик для детей.m4a.f140.m4a
[download] 100% of 8.52MiB in 00:00
[ffmpeg] Merging formats into "/dev/shm/test-ytd/По полям Синий трактор едет к нам - Песенка мультик для детей.mp4"
Deleting original file /dev/shm/test-ytd/По полям Синий трактор едет к нам - Песенка мультик для детей.f137.mp4 (pass -k to keep)
Deleting original file /dev/shm/test-ytd/По полям Синий трактор едет к нам - Песенка мультик для детей.m4a.f140.m4a (pass -k to keep)
[ffmpeg] Adding thumbnail to "/dev/shm/test-ytd/По полям Синий трактор едет к нам - Песенка мультик для детей.mp4"

@themaddoctor
Copy link

Using m.youtube.com worked only for about 20%, then it starting giving the 403 error again.

@rubensamuelJesus
Copy link

Is there an explanation of the bug in a way that doesn't require hours of YouTube API context to understand? For me this simple example fails: youtube-dl 'https://www.youtube.com/watch?v=LbOve_UZZ54'.

Try with m.youtube.com or pass: --user-agent "Mozilla/5.0 (Android 14; Mobile; rv:128.0) Gecko/128.0 Firefox/128.0".

Works (for now...)

youtube-dl 'https://m.youtube.com/watch?v=LbOve_UZZ54'
[youtube] LbOve_UZZ54: Downloading webpage
[youtube] LbOve_UZZ54: Downloading player 2f238d39
[youtube] LbOve_UZZ54: Downloading API JSON
[youtube] LbOve_UZZ54: Downloading thumbnail ...
[youtube] LbOve_UZZ54: Writing thumbnail to: /dev/shm/test-ytd/По полям Синий трактор едет к нам - Песенка мультик для детей.jpg
[dashsegments] Total fragments: 9
[download] Destination: /dev/shm/test-ytd/По полям Синий трактор едет к нам - Песенка мультик для детей.f137.mp4
[download] 100% of 88.64MiB in 00:03
[dashsegments] Total fragments: 1
[download] Destination: /dev/shm/test-ytd/По полям Синий трактор едет к нам - Песенка мультик для детей.m4a.f140.m4a
[download] 100% of 8.52MiB in 00:00
[ffmpeg] Merging formats into "/dev/shm/test-ytd/По полям Синий трактор едет к нам - Песенка мультик для детей.mp4"
Deleting original file /dev/shm/test-ytd/По полям Синий трактор едет к нам - Песенка мультик для детей.f137.mp4 (pass -k to keep)
Deleting original file /dev/shm/test-ytd/По полям Синий трактор едет к нам - Песенка мультик для детей.m4a.f140.m4a (pass -k to keep)
[ffmpeg] Adding thumbnail to "/dev/shm/test-ytd/По полям Синий трактор едет к нам - Песенка мультик для детей.mp4"

Am I doing something wrong
I also updated youtube-dl too

youtube-dl -f mp4 --verbose -x --user-agent "Mozilla/5.0 (Android 14; Mobile; rv:128.0) Gecko/128.0 Firefox/128.0" 'https://m.youtube.com/watch?v=LbOve_UZZ54'
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-f', 'mp4', '--verbose', '-x', '--user-agent', 'Mozilla/5.0 (Android 14; Mobile; rv:128.0) Gecko/128.0 Firefox/128.0', 'https://m.youtube.com/watch?v=LbOve_UZZ54']
[debug] Encodings: locale utf-8, fs utf-8, out utf-8, pref utf-8
[debug] youtube-dl version 2021.12.17
[debug] Python version 3.13.0 (CPython) - Linux-5.15.0-105-generic-x86_64-with-glibc2.35
[debug] exe versions: none
[debug] Proxy map: {}
[youtube] LbOve_UZZ54: Downloading webpage
WARNING: unable to extract uploader id; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
[youtube] LbOve_UZZ54: Downloading API JSON
[debug] Invoking downloader on 'https://rr1---sn-2vgu0b5auxaxjvh-v2vd.googlevideo.com/videoplayback?expire=1729224414&ei=fooRZ9PIC_GAp-oP_crs0Ao&ip=2001%3A8a0%3Af3a6%3A8100%3A1e83%3A41ff%3Afe29%3A1d91&id=o-AHnThXNRVL9SdV6OygCOobLCDp3AtzEmHmOUQ-BpnL6o&itag=18&source=youtube&requiressl=yes&xpc=EgVo2aDSNQ%3D%3D&met=1729202814%2C&mh=2f&mm=31%2C29&mn=sn-2vgu0b5auxaxjvh-v2vd%2Csn-apn7en7l&ms=au%2Crdu&mv=m&mvi=1&pcm2cms=yes&pl=42&rms=au%2Cau&initcwndbps=1743750&bui=AXLXGFRZFjzR-xKVPG3B86v6RLkdRd7mnWmyUZ9WOrnNmr0-0F_mgnXBWUU0GwbjVDYVW6CxDUAgTOEa&spc=54MbxdhXRaSc5NgWq9V_5B14KKxTQwu1P3TV4mQnjpL91V1De8V22QRrbFE3LA3RCw&vprv=1&svpuc=1&mime=video%2Fmp4&ns=4jZtd34kFDwO0505g9srxBEQ&rqh=1&cnr=14&ratebypass=yes&dur=552.263&lmt=1726441655211820&mt=1729202495&fvip=4&fexp=51300761%2C51312688&c=MWEB&sefc=1&txp=4538434&n=GREd-dtXOFwZUAt&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cxpc%2Cbui%2Cspc%2Cvprv%2Csvpuc%2Cmime%2Cns%2Crqh%2Ccnr%2Cratebypass%2Cdur%2Clmt&sig=AJfQdSswRQIgVI7wbB6WR3yE2qYXBPYSgFzFfc6HglYCEJzdkharq2wCIQCHqzlQ1kbTw9M25PLt_uU1SIPwzhMjDYqo3CSPN2MVZQ%3D%3D&lsparams=met%2Cmh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpcm2cms%2Cpl%2Crms%2Cinitcwndbps&lsig=ACJ0pHgwRQIhANCLfBEA4J4DhPNTf_mCT8rsAiqfl-ctNu-w49cKx9xSAiBauqb-P1ESFQAyeZa0kZ33Ce6nAUbHpoj9HWh43udygQ%3D%3D'
ERROR: unable to download video data: HTTP Error 403: Forbidden
Traceback (most recent call last):
File "/home/linuxbrew/.linuxbrew/Cellar/youtube-dl/2021.12.17/libexec/lib/python3.13/site-packages/youtube_dl/YoutubeDL.py", line 1976, in process_info
success = dl(filename, info_dict)
File "/home/linuxbrew/.linuxbrew/Cellar/youtube-dl/2021.12.17/libexec/lib/python3.13/site-packages/youtube_dl/YoutubeDL.py", line 1915, in dl
return fd.download(name, info)
~~~~~~~~~~~^^^^^^^^^^^^
File "/home/linuxbrew/.linuxbrew/Cellar/youtube-dl/2021.12.17/libexec/lib/python3.13/site-packages/youtube_dl/downloader/common.py", line 366, in download
return self.real_download(filename, info_dict)
~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
File "/home/linuxbrew/.linuxbrew/Cellar/youtube-dl/2021.12.17/libexec/lib/python3.13/site-packages/youtube_dl/downloader/http.py", line 351, in real_download
establish_connection()
~~~~~~~~~~~~~~~~~~~~^^
File "/home/linuxbrew/.linuxbrew/Cellar/youtube-dl/2021.12.17/libexec/lib/python3.13/site-packages/youtube_dl/downloader/http.py", line 116, in establish_connection
raise err
File "/home/linuxbrew/.linuxbrew/Cellar/youtube-dl/2021.12.17/libexec/lib/python3.13/site-packages/youtube_dl/downloader/http.py", line 110, in establish_connection
ctx.data = self.ydl.urlopen(request)
~~~~~~~~~~~~~~~~^^^^^^^^^
File "/home/linuxbrew/.linuxbrew/Cellar/youtube-dl/2021.12.17/libexec/lib/python3.13/site-packages/youtube_dl/YoutubeDL.py", line 2288, in urlopen
return self._opener.open(req, timeout=self._socket_timeout)
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/linuxbrew/.linuxbrew/opt/[email protected]/lib/python3.13/urllib/request.py", line 495, in open
response = meth(req, response)
File "/home/linuxbrew/.linuxbrew/opt/[email protected]/lib/python3.13/urllib/request.py", line 604, in http_response
response = self.parent.error(
'http', request, response, code, msg, hdrs)
File "/home/linuxbrew/.linuxbrew/opt/[email protected]/lib/python3.13/urllib/request.py", line 533, in error
return self._call_chain(*args)
~~~~~~~~~~~~~~~~^^^^^^^
File "/home/linuxbrew/.linuxbrew/opt/[email protected]/lib/python3.13/urllib/request.py", line 466, in _call_chain
result = func(*args)
File "/home/linuxbrew/.linuxbrew/opt/[email protected]/lib/python3.13/urllib/request.py", line 613, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

youtube-dl 'https://m.youtube.com/watch?v=LbOve_UZZ54'
[youtube] LbOve_UZZ54: Downloading webpage
WARNING: unable to extract uploader id; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
ERROR: unable to download video data: HTTP Error 403: Forbidden

@nicolaasjan
Copy link

[debug] youtube-dl version 2021.12.17

Use the nightly version (is built from current source):
https://github.com/ytdl-org/ytdl-nightly

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests