-
Notifications
You must be signed in to change notification settings - Fork 10k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Youtube] Got server error HTTP Error 403: Forbidden(latest master version) #32905
Comments
Indeed this seems to be a pathological video where almost all video formats fail on the first fragment and 299 may fail later, regardless of Python 2.7/3.5/3.9 and User-Agent settings. yt-dlp 2024.08.06 still works, apparently. It has fancy networking that we can't easily replicate: maybe punt to curl for all requests? |
I have that with every single video I try. Curiously enough format '18' work all the time. Other formats that work are '136', '137', '248' and/or '160', but it depends on video - not always the case. Still, format '18' is the most reliable to work. |
So
|
Can confirm that a lot of the video-only formats are just being 403-ed in the middle with their downloads, resulting in me getting files that stop after about 10-20 minutes into the video, but still have full sized audio. By now I have written something into my scripts to just pick format 18 as long as a flag is set, because i foresee this issue happening again in the future once it is eventually fixed... >.> |
So has anyone tried fetching fragments in fragments of <1MB? We already had a work-around to download in fragments to avoid throttling IIRC. Otherwise:
|
Apparently the latest fix worked for not even a day, that doesn't bode well. Personally I keep getting "giving up after 0 fragment retries" in my python stuff. From what I read in yesterdays thread, it seems like this will just not work out with fake JS interpretation if they try to combat this in the slightest. Like, that almost doesn't deserve the name attack vector, that's an attack landscape. |
This change is significant. I checked old, pre quantum Firefox and videos don't work any more, when 3 days ago they did. |
Maybe the new player JS uses some G JS syntax extension (aka ECMA2021+) that hadn't been contemplated in those FF versions. Is there an error in the JS console? |
It used to work as embedded or as mobile (when used mobile user agent). Now all of them display all saying error:
Loading any video at https://www.youtube.com/embed/1234567890a:
Then pressing play:
ED. If it's of any help, despite what was said before, there are some videos that work. First - this one doesn't, and gives following console log
This one does and, with this log:
So, after clicking 'play' it gives this error: |
No matter what chunk size I use I'm seeing hard 403 errors at 1Meg as others have reported - I'm able to download as many fragments as I want up to 1Meg and then get a 403. Have experimented with generating It feels like they've added a check somewhere which fails at the 1Meg mark but I haven't found anything yet where that might be. Checking via the browser I can see that youtube is happily downloading |
But in the browser the media links have the In line with step 1 above, I'm gradually pulling stuff from the yt-dlp extractor, enough to download HLS with client |
I'm not seeing a Edit: Looks like the playbackCookie / POST data is extracted from the bytes of the previous fragment response somehow |
This is the procedure that I am using in my own code. Load https://www.youtube.com/embed/<id#> and find the base.js link. Do the usual to extract the sig and n-sig. Extract the signatureTimestamp for the next step. Load https://www.youtube.com/youtubei/v1/player with the signatureTimestamp and TVHTML5_SIMPLY_EMBEDDED_PLAYER as the client name. If the JSON response contains "formats" and/or "adaptiveFormats" then we're good. This covers most videos, including age-gated ones. The 403 problem occurs when we have go to the next step. We can't use "www.youtube.com". We must use "m.youtube.com" with the user agent set to something like "Mozilla/5.0 (Android 14)" which is what I'm using. Load https://m.youtube.com/watch?v=<id#> and extract the JSON structures that you would otherwise have gotten from the previous step. And that's it. The extra step is only required for videos that disallow embedding. |
Please don't bother to supply any "me too" reports unless the log shows some novelty that may help with rectification. Just "Like", or whatever, an existing similar report. You can see how a @8chanAnon's algorithm is what is currently done for age-gate videos, up to the last step with Step 2 will only work if |
Indeed, Android 14/FF 122 at m.youtube.com didn't list the |
I don't think it's worth trying to get around the poToken, it will eventually be required in all clients. I keep digging into There's a |
At least it would be good to have a program that is not not-youtube-dl while a long term solution to the twattery is being investigated. |
On a lot of websites I want to download from youtube-dl and curl dont even get the correct html, Instead I have a bash-script that predownloads the non-captcha html via "https://github.com/lwthiker/curl-impersonate" (runs in docker container , i use tag: 0.5.2-ff-alpine) AFAIK lwthiker/curl-impersonate is the only http client that completly impersonates an actual browser like firefox. |
Yes, but so far as captcha is generally understood (G/recaptcha, hacaptcha, Cloudflare challenge aka breaks the Web), that is not the problem. Even if it solved the |
Yeah not relevant here but I did bookmark it for other things, looks like a decent tool :) |
I only said that i run it in docker. |
Because someone needs to take the time and port the improvements made to yt-dlp, over to youtube-dl. |
@8chanAnon's heart is in the right place to judge from Alleycat BBS, but time spent understanding the context of an issue is rarely wasted. Part 1 of the program above is working. That also opens the way to part 2 (multiple clients) but of course if pottery should be applied to all the InnerTube clients further invention would be needed. |
I learned that the hard way in the last one hour investigating it. Having
However when completely removing the uppercase letters in the parenthesis ( Not sure if this is the actual reason it's not supposed to work. |
Anything new on this? I am using the latest snapshot and it's still not working, I'm getting the same errors as in this issue. In addition other formats such as 140 or 18 aren't working as well. I'm getting the And interestingly, Edited to add: This is the commandline I'm using:
I created the cookies file using the chrome extension "Get cookies.txt" right before I executed the command after visiting the uri I was about to download. |
@TLINDEN thank you, It's been a long time, I don't know if this problem can still be solved. :( |
Update to 2024.08.06 or later. If the issue persists open an issue on the yt-dlp issue tracker. |
@3052 https://www.youtube.com/watch?v=VaSV4NtZCXU I have successfully downloaded this using the latest version of yt-dlp
|
Please confine comments to relevant matters and questions of fact. What anyone thinks about a member's behaviour belongs elsewhere. |
Not "just" asking but imputing motives, which is not relevant. yt-dlp reports may well offer good evidence of YT site behaviour (and its YT extractor can test the known InnerTube clients). There is no adverse competition with yt-dlp since each project has its own goals and code is compatibly (un)licensed. For avoidance of doubt, that is the end of this meta-discussion. |
fair enough. lets go ahead and put some data on the table. here is a test: https://github.com/3052/youtube/blob/v1.0.2/get_watch_test.go it calls
this isnt the YT-DLP tracker. if user want to be taken seriously, they should be including evidence HERE to back up their claims. the comment is incomplete at best, and misleading at worst. in my mind these type of comments are worse than spam, and should not be allowed in a technical discussion. |
One area where there could be differences is the https implementation in use, urllib2/OpenSSL 1 (here) vs urllib3/OpenSSL (n?) or other (yt-dlp) vs whatever the Go runtime library uses (@3052) |
Hello, I want to ask about any updates on this issue. cookies were exported using EditThisCookie for Chrome
|
Like yt-dlp, yt-dl already applies 10MB chunking. |
Is there an explanation of the bug in a way that doesn't require hours of YouTube API context to understand? For me this simple example fails: |
A tldr; may not be out of place, even if all this can be read above. Many programs, including this one, use an internal YT API to provide services that cannot be supported through the official YT API. The internal API has been changed such that media links extracted using the Fixing this in line with the long established multi-client capability in yt-dlp is resulting in a long twisty set of commits that are not yet quite ready for PR. |
its not just some well known clients, such as 1/WEB, 3/ANDROID, 7/TVHTML5, 18/ANDROID_KIDS, 21/ANDROID_MUSIC are making |
come on,baby, we need your solution, hurry, very thanks! |
Try with Works (for now...)
|
Using m.youtube.com worked only for about 20%, then it starting giving the 403 error again. |
Am I doing something wrong youtube-dl -f mp4 --verbose -x --user-agent "Mozilla/5.0 (Android 14; Mobile; rv:128.0) Gecko/128.0 Firefox/128.0" 'https://m.youtube.com/watch?v=LbOve_UZZ54' youtube-dl 'https://m.youtube.com/watch?v=LbOve_UZZ54' |
Use the nightly version (is built from current source): |
Checklist
Verbose log
Description
youtube-dl 'https://www.youtube.com/watch?v=lLSkbZ3-EOs'
The text was updated successfully, but these errors were encountered: