[YT] Throttling of (60fps) streams

[litetex]

Checklist

• I am able to reproduce the bug with the latest version.
• I made sure that there are no existing issues - open or closed - which I could contribute my information to.
• I have taken the time to fill in all the required details. I understand that the bug report will be dismissed otherwise.
• This issue contains only one bug.
• I have read and understood the contribution guidelines.

Affected version

0.22.2 / dev

Steps to reproduce the bug

This is sometimes quite hard to reproduce reliable but I can reproduce it doing the following:

0. Clearing all caches and stop the app
1. Open a YT video with 60fps and quality >720p - it might work normally but let it run for at least 1min.
2. Open another YT video with 60fps
3. Notice that it (initially) loads forever and the network speed is at ~33kB/s.

I can reproduce the issue on multiple phones/emulators with the current released version (for 60fps video streams) and the current dev branch (happens on ALL video streams).

Expected behavior

No throttling occurs

Actual behavior

YT (video-only) streams are (initially) throttled to ~33kB/s = ~256kBit/s (suspicious number since this is exactly or very close to 2^8)

Screenshots/Screen recordings

Example video:
https://www.youtube.com/watch?v=hWihEDctPKA

Version 0.22.2

Huawei P20 lite:

Android studio network inspector with emulator:

Dev-Branch
Android emulator:

Affected Android/Custom ROM version

doesn't matter

Affected device model

doesn't matter

Additional information

After some digging down we believe that the problem is likely caused by a ExoPlayer bug or bad combination with YT:

• [YT] Throttling of (60fps) streams #8238 (comment)
• [YT] Throttling of (60fps) streams #8238 (comment)

[litetex]

My guesses what could cause this problem/Maybe where we can look further into:

• The de-throttling function fails
  • Check the returned values
• (0.22.2) Maybe 60fps streams are provided by a different source?
  • Check the sources
• (0.23.0/dev-branch) We change a lot of code (looking at [YouTube] Add more parameters to InnerTube requests, use the iOS client for livestreams and fix extraction of embeddable age-restricted videos and contents with a warning before playback NewPipeExtractor#780) maybe there is a bug inside there that causes all video streams to become throttled
• Is there is a bug inside Exoplayer?
  • Does the problem also occur when downloading videos?
  • the bug is worsened by updating ExoPlayer ExoPlayer 2.17.1 update and MediaSource management rework #8020
  • the bug is triggered by the streaming data which appears to have changed between 0.22.0 and the current dev-Branch

[lychnecross]

I was able to recreate this on my Pixel 4XL using 0.22.2. TiA4f8R's delivery-methods-v2 debug build with a playback load interval size of 16KiB works fine for me (no throttling). Not sure if it is the smaller playback load interval or the other changes in that build.

[AudricV]

Not sure if it is the smaller playback load interval

No, it is not as this setting only applies to progressive sources, here I am now using DASH to fetch YouTube contents (using the same way as YouTube + spoofing clients).

YouTube is known to throttle chunk sizes > 10MB (I will let you search this on the youtube-dl repository) and by default, ExoPlayer uses a 10MB chunk at initialization (and that's what #7919 should allow us to control).

YouTube clients are not using also this way to fetch contents, they are only using DASH (except for legacy streams (itags 17, 18 and 22), which have more tolerance in terms of bandwidth and throttling (related to the ratebypass parameter?), used right now for 720p and 360p qualities, but this has been changed with #7349 to keep a seamless transition), we are right now using progressive HTTP.

Finally, we are now using streams from the WEB client first, instead of the ANDROID ones (it wasn't the case since 0.21.8 but it is now the case because of a requested change by litetex (I should have noticed that using a HashMap will break the order of streams and use an ArrayList instead) (I am not blaming at all litetex who just flagged a improvement which can be made)).

[litetex]

I tested the Extractor a bit and I found some interesting things:

1. (Notice) Currently (despite HashMaps) the Android-endpoint is used first, then web, then iOS, ...
   • The resolution is very weird (HashMaps with weird keys like CPN parameters and URLs) and I think this should be refactored
2. (Notice) Even if the stream is not added/used (containsSimilarStream) later the de-throttling function is executed on them
3. (Notice) None of the streams I saw has an active cipher (was added in Signature cipher extraction failing NewPipeExtractor#327) we might be able to remove that code
4. Video-streams (with audio) are definitely a lot faster (download with 1-1.5MB/s) than video-only-streams (~500KB/s), I can reproduce this when I open the returned links from the Extractor in a browser.
5. When comparing the video and video-only stream urls I noticed some differences

Feature	Video-Stream	Video-Only-Stream
container format	MP4	WEBM/MP4
notable different parameters	ratebypass=yes and cnr=<number> present, also inside sparams	keepalive=yes present gir=yes and clen=<number> (seems to be content-length) present, also inside sparams

6. The N-Param decoder/YoutubeThrottlingDecrypter seems to work as expected

So my guess at this point of time is that the problem is primarily related to video-only streams.
I guess I notice the problem now more often in 0.23.0/dev because the video and audio streams are now separated (done in #7349).

I'm however still not sure why ExoPlayer goes into this weird 33KB/s phase.
I didn't notice that when playing the video-only-stream urls manually in the browser.
When the "Playback load interval size" is set to < 64KiB the problem is gone.
Maybe we should set the default value to <= 64KiB and not the ExoPlayer default (1MB)?

[AudricV]

(Notice) None of the streams I saw has an active cipher (was added in TeamNewPipe/NewPipeExtractor#327) we might be able to remove that code

No, it is present, especially on musical contents.

I'm however still not sure why ExoPlayer goes into this weird 33KB/s phase.

Probably because of the big initial chunk size + the HTTP version used (HTTP/1 is used to fetch contents, where HTTP/3 is used in your browser, take a look at what I said in this Reddit post for more details).

[Stypox]

Maybe we should set the default value to <= 64KiB and not the ExoPlayer default (1MB)?

@litetex thank you for your insights. Could you build an APK for that and ask users to test? I also think this problem is related to exoplayer, since usually switching the resolution to a different resolution than the initial one almost instantly loads the video, independently of the initial and the chosen resolution.

[litetex]

@Stypox
Done, see #8288

[SC1040-TS2]

(Notice) None of the streams I saw has an active cipher (was added in TeamNewPipe/NewPipeExtractor#327) we might be able to remove that code

No, it is present, especially on musical contents.

I'm however still not sure why ExoPlayer goes into this weird 33KB/s phase.

Probably because of the big initial chunk size + the HTTP version used (HTTP/1 is used to fetch contents, where HTTP/3 is used in your browser, take a look at what I said in this Reddit post for more details).

Having tested the MPEG-DASH NewPipe .APK, I can say with confidence that using this codec from now on is a must. Testing video playback revealed only milliseconds of difference between the 64KiB playback interval and the 1MB ExoPlayer Default interval settings. Negligible to the end user, and easily reconfigured as one desires without having to adhere to one preference over the other.

That all said, necessary code refactoring and cleanup as mentioned in #8238 (comment) is also a must. This will likely reduce further overhead by deprecating checks that no longer matter to the current state of NewPipe, and app, extractor, and player overhead are crucial as this thread and others related to it have shown.

This heavy cleanup will likely have to be done before pushing it as a major release candidate.
———
As an added bonus to the MPEG-DASH .APK release, phone temperatures when playing videos on a Samsung GS9 appear to be reduced considerably, with a slight chill noted. This may emphasize just how much of an improvement cutting down on playback and extractor overhead has.

Regards for the time being.
—
Small edit: Compression artifacts may be slightly worse in the MPEG-DASH codec .APK compared to the current version. This is consistent in both MP4 and WebM videos.

[lf-]

re: occurring on download, I can confirm observing periodic massive throttling on download (more details in my other issue comment here: #6949 (comment))

[litetex]

This was fixed with the introduction of DASH streams.