[Question] Abort segment requests if client bandwidth changes drastically

[SemihGk]

I am working to decrease buffering rates on our applications. It looks like we may need to implement a more advanced AbrManager since shaka player has already configuration that allows to accept a custom abrManager. However, I am curious about a few points. Are you planning to improve abrManager's ability in the future instead of keeping a simple one?

If you have a plan, let me share a few ideas that I have:

1. Let's say, abrManager is downloading a segment at 8000kps, then suddenly client bandwidth drops to 1000 kbps. So, abrManager may abort the request and may request the lowest variant. Also, consider vice versa.
2. Unless I am wrong, shaka retries requests as long as we configure the retry times. Let's say, first few requests have failed and I assume shaka would not retry different variant which has a different bitrate. So, what if abrManager detects this failed requests and lower the bitrate. This would be also valuable if the request timeout exceeds.
3. Some devices might have performance issues. I specifically work on Samsung and LG TVs. Those platforms are definitely non-performant and the app can be stuck when the CPU reaches its capacity. So, what if we implement to find/estimate the CPU performance of the device and perform a better variant selection by abrManager.
4. AbrManager can send multiple segment requests by using web workers. Since other requests would be made on different thread, it would not make any performance issue on the main thread. So, abrManager can append available buffers by considering these different responses.

Please let me know what your thoughts are on these.

Thank you,
Semih.

[chrisfillmore]

I think this is similar to #1051

[SemihGk]

Thank Chris. How did I miss this ticket :) I will definitely track this ticket and this would be a quite useful improvement on buffering rates. Another note, #1051 is almost same to point 1, but I would still appreciate if I can get some suggestions for other ones as well.

[TheModMaker]

1. Let's say, abrManager is downloading a segment at 8000kps, then suddenly client bandwidth drops to 1000 kbps. So, abrManager may abort the request and may request the lowest variant. Also, consider vice versa.

Handled in #1051.

2. Unless I am wrong, shaka retries requests as long as we configure the retry times. Let's say, first few requests have failed and I assume shaka would not retry different variant which has a different bitrate. So, what if abrManager detects this failed requests and lower the bitrate. This would be also valuable if the request timeout exceeds.

If there is a network error, the Player will retry the exact same request up to the retry limit. If there are errors, AbrManager isn't involved and won't change the bitrate. Although #1542 has requested we change that.

3. Some devices might have performance issues. I specifically work on Samsung and LG TVs. Those platforms are definitely non-performant and the app can be stuck when the CPU reaches its capacity. So, what if we implement to find/estimate the CPU performance of the device and perform a better variant selection by abrManager.

We only use the network bandwidth to adjust the current stream. But you should look at #907 and #745.

4. AbrManager can send multiple segment requests by using web workers. Since other requests would be made on different thread, it would not make any performance issue on the main thread. So, abrManager can append available buffers by considering these different responses.

Network requests are handled by the browser in background threads, so using a web worker would probably not change anything involving concurrent requests. We do request different streams (e.g audio and video) concurrently. But we shouldn't request too many at once since that will take up more bandwidth and slow the request time.

[joeyparrish]

3. Some devices might have performance issues. I specifically work on Samsung and LG TVs. Those platforms are definitely non-performant and the app can be stuck when the CPU reaches its capacity. So, what if we implement to find/estimate the CPU performance of the device and perform a better variant selection by abrManager.

We only use the network bandwidth to adjust the current stream. But you should look at #907 and #745.

The MediaCapabilities API would also help with this (see #1391), but these existing TV platforms may not implement that API.

[SemihGk]

2. This is great if you are going to change this behaviour. I agree this must be fixed on back-end side, I have never seen things are perfect at BE.
3. Hmm, Although this PR is not exactly what I was considering, it is very helpful to see a different point of view. I will discuss this with my coworkers.
4. I thought network requests using web workers would be handled in different thread. I made a simple benchmark and literally there is no difference( https://codepen.io/semihgk/pen/jvrOdr?editors=0010 ). I was considering this approach as a backup segment, but it looks like it would not necessary after implementing 1 and 2.

Thank you very much for your reply, TheModMaker. Those are quite helpful.

Joey, this API would have been absolutely helpful. However, as you mentioned, those TV platforms would not implement it within a near future as they maintain very old version of chromium browsers.

Thanks again. I look forward to waiting those features. I really appreciate it.