Video track publishing issues with VP9

[mpnri]

Describe the bug

Issue Description

I was testing video codecs in livekit and I have encountered some problems with the video quality when using the VP9 codec. As you know, VP9 is an improved version of VP8, which offers better compression efficiency, quality, and support for high-resolution video. However in my tests livekit does not seem to work well with VP9, as I have experienced worse latency and FPS than with VP8.

In the first video below, you can see the result of one of the tests.
There are 4 clients in this test (3 web and 1 Android named "Android Me")
And the recorded video is from the web client called "Web Me".

In this test, two other web clients are in the same system and are sharing same camera device, but one with vp9 codec and the other with vp8 codec are publishing that video.
As you can see, the video received from the vp9 client has lower fps and more latency than the vp8 client.

web-publish-test-in-web.mp4

You can also see the result of this test in the "Android Me" client:

web-publish-test-in-android.mp4

• android sdk version: 2.0.0 (origin/main)

• server version(golang): 1.5.2

• components-js: commit hash: 5fa2d6b33c591a5cf511c015d066546c983bf2aa) Update to livekit-client v2 (#733)

• publish options for "vp9" client:

options: { publishDefaults: { videoCodec: 'vp9' } }

• publish options for "vp8" client:

options: { publishDefaults: { videoCodec: 'vp8' } }

Reproduction

To reproduce this problem and check, just connect up to 2 web clients to the same camera and publish one with vp9 and the other with vp8.

Logs

No response

System Info

Web clients: Linux

Severity

annoyance

Additional Information

We performed a similar test for two Android clients, one publishing vp9 and the other vp8. The result was that we did not observe a similar problem. Therefore, we can probably conclude that this problem is from the js client or from the server side.

[lukasIO]

vp9 generally still needs more system resources to do the encoding. This might be related to bad encoding performance on linux as well, but hard to tell. If you don't experience the same issue when publishing from an android device it sounds definitely like the issue is more likely to be found on the publishing client's system where it performs worse.

[lukasIO]

we did some optimizations in #1069 as part of v2.0.7.
Closing this issue, but feel free to re-open if you still run into issues even after applying that update