Make filter_graph_->nb_threads be 1, and make number of decoding threads customizable

[innerlee]

The makes the time spent on this function deterministic, and hence faster.
Testing on 32 videos shows 1.3x speedup:

Testing code:

[huww98]

We also work on Kinetics dataset and found an issue similar to #62. I suggest adding an option to totally disable FFmpeg multi-threading. My patch is similar, so I will not open a new PR. see huww98@76333af

In our use case, we see not only 1.24x speedup, but CPU usage also drops from 245% to 110%. That's a total of 2.7x more efficient.

Here is my test code. In short, I open each file, then only read first 16 frames.

from pathlib import Path
import time
from decord import VideoReader

BASE = Path('/tmp/answering_questions')

def main():
    files = list(BASE.glob('**/*.mp4'))
    frames = list(range(16))

    print(f"Reading {len(frames)} frames from {len(files)} files")
    for i in range(3):
        print(f'pass {i + 1}')
        t1 = time.perf_counter()
        for f in files:
            v = VideoReader(str(f))
            v.get_batch(frames)
        t2 = time.perf_counter()
        print(f'Time: {t2-t1}')

if __name__ == "__main__":
    main()

The CPU usage data is reported by /usr/bin/time -v python test.py

[zhreshold]

Good catch! However, I think a better option is to expose the multithreading config to python side with arguments to turn ON/OFF the multithreading.
@innerlee Do you want to add that option to VideoReader?

@yjxiong @bryanyzhu FYI, there's possibly a good margin for speed up in multi-worker loading if we turn of the internal mt.

[innerlee]

@huww98 I tested your changes. If we

• comment the decoding thread line as in huww98@76333af#diff-2528f2176a6afdf33b9891ecd924c3e5R110 , or set dec_ctx->thread_count = 1;, then the average decoding time is 29.4ms
• the original setting, the average decoding time is 11.6ms

Clearly in the single worker case, we should leave the decoding thread num to auto.
Note the above test is not in parallel. I'm not sure if disabling this would lead to performance gain in multi-worker loading. More tests should be done.

@zhreshold Yeah its nice to implement that option. I'm not familiar with c++, so if this change is trivial for you, could you help implementing it? I would like to continue tracing other performance issues.

[innerlee]

@zhreshold I implemented the customizable number of decoding thread. And leave the filter graph thread to the fixed number 1.

[huww98]

@innerlee Thanks. My test is performed in WSL2, maybe threading overhead is larger in WSL.

However, in our use case, we do not care about single video decode time. Since we always use multiple workers to increase throughput, I think multi-thread in FFmpeg can only add more overhead. Disabling FFmpeg threading can always benefit us.

[innerlee]

yes, with this pr, you can set num_threads=1 in the python constructor of VideoReader. Could you try it and check if the speed is similar to your changes?

[yjxiong]

Late to the party. Great to know this is being figured out.

The filter-graph threading seems not needed in the use case. The codec threading seems helpful when the use case just has one worker. For multiple workers, it is better to just use no ffmpeg-level threading.

Maybe we can add this guidance in the usage doc.

[zhreshold]

looks good to me

[zhreshold]

I will be merging this now, once we figure out the impact of multi_worker we can add guidance in the README.