PR: Add PyTorch FFmpeg to wheel and conda distributions

[andfoy]

This PR enables the inclusion of FFmpeg LGPL binaries on wheel distributions and also updates the conda recipe dependencies to include FFmpeg as well

Fixes #2365

[andfoy]

@fmassa, it seems that the decoder backend is broken in Windows, is this expected?

[fmassa]

@andfoy this is not expected, and I believe we at some point tested the Windows compilation of the decoder

[fmassa]

@andfoy what do you think about the following: you disable Windows for FFmpeg for now, revert the changes that are specific to trying to fix Windows, and we open a new issue to track fixing Windows for this and add the Windows label, so that @peterjc123 and colleagues can have a look at it?

[andfoy]

@fmassa, I managed to fix the Windows compilation issues, right now we're having test failures on both Linux and Windows

[fmassa]

Does it pass locally for you? Maybe the machine is running out of memory? Can you ssh to the machine to debug?

[andfoy]

Does it pass locally for you?

No, they fail and segfault as well, I don't know if the errors are related to the ffmpeg binaries that av uses vs ours, I'll try again with gdb and see where it fails

[andfoy]

It seems that --disable--mmx flag when compilling FFmpeg fixes the segfaults on the tests

[fmassa]

It seems that --disable--mmx flag when compilling FFmpeg fixes the segfaults on the tests

So this means that we would need to ship ffmpeg without MMX optimizations? Do you have an idea if it would make the binaries much slower than the ones we currently have for some machines?

[andfoy]

So this means that we would need to ship FFmpeg without MMX optimizations? Do you have an idea if it would make the binaries much slower than the ones we currently have for some machines?

It seems that on modern architectures MMX was superseded by SSE/2/3/4 and AVX, so in principle, disabling MMX should only affect very old processors, as FFmeg has individual flags to disable SSE and AVX: https://github.com/FFmpeg/FFmpeg/blob/05c8d0bce64888c5312822fbc9cdb63934b86519/configure#L430

[fmassa]

Interesting, test failures could seem related, although most of them are on test_io.py which uses pyav, so I'm not 100% sure what's happening here. Maybe pyav is picking a different ffmpeg version than the one it was compiled to use?

For the error in test_read_video_from_memory_scripted, there is an easy fix: replace the % for string formatting in

vision/torchvision/io/_video_opt.py

Line 91 in 8dfcff7

start pts: %d and end pts: %d""" % (

with the .format equivalent. There might have been a change in torchscript that made the % stop working.

[andfoy]

The error here is that some tests are trying to use x264 and x264rgb, which are not available on our LGPL FFmpeg distribution. I tried to switch some of them to h264, but it fails when x264rgb is required

[andfoy]

For some reason, tests pass when test_io.py is executed with pytest -v test/test_io.py

However, it fails when they are executed alongside all the testsuite (pytest -v test)

[fmassa]

@andfoy if tests pass when only running on test_io, but not the full test suite, this might indicate that another test (test_video_reader?) is also loading ffmpeg from our binaries and affects test_io. To make sure that this is the case, can you run test_io and test_video_reader in the same run to double-check (although the order on which the tests are run might be important)

[andfoy]

Found more information related to the test failure cases:

#2745 (comment)

[fmassa]

Great, this is awesome, we will soon be unblocking this and @bjuncek PR I believe

[fmassa]

Let's merge this PR and see how it goes.

We can clean a few things up (following @peterjc123 comment) on a follow-up PR.

Also, ccing @malfet and @seemethere for visibility -- we are adding ffmpeg as a dependency for torchvision.