FFmpeg has native aptX codec support

[joerg-krause]

FFMpeg devel added native aptX support https://patchwork.ffmpeg.org/patch/5879/. Do you think it is possible to add support for it in bluez-alsa?

[arkq]

Wow, indeed it looks like aptX. It's a pity, that they've added it to the ffmpeg core... Anyway, yes, I think it is possible to integrate ffmpeg with bluez-alsa.

EDIT:

The encoder was reverse engineered from binary library and from EP0398973B1 patent.

I was wondering why the code looks like an exact rip off of the proprietary Qualcomm BT aptX encoder :) So now, the question is: will it be merged? Because it smells like a copyright violation. :D

[joerg-krause]

It's already in master: https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/aptx.c.

[Samt43]

Hi guys ! I worked on this and could have it to work !
This permits to enable Aptx support using FFmpeg
https://github.com/Samt43/BluetoothAPTXForLinux

If you that made your day, don't hesitate to pm me or pay me a beer ;)
Samt43

[diizzyy]

@arkq @Samt43
ffmpeg also includes an sbc codec so it would be nice if we could use ffmpeg instead of separate libraries in the end.

[arkq]

@diizzyy as an optional configuration why not, but as a default I'm not sure.

$ qsize ffmpeg 
media-video/ffmpeg-3.3.6: 235 files, 24 non-files, 21612.193 KiB
$ qsize sbc 
media-libs/sbc-1.3: 12 files, 9 non-files, 221.750 KiB
$ qsize fdk-aac
media-libs/fdk-aac-0.1.5: 10 files, 8 non-files, 906.194 KiB

Bear in mind, that my ffmpeg configuration is pretty "light" :) Furthermore, (not sure if still valid, though) the ABI of ffmpeg libraries is a nightmare, so practically ffmpeg upgrade forces depending applications to be rebuilt.

[graham8]

Hmm but aren't systems going to have ffmpeg anyway (if you are worried about size)?

[dagbdagb]

Allow me to pose a naive question: how much work would it be to rip out the aptx en/de-coder from ffmpeg and create a standalone aptX encoder/decoder library useful for bluez-alsa?

Also, where exactly does the SBC codec live? In code form? Is it bundled with bluez?

Does the code in this repository only provide the glue between bluez and ALSA? Sorry, but I have never been able to overcome the confusion w.r.t. bluetooth audio and dependencies. Being able to eliminate as many dependencies as possible (including PA....) may prove helpful.

[diizzyy]

I think you can safely assume that ffmpeg is going to be installed on most systems however sbc requires ffmpeg 4.0 which may not be available on all systems. To be fair, without any major size optimization ffmpeg is about 4Mbyte (tar.gz) in compiled form if you handle "common" formats and include a few encoders. You can slim it down further by quite a bit if you want to. Not saying that it should replace the current setup but it would sure be a nice feature.

As a side note, I'm playing around with bluez-alsa in OpenWrt which is an embedded "router" linux distribution and being able to reduce dependencies would be really nice if possible.

@dagbdagb
I'm pretty sure that no one wants to extract aptX rather being able to link bluez-alsa with ffmpeg.

[dagbdagb]

Ok. I was just curious. From this, I understand that even SBC will be provided through ffmpeg? How does the code in this repo provide support for SBC today?

[diizzyy]

@dagbdagb
It links with the sbc library from bluez (standalone lib).

@arkq
Seems pretty stable? https://abi-laboratory.pro/tracker/timeline/ffmpeg/

[arkq]

OK, I admit, my previous comment might have a little bit of naive thinking :/

So, to be fair with you guys. I've got mostly one concern (and in general two concerns) when creating software for fun (and bluez-alsa is my spare time project, which to my surprise gained same attention). Firstly, it has to run fast (PA seems to fail in this area), and I'm not saying about compiler optimizations, but design which will allow fast computing (e.g. in bluez-alsa I'm not using mutexes for threads synchronization). Secondly, it should have small memory footprint. These points determine what I'm willing to do, and what I'm not :)

So, back to this issue. As far as I know, there is (was) a quarrel in the ffmpeg community about compatibility and new functions (I might be wrong, though). So the libav emerged which supposed to be a remedy for API breaking. This case alone is a big "warning" sign for me. But OK, let say, that right now the project is properly managed - there is no more "happy" development in the ffmpeg. ffmpeg library contains a lot of unnecessary codecs (from the Bluetooth point of view), which I've been trying to express in my previous comment (maybe not in the healthiest way) :D HOWEVER, if codecs (SBC, APTX, AAC, etc.) from ffmpeg produce better overall performance I will switch to ffmpeg - performance is more important for me.

Right now, I've got no time for tinkering with codecs, because I'm trying to make the msbc branch working. However, with my limited testing resources, it seems to be more complicated than I thougt.... :D

[diizzyy]

So I had a brief look at this and ffmpeg's sbc codec is based on the one found at https://www.kernel.org/pub/linux/bluetooth/ which is what bluez-alsa uses so performance should be equal or better. However I don't understand how it actually handles different settings such as subbands, bitpool, joint stereo(?) and blocks. They're not exposed as far as I can tell and there's no documentation available. This might be tied to the sbc format itself but I'm not sure. The author is on GitHub so let's give it a try @aurelj .

This is what ffmpeg 4.0 exposes:
ffmpeg -h encoder=sbc

Encoder sbc [SBC (low-complexity subband codec)]:
    General capabilities: small
    Threading capabilities: none
    Supported sample rates: 16000 32000 44100 48000
    Supported sample formats: s16
    Supported channel layouts: mono stereo

Edit: Make this post readable... ;-)

[aurelj]

@diizzyy
In my initial SBC patch submit to ffmpeg I did include options such as subbands, bitpool... But it was suggested to me that this was not a nice interface and that it didn't match how other codecs are used. You can follow this thread for the details: http://ffmpeg.org/pipermail/ffmpeg-devel/2018-February/thread.html#225644

So now, all those SBC settings are calculated automatically for best quality. The main option you can use to constrain those settings is the bitrate (eg. -b:a 320k). If you want to reduce encoding delay you can set a delay constraint (eg. -max-delay 5m for 5 milliseconds), which will also affect the number of subbands and blocks (possibly reducing quality).

Of course you can also choose the number of channel (1 or 2, eg. -ac 1) and the samplerate (eg. -ar 16000), which will also affect the settings computed to match the target bitrate.

If you want to dive into the details, have a look at the sbc_encode_init() function: https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/sbcenc.c#L195

Also if it reveals to be absolutely necessary to be able to set subbands, bitpools and other settings manually to fit in the A2DP protocol, let me know. In this case, it would certainly be possible to add direct control of those settings in ffmpeg.

[diizzyy]

@aurelj
Ahh, I see. Many thanks for replying.

[mirh]

This case alone is a big "warning" sign for me. But OK, let say, that right now the project is properly managed

It should be indeed. Libav is pretty much on life support nowadays AFAIK. See also HandBrake/HandBrake#974

[pali]

Anyway, if you are interested in a small standalone aptX library or small command line encoding/decoding utilities, I created one: https://github.com/pali/libopenaptx. Codec itself is based on ffmpeg sources.

[pali]

And nice description of aptX codec is in following IEEE Xplore SMPTE Journal: https://ieeexplore.ieee.org/document/7247105/

[pasikarkkainen]

@pali : nice! Is aptX low-latency (LL) supported aswell, or is that just a matter of configuring the right parameters?

[pali]

No, aptX Low Latency is not there. (update: it was supported since beginning)

[c2h5oh]

No aptX Low Latency, but it does support aptx-hd "codec" (it's more of a aptx profile) which is really nice.

[pali]

Correction about aptX low-latency: Stream produced by aptX and aptX-low-latency is same, so libopenaptx already supports aptX-low-latency. The difference between aptX and aptX-low-latency is on lower layer part bluetooth part (size of bluetooth packets, codec selection) which is irrelevant for libopenaptx.

[joerg-krause]

@pali Thanks for your sharing your insight. Can you explain this in more detail? Especially how LL can be actually used.

[pali]

@joerg-krause I have aptX-LL patches for pulseaudio and I will publish them after all my other pulseaudio a2dp patches are merged as aptX-LL is based on them.

[joerg-krause]

That's good news! What does it mean for bluez-alsa? Are there any patches required to enable aptX-LL, too?

[pali]

Of course, it is needed to implement A2DP handshake part.

[arkq]

@joerg-krause in bluez-alsa you can decrease aptX latency by feeding encoded data into the bluez socket more frequently. This loop should iterate over smaller number of samples: https://github.com/Arkq/bluez-alsa/blob/a30207dd173111a9b3618adbebe7469a60501576/src/a2dp.c#L1557-L1562 Beyond that, you will might use bluealsa D-Bus API directly (omit ALSA layer which introduces extra latency).

EDIT:
And of course as @pali has written, you need to register aptX-LL codec: https://github.com/Arkq/bluez-alsa/blob/a30207dd173111a9b3618adbebe7469a60501576/src/bluez-a2dp.c#L455

[pali]

aptX-LL supports also optional voice backchannel, so you can use it for parallel music streaming and microphone. And when aptX-LL is used on headset (instead of aptX) then it it more cases provides lower latency at headset side.

[arkq]

I'm closing this issue, because I will not switch to FFmpeg for apt-X encoder (and/or decoder). In the openaptx repository, I've posted benchmark results of various publicly available encoders, and the performance of the FFmpeg based encoder is almost two times worse than that of the original Qualcomm library (in some part due to FFmpeg generic API).