Improve IMA-ADPCM compression

[DeeJayLSP]

Describe the project you are working on

A game with lots of sound effects, with varied pitches and duration.

Describe the problem or limitation you are having in your project

In cases where lots of sound effects have to be played, it's best to use an uncompressed format (like WAV) to save some processing power. However, when compared to the other two formats (MP3 and Vorbis), the space used is considerably high (after all, it's uncompressed).

One solution would be to use FLAC, a lossless format which doesn't save much space as the lossy ones, but has a much faster decoding speed. However, that would mean adding a decoder for another format, and an attempt to implement it showed that the binary size increase is significant. And if used for extremely short effects (usually less than 0.05s) there is actually an increase in size. Busted. FLAC does have a slow decoding speed.

Another solution is to use WAV and import with the already built-in IMA-ADPCM compression, which drastically decreases the size of a WAV PCM (more than FLAC but less than MP3) and has a cheap decoding, but the resulting quality is somewhat low and on some pitches it introduces audible artifacts.

There is also QOA, which is made as a greatly improved alternative to IMA-ADPCM (the quality is better than Vorbis at a reasonable bitrate and decodes much faster than FLAC) and might have a minimal impact on size like the the implementation of its image counterpart QOI, but the format is quite new, and it claims the encoder might produce a dubious result.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

An attempt to improve ADPCM was made with ADPCM-XQ, made by the creator of WavPack.

By default it uses dynamic noise shaping and lookahead, techniques that greatly improve the coding quality. The procedures of those are better explained on its README.

I did a test by myself and noticed the sound artifacts were somewhat reduced. Sometimes the techniques make the coding take longer, but the result is worth the wait.

Here's a preview, carefully encoded to have maximum quality, of how it improves (sorry for the rickroll, I wanted to show the result on a instrument that gets heavily affected by ADPCM but ended up arranging that):

adpcm_preview.mp4

Of course, IMA-ADPCM shouldn't be used for music, but implementing it improves what we currently have.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

Implement ADPCM-XQ (it's C file and a header) and use it with the default parameters when IMA-ADPCM compression is requested instead of the current implementation.

ADPCM-XQ is simply an enhanced IMA-ADPCM encoder, therefore it shouldn't have compatibility issues.

Unlike implementing another format, this would just modify the code for the existing WAV resource importer, therefore not impacting the binary size for export builds.

This proposal somewhat counters #4264, which proposes the deprecation of the IMA-ADPCM encoder (or maybe complements it, since it's about removing "low-quality/limited" and this one proposes changing it).

If this enhancement will not be used often, can it be worked around with a few lines of script?

Not with a few.

Is there a reason why this should be core and not an add-on in the asset library?

I don't think you can change a built-in coding procedure with an addon.