You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The LMS state is stored within QOA files using 16-bit integers, however qoa_lms_t uses 32-bit integers on most platforms. Some third-party implementations use 16-but integers for the currently loaded LMS state, and it would be nice to do so in general for reducing memory usage and possible performance improvements, but there is a possibility of the weights overflowing the range of a 16-bit integer, and the documentation is somewhat unclear about whether this is required behaviour or not.
The text was updated successfully, but these errors were encountered:
lms->history[n] is guaranteed to stay within the 16bit range, as the samples are clamped before being handed over to qoa_lms_update(). Sadly, this is not the case for lms->weights[n] which in certain problem cases will exceed the 16 bit range with the current encoder.
For what it's worth, all of the 150 test cases on https://qoaformat.org/samples/ stay within the 16bit range for the weights.
You could modify the encoder so that it never exceeds 16bit for the weights – even for problem cases. This would still produce QOA files that adhere to the spec, but would likely involve some backtracking and have performance implications during encoding. It should not have any impact on decoding performance.
This repo also uses int instead of short, because intshould be faster on 32/64 bit systems. Whether that's actually the case is another question...
The LMS state is stored within QOA files using 16-bit integers, however qoa_lms_t uses 32-bit integers on most platforms. Some third-party implementations use 16-but integers for the currently loaded LMS state, and it would be nice to do so in general for reducing memory usage and possible performance improvements, but there is a possibility of the weights overflowing the range of a 16-bit integer, and the documentation is somewhat unclear about whether this is required behaviour or not.
The text was updated successfully, but these errors were encountered: