librade V2 feature requests

[drowe67]

A place to jot down notes for features we are considering adding to V2 of the RADE API. We'll triage these before the coding starts - they may not all make it into V2.

1. text channel. Required for FreeDV reporter. There is currently 25 bit/s available for auxiliary data, this is being used for acquisition in V1. In particular it enables us to trap any false sync states (e.g. a bad freq offset). Allocating some of these bits for txt will affect the robustness of acquisition, which has been a problematic and time consuming area of development. It will need to be re-tested in simulation and perhaps OTA with a short test campaign to make sure robustness is not affected. We also need build API support, and add some sort of protocol e.g. a high rate (LDPC?) code to mop up errors, a CRC, and some framing (e.g. codec2 reliable txt system). However this is all DSP work, no ML re-training rqd. So if we want to add a text channel to RADE V1 waveform two tasks (a) work out a way to steal x/bits from the acquisition system (b) build a "reliable txt" library that is pretty similar to whats in libcodec2, and txs/rxs 1 bit a at time. This could be completely separate from librade, and joined at link time. Another approach might be to use the "End of Over" frame to send a bunch of text data using conventional OFDM.

2. SNR estimation. This is a tricky, and we haven't found a good solution to date. In legacy algorithms we measured the variance in the scatter diagram dots based on their expected positions. For RADAE the expected positions are time varying. We do have a "poor mans" DSP algorithm that works from the pilot symbols - it gives good results for AWGN but is inaccurate for multipath channels. So this is a R&D task with uncertain effort est. I suspect a ML approach is required, e.g. using symbol mag over time and freq as features. Training already runs at a variety of SNRs so could be used as output feature. Continuous output or probability from ML network?

3. API Doxygen support. Add comments to code, Doxygen post processing.

4. C port of core ML - will speed up RADAE enc/dec, greatly reducing CPU (FARGAN decoder will then dominate), and getting us closer to a pure C implementation.

5. Cython for dsp.py - will significantly reduce CPU load, and (I think) gives us C code to move us closer to a pure C implementation.

6. Further ML development - we may be able to get several dB lower, reduce latency, and improve acquisition. R&D required.

7. Limit 99% power bandwidth to approximately 1500 Hz. Needs to be done carefully to avoid PAPR reduction and carefully tested to ensure no performance degradation. All ctests need to be run with BPF signal, and effort to tweak tests for reliable passes. Initial investigation in TX Band Pass Filter #30. Having a waveform that is "tightly defined in frequency" will mean it passes thru SSB radio Tx filters while maintaining low PAPR.

8. doc/radae_intro_waveform suggestions: table comparing other waveforms, e.g. legacy FreeDV, break down enc/dec to include ML and DSP components, figure showing time versus freq, pilots, symbol breakdown to CP for OFDM waveform. References.

9. Clean up of radae repo, or creation on new "release" repo specifically for production quality code, support of target operating systems. The radae repo is Davids experimental playground (and we probably need such a repo moving fwd).

10. Improve error handling in the API. Errors should never result in the application exiting, for example (unless the application itself chooses to in response to an error).

11. Allow users of the API to provide a callback that the API can use to perform its own logging (i.e. for debugging).

12. Tuning PTT operation (aka reconciling real time PTT operation with stored file results). It's unclear if we have any quality drop with RADE in real time PTT operation (e.g. with freedv-gui. There are a great many new variables with it in use by the general Ham population, e.g. (a) Tx drive levels (b) radios (c) Tx and Rx filtering (d) audio levels, microphones. It is also very hard to evaluate PTT mode quality. This item would be a systematic study of real world PTT operation and how to optimise it (or could provide feedback on RADE V2 design). Measure PAPR after PA, to see if it is maintained. A key would be a tool for objective measurement Dump feature vectors freedv-gui#770

13. Reverb/digital sound on some samples/e.g. from Librispeech. Quite clear distortion. Is this present without aux data?

14. Way of including opus source (nnet.h, FARGAN code) without patching libopus or including opus code (or library) twice. Avoid us copying (forking) opus code to take advantage of maintenance by opus team. Idea from JMV: opus source could be included as a subproject and required source files included in librade build (including all FARGAN stuff).

15. Expanding reliable text to send larger callsigns and version number of protocol.