FreeDATA Feature Request 001 - FreeDV API support for custom OFDM raw data modes

[drowe67]

Lets the advanced API programmer configure a OFDM raw data mode at init time, without having to modify libcodec2. It assumes the programmer understands how to configure the waveforms, use the waveform design spreadsheet, and has prototyped in Octave.

An example has been coded into freedv_data_raw_tx and freedv_data_raw_rx that takes the 4 carrier, 0.69 second duration datac14 packet waveform as a template, and converts it to a 3 carrier, 0.92 duration packet:

./src/freedv_data_raw_tx --bursts 3 --testframes 3 custom /dev/zero - | ./src/ch - - --No -14 -f 20 | ./src/freedv_data_raw_rx --testframes custom - /dev/null

You can hear the difference in duration using aplay:

./src/freedv_data_raw_tx --bursts 3 --testframes 3 datac14 /dev/zero - | aplay -f S16_LE
./src/freedv_data_raw_tx --bursts 3 --testframes 3 custom /dev/zero - | aplay -f S16_LE

TODO:

• tx/rx filters are not working yet
• adjustment of LDPC code rate via api
• adjust/tune/measure clipping

[drowe67]

@DJ2LS ☝️ This is branched off #44 so you can try both together.

[drowe67]

@DJ2LS - I've had a go at a non-table driven interleaver algorithm in C and Octave. I haven't tested it on the on (16200,9720) code yet as I don't have a waveform that uses that code, but the ctests pass so it works on a bunch of other codes.

[DJ2LS]

thanks, @drowe67 , as soon as I managed to access the API successfully via Python, I'll test a mode prototype which uses the 16200,9720 code, then we can test the interleaved algorithm, as well.

[DJ2LS]

@drowe67 , the gp interleaver might work, the modem didnt run into a crash with a 16200,9720 code.

but it seems we have to adjust allocate_tx_bpf and allocate_rx_bpf functions in https://github.com/drowe67/codec2/blob/dr-freedata-001/src/ofdm.c ,` the 'CUSTOM' mode seems to be missing, I guess.

[drowe67]

but it seems we have to adjust allocate_tx_bpf and allocate_rx_bpf functions in https://github.com/drowe67/codec2/blob/dr-freedata-001/src/ofdm.c ,` the 'CUSTOM' mode seems to be missing, I guess.

Yes we'll have to figure out a way to handle the filters. Suggest you disable them for now, tx_bpf_en=False & rx_bpf_en=False

[DJ2LS]

The set_data_bits_per_frame function for adjusting the used data bits for a ldpc code might be of interest 🤔 Is there a chance, we can implement it?

[drowe67]

The set_data_bits_per_frame function for adjusting the used data bits for a ldpc code might be of interest 🤔 Is there a chance, we can implement it?

So run run time design of the LDPC code? Tricky, however it might be possible for the WiMax or DVSB2 (CML built in codes), as there is already an Octave function ldpc_init_builtin for this. There are some restrictions to the code sizes, you can play around with this function to test.

This would be a significant chunk of work, probably best placed in another feature request or PR.

[DJ2LS]

At least the intention is, modifying the amount of data bits of a code, like done with datac14 for example. It seems, this is important for mode designing when having limitations like a specific bandwidth for example.But I probably there is another way for doing this?

[DJ2LS]

If the effort is too big, we could also keep this in mind, gathering experience with the existing way of adjusting parameters, also building the environment, like documentation, first. Then focusing on this somewhen later.

[drowe67]

At least the intention is, modifying the amount of data bits of a code, like done with datac14 for example. It seems, this is important for mode designing when having limitations like a specific bandwidth for example.But I probably there is another way for doing this?

Ok OK - so a fixed code, but we vary the # of payload data bits used. Yep, that can be done easily 👍

[drowe67]

@DJ2LS - I've worked out a way to set the number of used data bits per LDPC codeword auto-magically.

[drowe67]

@DJ2LS - I've had a first pass at making the Tx BPF config driven. You'll need to add:

  float *tx_bpf_proto; /* low pass prototype for complex BPF */
  int tx_bpf_proto_n;  /* number of taps in low pass prototype */

to your Python OFDM_CONFIG structure. Example usage in ofdm_mode.c. List of available filters in filter_coeff.h, also sample Octave command lines for creating new filters.

Setting up the clipper requires some experimentation, I do it at the Octave simulation level. Some notes I had left for myself in ofdm_demo.m, line 94.

[DJ2LS]

Thanks, @drowe67 . Problem is actually, that I can't directly access filter_coeff.h, I could copy and implement it in Python so we can still use it, but if we could access the codec2 related files directly, this would be more mature I think. Is there a way, just providing a name of the corresponding filter, like this:

  char tx_bpf_proto; /* low pass prototype for complex BPF */
  tx_bpf_proto = "filtP200S400"

[DJ2LS]

Another question is then the implementation of custom filters. How could this achieved? If the pointer to the filter on init would be used everywhere internally, this part could be outsourced, but with the problems mentioned before 🤔

[drowe67]

@DJ2LS - I've just pushed a feature that dumps the OFDM config using the new freedv_ofdm_print_info function:

./src/freedv_data_raw_tx --bursts 3 --testframes 3 custom /dev/zero - | sox -t .s16 -r 8000 -c 1 - tx_custom_n3.wav
ofdm->tx_centre = 1500
ofdm->rx_centre = 1500
ofdm->fs = 8000
ofdm->ts = 0.018
ofdm->rs = 55.5556
ofdm->tcp = 0.005
ofdm->inv_m = 0.00694444
ofdm->tx_nlower = 25
ofdm->rx_nlower = 25
ofdm->doc = 0.0436332
ofdm->timing_mx_thresh = 0.45
ofdm->nc = 3
ofdm->np = 6
ofdm->ns = 5
ofdm->bps = 2
ofdm->m = 144
ofdm->ncp = 40
ofdm->ftwindowwidth = 80
ofdm->bitsperframe = 24
ofdm->bitsperpacket = 144
ofdm->rowsperframe = 4
ofdm->samplespersymbol = 184
ofdm->samplesperframe = 920
ofdm->max_samplesperframe = 1840
ofdm->nrxbuf = 10672
ofdm->ntxtbits = 0
ofdm->nuwbits = 48
ofdm->foff_est_gain = 0.1
ofdm->foff_est_hz = 0
ofdm->timing_mx = 0
ofdm->coarse_foff_est_hz = 0
ofdm->timing_norm = 3.83333
ofdm->mean_amp = 0
ofdm->clock_offset_counter = 0
ofdm->verbose = 0
ofdm->sample_point = 0
ofdm->timing_est = 0
ofdm->timing_valid = 0
ofdm->nin = 920
ofdm->uw_errors = 0
ofdm->sync_counter = 0
ofdm->frame_count = 0
ofdm->sync_start = false
ofdm->sync_end = false
ofdm->sync_mode = autosync
ofdm->timing_en = true
ofdm->foff_est_en = true
ofdm->phase_est_en = true
ofdm->tx_bpf_en = true
ofdm->rx_bpf_en = true
ofdm->tx_bpf_proto_n = 3
ofdm->tx_bpf_proto:
1.000000	1.000000	1.000000	
ofdm->dpsk_en = false
ofdm->phase_est_bandwidth_mode = auto

[DJ2LS]

@DJ2LS - I've just pushed a feature that dumps the OFDM config using the new freedv_ofdm_print_info function:

./src/freedv_data_raw_tx --bursts 3 --testframes 3 custom /dev/zero - | sox -t .s16 -r 8000 -c 1 - tx_custom_n3.wav
ofdm->tx_centre = 1500
ofdm->rx_centre = 1500
ofdm->fs = 8000
<snip>

Thanks, @drowe67 thats definitely a usefull function! Problem with ctypes <--> C is, that it is hardly crashing when trying to open the codec2 instance, means I can't use the function as long as we don't have a running mode instance.

[DJ2LS]

@DJ2LS I got the tx bpf working.

[drowe67]

@DJ2LS has tested this by implementing a new, custom waveform. He worked through the Tx BPF and clipper adjustment steps. So ready for merge 🙂