Revisit AZFP unprocessed variables currently in Vendor group

[emiliom]

AZFP has several variables that are unprocessed (eg, sensor signals) or have no analog in SONAR-netCDF4 v1. In echopype v0.6.0 we're moving many of these variables from the Beam_group1 group (formerly Beam) into the Vendor group. See #642 and https://github.com/OSOceanAcoustics/echopype/issues/520#issuecomment-1104626403520#issuecomment-1104626403.

This issue is a reminder to revisit these variables eventually, especially to explore the possibility of processing currently unprocessed raw variables such as temperature_counts.

[leewujung]

@emiliom : Now that #1075 is merged. Could you do this along with OSOceanAcoustics/echopype-checker#5 (comment) ? I'll get the AZFP vendor group nc files to you shortly. Thanks!

[emiliom]

especially to explore the possibility of processing currently unprocessed raw variables such as temperature_counts.

@leewujung I don't remember what this was about, specifically. Do you? We're already computing temperature per the AZFP equations, which start with counts.

I'm going over other variables in the Vendor_specific group.

[leewujung]

We're already computing temperature per the AZFP equations, which start with counts.

Interesting... I do not remember what this context was, but I think we have always had temperature computing since the beginning.

[emiliom]

temperature_counts

So, neither of us remembers the context for the temperature_counts comment. I don't remember if it came from me alone or a conversation between us! But I think we can ignore it. temperature is already calculated in _compute_temperature based on what looks like the index-4 "ancillary" parameter, which is a count:

echopype/echopype/convert/parse_azfp.py

Line 164 in 6fc5a1d

counts = self.unpacked_data["ancillary"][ping_num][4]

And index-4 is also the ancillary parameter written out to Vendor_specific.temperature_counts:

echopype/echopype/convert/set_groups_azfp.py

Line 478 in 6fc5a1d

"temperature_counts": (["ping_time"], anc[:, 4]),

So, case closed, I think.

Other variables

As for other variables in Vendor_specific that we could promote to another group, I looked at the list of variables and nothing strikes me as matching Beam_group1 or Platform M or MA variables we're not currently populating. But I'm not in a good position to make that determination, especially when none of the Vendor_specific variables have attributes, and checking against the manual would be pretty tedious.

One thing I'll note is that there is only one AZFP Beam_group1 variable with an MA (not M) obligation that is currently not being created: transmit_power. Can that be calculated from other, existing data??

Here are the mandatory (M) Beam_group1 variables not currently created:

beam_direction_x
beam_direction_y
beam_direction_z
beam_stabilisation
beam_type
non_quantitative_processing
sample_time_offset
transmit_type

I'll create a separate issue to track the inclusion of these variables into set_groups_azfp

[leewujung]

One thing I'll note is that there is only one AZFP Beam_group1 variable with an MA (not M) obligation that is currently not being created: transmit_power. Can that be calculated from other, existing data??

Yes it is possible to calculate that from existing data, but I think it would be better if we keep the same quantities and variable names as in the AZFP manual, so that it would be more straightforward for users to trace the code or compute things from scratch if they need to.

I'll create a separate issue to track the inclusion of these variables into set_groups_azfp

Sounds good! Please also include that in this release, so that we can be "done" with this type of changes/additions...

But I'm not in a good position to make that determination, especially when none of the Vendor_specific variables have attributes, and checking against the manual would be pretty tedious.

I think would be valuable to cross check with the manual and comments in parse_azfp.py (which are comments from the Matlab code) just so that we can get this over with. I don't think there are really things that would make sense to be in Beam_group1, but it is worth to check. I think it is fine to not have attributes for these variables, but if it is possible to put in simple descriptions like those in the code comment into the attributes, that would be great.

Thanks for tracking down everything!

[emiliom]

I think we can close this issue now. @leewujung can you confirm?

[leewujung]

Yes, thank you!