Bug: Generated processed NWB files do not contain any NWB, just the output

[rachelstephlee]

I was analyzing some of the processed data, noticed some of the processed NWBs only have an output file, but no NWBs.

The processed NWBs that had this are:

behavior_700708_2024-06-05_08-25-30_processed_2024-06-28_23-38-01

behavior_700708_2024-06-27_08-43-37_processed_2024-06-28_23-16-02

behavior_705599_2024-06-21_13-37-22_processed_2024-06-28_23-45-35

[alexpiet]

Lots of older assets need to be reprocessed. Anything processed earlier than 09-12 has faulty FIP preprocessing.

[rachelstephlee]

oops thanks. Yea Ahad let me know. We found the error, and I'll attach it here. Ahad can take a look next week when he's free-er.

[Ahad-Allen]

@rachelstephlee to add some details but it seems that several assets might be missing their reward label info or have it stored using different keys. Will investigate when I get a chance to see if this affects a specific range or only certain assets

[Ahad-Allen]

I think the issue is related to keys, but it may be better for someone with more understanding of what the keys and experimental inputs should look like to take a look. I created a debug version of the capsule that may be helpful to work in:
https://codeocean.allenneuraldynamics.org/capsule/9612098/tree
It looks like earned = getattr(Obj, "B_Earned{}WaterStartTime".format(side), []) manual = getattr(Obj, "B_Manual{}WaterStartTime".format(side), []) auto = getattr(Obj, "B_Auto{}WaterStartTime".format(side), []) rewards = getattr(Obj, "B_{}RewardDeliveryTimeHarp".format(side), [])
will all fail due to this switch up in keys.

[alexpiet]

What I think is happening:
EarnedWater, ManualWater, and AutoWater are labels that were added at different times. RewardDeliveryTime will also contain the actual time of reward delivery. Most of the time, most rewards will be earned water

What I think we should:
If AutoWater labels do not exist, assume there were no AutoWater rewards
If ManualWater labels do not exist, assume there were not ManualWater rewards
If EarnedWater labels do not exist, assume all rewards not AutoWater or ManualWater were EarnedWater

Add a note to the processing.json for the packaging capsule that mentions that manual and auto water labels were missing and we assumed every is earned