New plankton model available from Turing work

[metazool]

paper

repository

model weights

via @noushineftekhari

We can be looking to integrate and evaluate this work both as a source of embeddings and of labels.

There are weights for a ResNet-18 and for a Vision Transformer. I couldn't see the model loading code for the latter on a quick scan. We should be asking any questions about that on an issue in the repository

[noushineftekhari]

Sure! Thanks, Noushin

[metazool]

cheers @noushineftekhari - please see the small pull request and the issue linked here, i've only been able to get the 3 class Resnet18 going with our data so far, I've just found the full set of ViT / Beit checkpoints and am looking at those. Thank you again for making this work available

cc @jmarshrossney

[noushineftekhari]

Hi Jo, Please let me know if you need help and any things you think we can improve. I'm excited to see the results! We can improve or generalize them in case they are not working well. It is a good starting point for building a sensor-agnostic model. Best Noushin

[metazool]

I've tried this on a (small set) of unlabelled FlowCam data with results that were not promising - 90% of the data classified as detritus (not unreasonable) with enough plankton in that class, and detritus classified as plankton, as not to be useful directly for classification

https://github.com/ukceh-rse/ViT-LASNet/blob/main/Fig1-ECCV_lr.png