Minimum dataset size of Novel Objects

[retazo0018]

Hi @tgjantos,

I have few novel objects with limited training data (<30 images per object) whose 6D pose need to be estimated using PoET. As I understand, I have to train PoET on my custom dataset before Inference. So, I will resume training from a pre-trained PoET checkpoint on my custom dataset.

I'd like to know if there will be good accuracy during Inference after re-training PoET for novel objects with limited training data. Is there a minimum number of images per object you recommend for a good accuracy?

Thanks,

Best,

[tgjantos]

Hi @retazo0018 ,

please be aware that the pre-trained PoET models consist of the backbone and transformer part. In the current version of PoET the backbone is not trainable and hence it would not be fine-tuned for your objects. In general, the transformer depends on the feature maps provided by the object detector backbone.

Regarding the dataset size. Less than 30 images per object sound very small. I cannot make an educated guess whether this will work or not. Just try and let me know, whether it works!

I would recommend that you train an object detector for your objects and then train PoET from scratch for your specific objects.

Kind regards,
Thomas

[retazo0018]

Thanks for your reply @tgjantos ,

If I have a backbone that is able to detect my specific objects well, then few images per object would be enough to train PoET (transformer) alone from scratch. Is my understanding correct?

Best,

[tgjantos]

You can definitely try to train it from scratch. Would be interesting to see how well it performs with such minimal data. I cannot guarantee that you will achieve satisfying performance.

Best,
Thomas