treat torch models as feature extractors when feature_extractor=True #3517
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes are proposed in this pull request?
New parameter
feature_extractor=False
added to torch models.If set to True, then the model output is treated as features without
forward hooks.
When using
dino
model with default forward hooks, its outputbecomes 3-demensional and can't be used as embeddings. With
this fix the output is expected 2-dimensional.
Also, if
embeddings_layer=xxxx
is provided, then the modelis replaced with a feature extractor for that layer using torchvision.
Finally, added a check on
embeddings_layer
making sure it'snot an
Identity()
layer is it does not return anything withforward hooks.
How is this patch tested? If it is not, please explain why.
Added a test with
dino
model from torch hub.Release Notes
Is this a user-facing change that should be mentioned in the release notes?
notes for FiftyOne users.
(Details in 1-2 sentences. You can just refer to another PR with a description
if this PR is part of a larger change.)
What areas of FiftyOne does this PR affect?
fiftyone
Python library changes