Use HF Auto classes in LanguageModelFeaturizer#10624

Closed

mleimeister wants to merge 43 commits intomainfrom lmfeaturizer-automodel

+475-527

Commits on Jan 4, 2022

Commits on Jan 7, 2022

Fix subtoken filtering bug. Add unit test for unknown default weights.
mleimeister
committed
Use distilbert model for test, since LaBSE weights are not cached on CI.
mleimeister
committed

Commits on Jan 10, 2022

Prevent error on non-existent cls token
mleimeister
committed

Commits on Jan 11, 2022

Commits on Jan 14, 2022

Commits on Jan 24, 2022

Commits on Feb 10, 2022

Commits on Feb 11, 2022

Update docs with models that are not supported and how to idenfify them. Unit tests for supported and failing models. Raise an exception if an unsupported model is loaded.
mleimeister
committed

Commits on Feb 14, 2022

Commits on Feb 15, 2022

Commits on Feb 17, 2022