Get sparse feature sizes #8750
Labels
area:rasa-oss 🎡
Anything related to the open source Rasa framework
area:rasa-oss/ml 👁
All issues related to machine learning
research:incremental-training
type:enhancement ✨
Additions of new features or changes to existing ones, should be doable in a single PR
In order to dynamically change sparse layers, we should be able to split a combined feature input into features that come from separate featurizers. The reason behind this is that new features will be appended to the corresponding splits and not to the combined representation. That's why we need to know the size of each sparse feature.
feature_sizes
collection will have a following structure:Dict[Text, Dict[Text, List[int]]]
It will be a dictionary of attributes that have sparse features(e.g.
text
,response
… we are excludinglabel
at this point) to the dictionary of different kinds of features(e.g.sequence
/sentence
) to the list of feature sizes of different featurizers.The collection should be created in the
DIETClassifier
class during the_create_model_data
call and it should be assigned to theDIETClassifier
class as an attribute.feature_sizes
should be persisted after training and loaded during fine-tuning. We will need to compare persistedfeature_sizes
to the currentfeature_sizes
to find out if there are anyDenseForSparse
layers to be updated.Things to be done:
feature_sizes
of current datafeature_sizes
feature_sizes
The text was updated successfully, but these errors were encountered: