Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get sparse feature sizes #8750

Closed
3 tasks
jupyterjazz opened this issue May 26, 2021 · 1 comment
Closed
3 tasks

Get sparse feature sizes #8750

jupyterjazz opened this issue May 26, 2021 · 1 comment
Assignees
Labels
area:rasa-oss 🎡 Anything related to the open source Rasa framework area:rasa-oss/ml 👁 All issues related to machine learning research:incremental-training type:enhancement ✨ Additions of new features or changes to existing ones, should be doable in a single PR

Comments

@jupyterjazz
Copy link
Contributor

jupyterjazz commented May 26, 2021

In order to dynamically change sparse layers, we should be able to split a combined feature input into features that come from separate featurizers. The reason behind this is that new features will be appended to the corresponding splits and not to the combined representation. That's why we need to know the size of each sparse feature.

feature_sizes collection will have a following structure: Dict[Text, Dict[Text, List[int]]]
It will be a dictionary of attributes that have sparse features(e.g. text, response… we are excluding label at this point) to the dictionary of different kinds of features(e.g. sequence/sentence) to the list of feature sizes of different featurizers.

The collection should be created in the DIETClassifier class during the _create_model_data call and it should be assigned to the DIETClassifier class as an attribute.

feature_sizes should be persisted after training and loaded during fine-tuning. We will need to compare persisted feature_sizes to the current feature_sizes to find out if there are any DenseForSparse layers to be updated.

Things to be done:

  • get feature_sizes of current data
  • persist feature_sizes
  • load persisted feature_sizes
@jupyterjazz jupyterjazz added type:enhancement ✨ Additions of new features or changes to existing ones, should be doable in a single PR area:rasa-oss 🎡 Anything related to the open source Rasa framework area:rasa-oss/ml 👁 All issues related to machine learning research:incremental-training labels May 26, 2021
@jupyterjazz jupyterjazz self-assigned this May 26, 2021
@tttthomasssss
Copy link
Contributor

@samsucik assigned as reviewer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:rasa-oss 🎡 Anything related to the open source Rasa framework area:rasa-oss/ml 👁 All issues related to machine learning research:incremental-training type:enhancement ✨ Additions of new features or changes to existing ones, should be doable in a single PR
Projects
None yet
Development

No branches or pull requests

3 participants