Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adjusting DenseForSparse layer size #8751

Closed
6 tasks
jupyterjazz opened this issue May 26, 2021 · 1 comment
Closed
6 tasks

Adjusting DenseForSparse layer size #8751

jupyterjazz opened this issue May 26, 2021 · 1 comment
Assignees
Labels
area:rasa-oss 🎡 Anything related to the open source Rasa framework area:rasa-oss/ml 👁 All issues related to machine learning research:incremental-training type:enhancement ✨ Additions of new features or changes to existing ones, should be doable in a single PR

Comments

@jupyterjazz
Copy link
Contributor

After completing #8749 and #8750, in order to dynamically change sparse layers during fine-tuning, we will need to adjust the DenseForSparse layer size in case there are any new sparse features.

This functionality will be implemented in the DIETclass. We will compare persisted feature_sizes to the current feature_sizes to find out if there are any DenseForSparse layers to be updated. After updating the layers, we will compile the whole model, update data signatures, create a RasaModelData object from a data example and activate new layers by fitting the model on the RasaModelData object.

Things to be done:

  • call adjust_layers from DIETClassifier during fine-tuning
  • find out which DenseForSparse layers should be updated
  • adjust each of them
  • compile the model
  • update data signatures
  • activate new layers
@jupyterjazz jupyterjazz added type:enhancement ✨ Additions of new features or changes to existing ones, should be doable in a single PR area:rasa-oss 🎡 Anything related to the open source Rasa framework area:rasa-oss/ml 👁 All issues related to machine learning research:incremental-training labels May 26, 2021
@jupyterjazz jupyterjazz self-assigned this May 26, 2021
@tttthomasssss
Copy link
Contributor

@ka-bu assigned as reviewer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:rasa-oss 🎡 Anything related to the open source Rasa framework area:rasa-oss/ml 👁 All issues related to machine learning research:incremental-training type:enhancement ✨ Additions of new features or changes to existing ones, should be doable in a single PR
Projects
None yet
Development

No branches or pull requests

3 participants