Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some CLI commands are unexpectedly slow #8058

Closed
alwx opened this issue Feb 26, 2021 · 3 comments
Closed

Some CLI commands are unexpectedly slow #8058

alwx opened this issue Feb 26, 2021 · 3 comments
Labels
area:rasa-oss/cli Issues focused on the rasa command-line-interface area:rasa-oss 🎡 Anything related to the open source Rasa framework effort:atom-squad/4 Label which is used by the Rasa Atom squad to do internal estimation of task sizes. type:bug 🐛 Inconsistencies or issues which will cause an issue or problem for users or implementors. type:maintenance 🔧 Improvements to tooling, testing, deployments, infrastructure, code style.

Comments

@alwx
Copy link
Contributor

alwx commented Feb 26, 2021

Rasa version: 2.3.1 (used main branch, commit hash: 2574c46e9576607a7b8de39f823ff3e5c01a475c)

Rasa SDK version (if used & relevant): 2.3.1

Rasa X version (if used & relevant):

Python version: 3.7 / 3.8

Operating system (windows, osx, ...): MacOS / Windows

Issue:
We made an investigation of how long it takes to execute tests, and one of the conclusions that can certainly be made is that some CLI tests are unexpectedly slow.
Several examples:

  • tests/cli/test_rasa_train.py::test_train_help, tests/cli/test_rasa_train.py::test_train_nlu_help and other similar commands often take 5 seconds. But all of them are just some simple commands that only print some help information.
  • tests/cli/test_cli.py::test_cli_start might take up to 20 seconds on Windows (it just executes rasa --help and should be really fast)
  • it seems to affect other commands as well. However, we decided to handle other cases separately.

Here you can find more information about the time it takes to execute CLI tests: https://gist.github.com/alwx/fc3334473b046231f1ed35b0bd6e30e3

See more details about time measurements: https://docs.google.com/spreadsheets/d/1tDRD0vWaLW91W1wPj5h__dwXuXioRt9aBPYyF1n4MjI/edit#gid=57091041

Definition of done:

  • --help commands run in less than 1s

Command or request that led to error:

pytest tests/cli
@alwx alwx added type:bug 🐛 Inconsistencies or issues which will cause an issue or problem for users or implementors. area:rasa-oss 🎡 Anything related to the open source Rasa framework area:rasa-oss/cli Issues focused on the rasa command-line-interface labels Feb 26, 2021
@alwx alwx added the type:maintenance 🔧 Improvements to tooling, testing, deployments, infrastructure, code style. label Feb 26, 2021
@TyDunn TyDunn added the feature:ux-cli+training-data Feature: Improve user experience with Rasa CLI and training data for developers label Feb 26, 2021
@joejuzl joejuzl added the effort:atom-squad/4 Label which is used by the Rasa Atom squad to do internal estimation of task sizes. label Feb 26, 2021
@alwx alwx assigned alwx and unassigned alwx Mar 4, 2021
@joejuzl
Copy link
Contributor

joejuzl commented Apr 8, 2021

Is this still relevant after #8141 ?

@TyDunn
Copy link
Contributor

TyDunn commented May 5, 2021

@alwx can we close this?

@TyDunn TyDunn removed the feature:ux-cli+training-data Feature: Improve user experience with Rasa CLI and training data for developers label May 5, 2021
@alwx
Copy link
Contributor Author

alwx commented May 6, 2021

@TyDunn yes, seems like it can be closed.

@alwx alwx closed this as completed May 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:rasa-oss/cli Issues focused on the rasa command-line-interface area:rasa-oss 🎡 Anything related to the open source Rasa framework effort:atom-squad/4 Label which is used by the Rasa Atom squad to do internal estimation of task sizes. type:bug 🐛 Inconsistencies or issues which will cause an issue or problem for users or implementors. type:maintenance 🔧 Improvements to tooling, testing, deployments, infrastructure, code style.
Projects
None yet
Development

No branches or pull requests

3 participants