-
Notifications
You must be signed in to change notification settings - Fork 4.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Response Selector in Cross Validation #4976
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Left a couple of comments, especially tests would be great.
Also, I'm getting this when running rasa test nlu --cross-validation
on the scaffold project:
Traceback (most recent call last):
File "/Users/tobias/.pyenv/versions/3.7/bin/rasa", line 11, in <module>
load_entry_point('rasa', 'console_scripts', 'rasa')()
File "/Users/tobias/Workspace/stack/rasa/__main__.py", line 76, in main
cmdline_arguments.func(cmdline_arguments)
File "/Users/tobias/Workspace/stack/rasa/cli/test.py", line 143, in test_nlu
perform_nlu_cross_validation(config, nlu_data, output, vars(args))
File "/Users/tobias/Workspace/stack/rasa/test.py", line 206, in perform_nlu_cross_validation
data, folds, nlu_config, output, **kwargs
File "/Users/tobias/Workspace/stack/rasa/nlu/test.py", line 1271, in cross_validate
train,
File "/Users/tobias/Workspace/stack/rasa/nlu/test.py", line 1178, in combine_result
) = compute_metrics(interpreter, data)
File "/Users/tobias/Workspace/stack/rasa/nlu/test.py", line 1359, in compute_metrics
intent_results, "intent_target", "intent_prediction"
File "/Users/tobias/Workspace/stack/rasa/nlu/test.py", line 1481, in _compute_metrics
results, target_key, target_prediction
ValueError: not enough values to unpack (expected 2, got 0)
Co-Authored-By: Tobias Wochinger <[email protected]>
Co-Authored-By: Tobias Wochinger <[email protected]>
@wochinge Added tests and types. I couldn't reproduce the error that you got. What dataset and config did you use? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good to go for me! Nice to see some love for the evaluation feature 🥇
Re the error with too few training data: create an issue and put in the maintenance board
) | ||
|
||
n_folds = 2 | ||
intent_results, entity_results, response_selection_results = cross_validate( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
optional: test if the report was written
Co-Authored-By: Tobias Wochinger <[email protected]>
Proposed changes:
Status (please check what you already did):
black
(please check Readme for instructions)