Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix for missing intent warnings when running rasa test #8388

Merged
merged 14 commits into from
May 18, 2021

Conversation

b-quachtran
Copy link
Contributor

@b-quachtran b-quachtran commented Apr 7, 2021

Proposed changes:

Status (please check what you already did):

  • added some tests for the functionality
  • updated the documentation
  • updated the changelog (please check changelog for instructions)
  • reformat files using black (please check Readme for instructions)

@m-vdb
Copy link
Collaborator

m-vdb commented Apr 15, 2021

@b-quachtran thanks for pushing fix -- should we ask someone from Enable to review this PR?

@m-vdb m-vdb linked an issue Apr 15, 2021 that may be closed by this pull request
@b-quachtran
Copy link
Contributor Author

Hey @m-vdb, yes definitely. I still need to wrap up a few things in this PR related to test cases, but will tag Enable as soon as it's ready.

@b-quachtran b-quachtran requested a review from joejuzl April 23, 2021 00:17
Copy link
Contributor

@joejuzl joejuzl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice one 👍
Is there an easy to test this is fixed with a unit test?

rasa/core/test.py Show resolved Hide resolved
@b-quachtran
Copy link
Contributor Author

@joejuzl Yep, I'll add a unit test as well 👍

@b-quachtran b-quachtran requested a review from joejuzl May 11, 2021 22:23
tests/core/test_test.py Outdated Show resolved Hide resolved
rasa/core/test.py Show resolved Hide resolved
@b-quachtran b-quachtran enabled auto-merge May 17, 2021 22:10
@b-quachtran b-quachtran merged commit f1d119d into main May 18, 2021
@b-quachtran b-quachtran deleted the rasa_test_missing_intent_warning branch May 18, 2021 09:07
@nearlyhuman-jwheat
Copy link

I've just updated to the recent release of 2.7.0 and the first thing I did was check to see if this issue was fixed.

I ran

rasa test --stories tests/test_stories.yml --fail-on-prediction-errors

And it can't find my domain.yml files.

/home/jwheat/anaconda2/envs/rasa263/lib/python3.7/site-packages/rasa/shared/utils/io.py:97: UserWarning: Loading domain from 'domain.yml' failed. Using empty domain. Error: 'Failed to load domain specification from '/mnt/d/Code/NearlyHuman/RASA2/rasa-demo-server/domain.yml'. File not found!'

I admit I have a more robust directory structure, but when I train I add --domain data and it finds them all, however rasa test doesn't accept that flag, so I get screens of

/home/jwheat/anaconda2/envs/rasa263/lib/python3.7/site-packages/rasa/shared/utils/io.py:97: UserWarning: Issue found in 'data/general-support/stories.yml':    
Found intent 'affirm' in stories which is not part of the domain.

More importantly, all my test stories fail, I'm guessing because it can't find my domain.yml files.

FWIW my structure looks like this where I have individual "skills" broken out into their own directory structure.

data/
    /skill1/
           domain.yml
           rules.yml
           nlu.yml
           stories.yml
    /skill2/
           domain.yml
           rules.yml
           nlu.yml
           stories.yml
etc.

I have no files in the root of my data directory and it looks like that's where this version is looking based on the first UserWarning.

Is there any way to get rasa test to see my structure and act like rasa train does with a --domain data flag?

Also curious what it does with the training data itself. Shouldn't rasa test only use the pre-trained model? Why does it need to look at all the domain files?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Rasa test/rasa test core can't find any intent in domain
4 participants