Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Meta entities #3889

Closed
wants to merge 3 commits into from
Closed

Meta entities #3889

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5ef0b05
wip - add methods to entityextractor base class for adding and removi…
amn41 Jun 27, 2019
d6d8d62
wip add component for role labeling
amn41 Jun 27, 2019
88c3f2f
wip markdown regex which supports extracting roles
amn41 Jun 27, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions rasa/nlu/extractors/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,3 +87,66 @@ def filter_trainable_entities(
)

return filtered


@staticmethod
def add_roles_to_entities(role_message: Message, message: Message) -> Message:
"""mark all predicted roles as roles not entities"""
starts = {ent["start"]: ent for ent in role_message.get("entities", [])}

entities = []
for ent in message.get("entities", []):
start_idx = ent["start"]
if start_idx in starts:
ent_with_role = ent.copy()
ent_with_role["role"] = starts[start_idx]["entity"]
entities.append(ent_with_role)
else:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we don't want the else statement. Currently, we get in case no role was detected:

  "entities": [
    {
      "start": x,
      "end": y,
      "value": "Buenos Aires",
      "entity": "location",
      "confidence": 0.9792075801341894,
      "extractor": "CRFEntityExtractor"
    }
  ],
...
  "roles": [
    {
      "start": x,
      "end": y,
      "value": "Buenos Aires",
      "entity": "location",
      "confidence": 0.9792075801341894,
      "extractor": "CRFEntityExtractor"
    }
  ],

The roles is just a duplicate of entities. Shouldn't roles only contain "entites" that are actual roles?

entities.append(ent)

data = message.data.copy()
data["entities"] = entities
return Message(
text=message.text,
data=data,
output_properties=message.output_properties,
time=message.time,
)



@staticmethod
def replace_entities_with_roles(message: Message) -> Message:
"""replace all entities which have a role with a role"""

entities = []
text = message.text
for ent in message.get("entities", []):
if ent.get("role"):
role_ent = ent.copy()
role_ent["entity"] = ent["role"]
role_ent["value"] = ent["entity"]
# TODO update start and end values
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So would you simply replace Buenos Aires by location and then just obtain the start and end value for location?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes!

text = message.text[:ent["start"]] + \
ent["entity"] + \
message.text[ent["end"]:]
entities.append(role_ent)

data = message.data.copy()
data["entities"] = entities
return Message(
text=text,
data=data,
output_properties=message.output_properties,
time=message.time,
)

@staticmethod
def create_role_examples(
entity_examples: List[Message]
) -> List[Message]:
"""Creates role examples.
"""

return [ EntityExtractor.replace_entities_with_roles(message)
for message in entity_examples ]
Loading