Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(backend,parser): remove stopwords from normalized tags on node update #369

Merged
merged 3 commits into from
Feb 1, 2024

Conversation

perierc
Copy link
Contributor

@perierc perierc commented Jan 31, 2024

What

This PR fixes the fact that the stopwords were not removed during normalization of the tags outside the parser (i.e during node update, or node creation).

  • Added an optional stopwords parameter (dict) to the normalize_text utils function, in order to remove stopwords from a given text.
  • Modified the update_node controller to remove the stopwords during normalization thanks to the new parameter
  • Refactored the parser to make it use the normalize_text utils function instead of its own method defined in its class

Part of

Fixes #367

@perierc perierc self-assigned this Jan 31, 2024
@perierc perierc requested a review from a team as a code owner January 31, 2024 22:57
Copy link
Contributor

@eric-nguyen-cs eric-nguyen-cs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
I approved but please ask for another look if you decide to add a Pydantic model

parser/openfoodfacts_taxonomy_parser/utils.py Outdated Show resolved Hide resolved
parser/openfoodfacts_taxonomy_parser/utils.py Show resolved Hide resolved
backend/editor/entries.py Outdated Show resolved Hide resolved
backend/editor/entries.py Outdated Show resolved Hide resolved
backend/editor/entries.py Outdated Show resolved Hide resolved
Copy link
Member

@alexgarel alexgarel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great :-)

parser/openfoodfacts_taxonomy_parser/utils.py Show resolved Hide resolved
@perierc perierc merged commit 24be684 into main Feb 1, 2024
7 checks passed
@perierc perierc deleted the perierc/remove-stop-words-from-normalised-tags branch February 1, 2024 22:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

Remove stop words from normalised tags when node updated
3 participants