Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/add chinese mapper #51

Merged
merged 8 commits into from
Nov 1, 2023
Merged

Feat/add chinese mapper #51

merged 8 commits into from
Nov 1, 2023

Conversation

chenhesen
Copy link
Collaborator

add chinese_convert_mapper: Mapper to convert Chinese between Traditional Chinese, Simplified Chinese and Japanese Kanji
add remove_non_chinese_character_mapper: Mapper to remove non chinese Character in text samples

configs/config_all.yaml Outdated Show resolved Hide resolved
Copy link
Collaborator

@HYLcool HYLcool left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New operators need to be added to the operator lists in docs/Operators.md and docs/Operators_ZH.md. Besides, the numbers of each type of operator need to be updated in these two docs as well.

@HYLcool HYLcool added the enhancement New feature or request label Oct 30, 2023
@chenhesen chenhesen merged commit 2768719 into main Nov 1, 2023
3 checks passed
@zhijianma zhijianma deleted the feat/add_Chinese_mapper branch November 15, 2023 05:57
@HYLcool HYLcool added the dj:op issues/PRs about some specific OPs label Jan 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dj:op issues/PRs about some specific OPs enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[feature] remove_non_chinese_character_mapper [feature] chinese_simplified_mapper
2 participants