Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add String Mapping Op #42

Merged
merged 12 commits into from
Mar 1, 2024
Merged

Add String Mapping Op #42

merged 12 commits into from
Mar 1, 2024

Conversation

apaniukov
Copy link
Collaborator

@apaniukov apaniukov commented Feb 27, 2024 •
edited by rkazants
Loading

Ticket: 132440

@apaniukov apaniukov requested a review from rkazants February 27, 2024 14:20
@apaniukov apaniukov marked this pull request as ready for review February 27, 2024 14:20
rkazants
rkazants reviewed Mar 1, 2024
View reviewed changes
src/vocab_encoder.cpp Outdated Show resolved Hide resolved
rkazants
rkazants reviewed Mar 1, 2024
View reviewed changes
src/vocab_encoder.cpp Outdated Show resolved Hide resolved
rkazants
rkazants requested changes Mar 1, 2024
View reviewed changes
Copy link
Collaborator

@rkazants rkazants left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now it is not suitable for TensorFlow needs:

  1. format of vocal keys needs to be changed
  2. need support of i64 values

@apaniukov apaniukov enabled auto-merge (squash) March 1, 2024 11:19
@apaniukov apaniukov merged commit c89b9fa into openvinotoolkit:master Mar 1, 2024
9 checks passed
@apaniukov apaniukov deleted the add-str-mapping-op branch April 4, 2024 15:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rkazants rkazants rkazants approved these changes

Assignees
No one assigned
Labels
tokenizers_cpp tokenizers_py tokenizers_test
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@apaniukov @rkazants