You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
* [demo] Fix token-classification (Closeshuggingface#116)
* Fix Bert tokenizer regex for numbers
* Update Bert pretokenizer regex
Should match the rust implementation: https://github.com/huggingface/tokenizers/blob/b4fcc9ce6e4ad5806e82826f816acfdfdc4fcc67/tokenizers/src/pre_tokenizers/bert.rs#L11
- Removes whitespace
- Splits on unicode punctuation and certain ascii characters
* Add tokenizer test cases with numbers
* Build demo website after release
* [package.json] Update keywords
* [version] Update to 2.0.1
* Freeze onnxruntime dependencies (huggingface#124)
Their latest version has a few issues, particularly with webgpu, and also uses .wasm files which are incompatible with their previous versions.
So, while those issues are sorted out, it's best to freeze their packages to the latest stable version.
* Use versioned links (#Closeshuggingface#114)
Prevents issues where users copy-paste the import code, and then a future update breaks it.
Also ensures that the default wasm files match the target version
* [version] Update to 2.0.2
* Update package-lock.json
* Update README.md
* Replace `Math.max` with custom `max` function
* Add `sentence-transformers` models to supported models/tasks
* Correctly use default module if present
* Use error mapping instead of switch block
* [docs] Fix numbering
---------
Co-authored-by: Joshua Lochner <[email protected]>
Congrats to version 2! 🎉
I'd just like to throw in that a note somewhere would be useful that the new version breaks sites written for the "old" logic of v1.4.3.
This
<script src="https://cdn.jsdelivr.net/npm/@xenova/transformers/dist/transformers.min.js"></script>
doesn't work anymore but instead you need to specify the version with
<script src="https://cdn.jsdelivr.net/npm/@xenova/[email protected]/dist/transformers.min.js"></script>
.This fixed my broken sites ad-hoc. Else, people will probably be able to find this issue and fix it themselves :)
The text was updated successfully, but these errors were encountered: