Fix to avoid overfloat and get rid of model_max_length #319
Conversation
There was a problem hiding this comment.
This does not fix the issue with the mentioned model
The correct value is actually under another field:
"max_length": 512
Also setting a parameter this high might be dangerous (e.g. if we ever set padding to max_length)
I would rather check both fields, take the minimum one, and if they are absent, set some default value.
But tbh, a default value also seems not to be the greatest idea, I think we should just be more careful with the models we are adding
(traceback regarding "does not fix the issue")
2024-08-06 19:45:13.789327 [E:onnxruntime:, sequential_executor.cc:516 ExecuteKernel] Non-zero status code returned while running Add node. Name:'/embeddings/Add_1' Status Message: /Users/runner/work/1/s/onnxruntime/core/providers/cpu/math/element_wise_ops.h:560 void onnxruntime::BroadcastIterator::Append(ptrdiff_t, ptrdiff_t) axis == 1 || axis == largest was false. Attempting to broadcast an axis by a dimension other than 1. 512 by 10002
Update:
Some models tokenizer configs have different values for max_length and model_max_length.
In this particular example it is possible to use the model with RPE as written in the card.
We do not support such modifications, but we should investigate the existing models to choose the right way to fix the issue.
Jupter warning disabled
fastembed/common/model_management.py
Outdated
| @@ -148,7 +148,7 @@ def decompress_to_cache(cls, targz_path: str, cache_dir: str): | |||
| # Open the tar.gz file | |||
| with tarfile.open(targz_path, "r:gz") as tar: | |||
| # Extract all files into the cache directory | |||
| tar.extractall(path=cache_dir) | |||
| tar.extractall(path=cache_dir, filter="fully_trusted") | |||
There was a problem hiding this comment.
| tar.extractall(path=cache_dir, filter="fully_trusted") | |
| tar.extractall(path=cache_dir) |
Got rid of max_length=512 and parameter
Replaced it with maxsize to work with such situations. Checkout
model_max_length