We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
I have considered if we might want the interface to be:
import mteb bench = mteb.Benchmark(tasks=[...]) bench.evaluate(model) bench = mteb.get_benchmark("MTEB(eng)") bench.evaluate(model)
Since the mteb.MTEB object is kinda of a benchmark object. (not something we should do now)
mteb.MTEB
This would deprecate the mteb.MTEB object.
The text was updated successfully, but these errors were encountered:
I suppose the current mteb.MTEB object is a more flexibly Benchmark. I am in favour of this and v2.0 would be fitting.
Benchmark
Sorry, something went wrong.
No branches or pull requests
I have considered if we might want the interface to be:
Since the
mteb.MTEB
object is kinda of a benchmark object. (not something we should do now)This would deprecate the
mteb.MTEB
object.The text was updated successfully, but these errors were encountered: