Installation on windows

[achibb]

Hi all!
I was trying to evaluate something / install mteb with pip install mteb and id id not seem to work on windows (colab however works).
Do you have any idea on how I could still evaluate my model "aari1995/German_Semantic_STS_V2" for German, e.g. on my GPU? :-)

Thank you!

` ValueError: path '/Users/cvangysel/Projects/pytrec_eval/trec_eval/convert_zscores.c' cannot be absolute
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> pytrec-eval

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.
WARNING: Ignoring invalid distribution -rotobuf (c:\users\aaron\anaconda3\envs\nlu\lib\site-packages)
WARNING: Ignoring invalid distribution -rotobuf (c:\users\aaron\anaconda3\envs\nlu\lib\site-packages)
WARNING: Ignoring invalid distribution -rotobuf (c:\users\aaron\anaconda3\envs\nlu\lib\site-packages)
`

[KennethEnevoldsen]

Thanks for reporting this @achibb, I sadly do not have access to a windows atm., but have added it to the test suite to make sure that it is validated not only now but going forward as well (#292).

Let us see how it goes.

One point that might cause an issue is the transition from a setup.py to a pyproject.toml for installation, which requires you to update your pip installation:

pip install pip --upgrade

[KennethEnevoldsen]

Hmm seems like the issues stem from the pytrec_eval-package (as you also see). Seems like others have the same issue as well cvangysel/pytrec_eval#32 (comment)

[KennethEnevoldsen]

@Muennighoff seems like pytrec_eval is not really developed anymore, it might be worth switching it out with something else?

[Muennighoff]

@Muennighoff seems like pytrec_eval is not really developed anymore, it might be worth switching it out with something else?

Great point, fine with me to switch it out if scores match! Maybe we don't even need a package and can just add the metrics we use from it to https://github.com/embeddings-benchmark/mteb/blob/main/mteb/evaluation/evaluators/utils.py

[KennethEnevoldsen]

A solution to this issue was added in #292 so will close this but feel free to open if there is still an issue.