Benchmark_repos specified with a relative path #483
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The motivation for this PR is a fix for the benchmark github action. Now that the benchmarks are in a different package the aren't importing mentat code from where they did previously in the file system. This change should have no effect for people calling clone_repo from the mentat root dir.
I think in general it makes sense that the caller of setup_repo would expect the repo to be places in benchmark_repos of their cwd not relative to some script that they may not know about and may be in a strange unpredictable place depending on how mentat is installed.
Pull Request Checklist