-
Notifications
You must be signed in to change notification settings - Fork 24
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use the full dataset for accuracy validation in the integration test #655
Comments
@Mostelk to check if we can get space from what Bruno is planning to buy. 20 GiB should be a safe bet for this purpose. |
@Mostelk to check with with Bruno for the progress. for public data, it's pretty easy.
The main issue: ImageNet What we need: 8.x GiB for all the full validation. |
Another solution would be using GitHub Release if each file is under 2 GB. |
@freedomtan check if we can put datasets other than ImageNet to GitHub Release. |
|
Let's use github release for datasets other than ImageNet. |
Waiting until #707 is merged so we save time and bandwidth downloading dataset. |
In our integration test we have a check to validate the accuracy results of the benchmarks. Currently, the app uses a tiny subset of the full dataset, so its result may not be reliable.
We should use the full dataset instead. Since this test is run exclusively on our CI, the full dataset can be stored privately on Google Cloud.
The text was updated successfully, but these errors were encountered: