Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FEATURE: Allow the user to reset their benchmark. #65

Open
KatieProchilo opened this issue Jun 11, 2020 · 0 comments
Open

FEATURE: Allow the user to reset their benchmark. #65

KatieProchilo opened this issue Jun 11, 2020 · 0 comments
Labels
enhancement New feature or request P2 Nice to have

Comments

@KatieProchilo
Copy link
Contributor

Let's say that first a user creates a baseline model using the sample test data we provide. Since it is the first run of the workflow, this will become the baseline that future models must improve upon.

Now let's say the user finishes the walk through and wants to test the baseline model using real data from their project. This would not become the baseline that future models must improve upon, which doesn't make sense and presents more work for users. They'd have to go into Azure and properly delete the benchmark file.

Instead we can have the user push a tag beginning with RESET. This tag would test the latest baseline model against the test data, and it would become the new benchmark WER to beat in the future.

Now that I've done the baseline changes, this is a super easy add. Initializing an env var and then adding a conditional.

@KatieProchilo KatieProchilo added enhancement New feature or request P2 Nice to have labels Jun 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request P2 Nice to have
Projects
None yet
Development

No branches or pull requests

1 participant