-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: create an azure-ml pipeline for eval_prompts() #36
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #36 +/- ##
==========================================
- Coverage 97.85% 97.32% -0.54%
==========================================
Files 5 5
Lines 233 224 -9
==========================================
- Hits 228 218 -10
- Misses 5 6 +1 ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just one comment inline.
Also, can you add a link to a run of this workflow in the description of the PR?
azureml/eval_prompts.yml
Outdated
command: > | ||
python -m autora.doc.pipelines.main eval-prompts | ||
${{inputs.data_dir}}/data.jsonl | ||
${{inputs.data_dir}}/all_prompt.json |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's make the prompts file its own input parameter. This will make it easier to run experiments.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should I parameterize the data file as well? The default value can be 'data.jsonl' but a user can modify input if they want. WDYT?
Change Description
Created an azure-ml pipeline for eval_prompts()
closes #35
Solution Description
Code Quality
Project-Specific Pull Request Checklists
Bug Fix Checklist
New Feature Checklist
Documentation Change Checklist
Build/CI Change Checklist
Other Change Checklist