Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CK2/CM] Development roadmap for 2022 #189

Closed
gfursin opened this issue Mar 24, 2022 · 3 comments
Closed

[CK2/CM] Development roadmap for 2022 #189

gfursin opened this issue Mar 24, 2022 · 3 comments
Assignees
@arjunsuresh @gfursin @hanwenzhu @petermattson
Labels
enhancement

Comments

@gfursin
Copy link
Contributor

gfursin commented Mar 24, 2022

No description provided.

@gfursin gfursin changed the title [CK2] Prepare development roadmap for Q2 and Q3 2022 [CK2/CM] Prepare development roadmap for Q2 and Q3 2022 Apr 13, 2022
@gfursin
Copy link
Contributor Author

gfursin commented Apr 19, 2022 •
edited
Loading

Main CM phases for 2022:

  • Develop and stabilize CM core to treat R&D projects as a database of components and automations (completed: 20220419 - CK2/CM v0.7.7)
  • Document CM core and provide a few tutorials
  • Develop CK2 scripts to detect and/or install all ML artifacts (platform description, OS scripts, ML frameworks, models, data sets, libraries, pre-/post-processing scripts, benchmarks, etc) (on-going): [CK2/CM scripts] support for ML frameworks and main ToDos #245
    • Support for TF
    • Support for ONNX
    • Support for TVM
    • Support for PyTorch
    • Support for TFLite
  • Convert our old CK-based MLPerf inference vision benchmark automations using CM
  • Implement CM-based "experiment" automation (record/reproduce/compare/visualize)
  • Implement CM-based MLPerf Design Space Explorer for open division submissions
  • Implement CM-based MLCubes for MLPerf DSE
  • Convert all MLPerf inference reference benchmarks to CM

@gfursin gfursin changed the title [CK2/CM] Prepare development roadmap for Q2 and Q3 2022 [CK2/CM] Prepare development roadmap for 2022 Apr 19, 2022
@gfursin gfursin changed the title [CK2/CM] Prepare development roadmap for 2022 [CK2/CM] Development roadmap for 2022 Apr 20, 2022
@gfursin gfursin mentioned this issue Aug 1, 2022
Closed
45 tasks
@gfursin
Copy link
Contributor Author

gfursin commented Aug 1, 2022

Since we have stabilized the CM (CK2) framework and added CM scripts to automate ML Systems benchmarking, we can now work on decomposing MLPerf inference benchmark into these CM scripts and automating the submission: #261 .

@gfursin
Copy link
Contributor Author

gfursin commented Nov 18, 2022

After validating the new CM automation workflow for MLPerf at the Student Cluster Competition, we close this ticket and move to the updated roadmap: #536 .

@gfursin gfursin closed this as completed Nov 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@arjunsuresh arjunsuresh

@gfursin gfursin

@hanwenzhu hanwenzhu

@petermattson petermattson

Labels
enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@arjunsuresh @gfursin @hanwenzhu @petermattson