Generic reports #377
Generic reports #377
Conversation
|
MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅ |
src/modelbench/run.py
Outdated
| "--custom-branding", | ||
| type=click.Path(file_okay=False, dir_okay=True, path_type=pathlib.Path), | ||
| help="Path to custom branding. Implicitly sets --generic-report to remove MLCommons copy.", | ||
| ) | ||
| @click.option( |
There was a problem hiding this comment.
Would it make more sense here to just have one CLI option that is either 1) --custom-branding which includes MLC branded stuff or 2) no --custom-branding which is always generic?
There was a problem hiding this comment.
Wouldn't users need to know the path of MLC branding in that case?
There was a problem hiding this comment.
I think for now the only users running reports with MLC branding will be the three of us. And we'll eventually put that branding directory in modellab, where we'll be running the official reports from.
There was a problem hiding this comment.
Yes, I like where you're going with this, but agree with Daniel. The default should be a generic report. Then something like a --branding option lets us point to something like a folder that contains our flavor packet. In a future pull request, we'll move that folder-or-whatever to ModelLab, but this PR just needs to start extracting it.
|
Okay, updated to remove |
A possibly wild stab at generating generic reports. content/ now contains all of the generic/default .toml files, and values in the new content_mlc/ directory will override the defaults if
--generic-reportis not provided.Just want to confirm that this is generally how we want generic reports to work, and then I’ll add the CSS stuff.