Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add script to truncate log files to the rules #49

Merged
merged 3 commits into from
Sep 10, 2020
Merged

add script to truncate log files to the rules #49

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
98dd308
add truncation of logs
guschmue Aug 26, 2020
e88d064
fix formating
guschmue Aug 26, 2020
95c9af4
merge with master
guschmue Sep 10, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 12 additions & 3 deletions submission_rules.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -327,10 +327,9 @@ System names and implementation names may be arbitrary.

Here is the list of mandatory files for all submissions in any division/category. However, your submission should still include all software information and related information for results replication.


* mlperf_log_summary.txt
* mlperf_log_detail.txt
* Mlperf_log_accuracy.json [ TODO: handle differently in v0.6?]
* mlperf_log_accuracy.json
* user.conf
* calibration or weight transformation related code if the original MLPerf models are not used
* actual models if the models are not deterministically generated
Expand All @@ -340,6 +339,16 @@ Here is the list of mandatory files for all submissions in any division/category
* <system_desc_id>.json
* compliance_checker_log.txt

For some models mlperf_log_accuracy.json can get very large. Because of this we truncate mlperf_log_accuracy.log in submissions
using a tool.
A submiter will run the tool before submitting to mlperf and ***keep*** the original mlperf_log_accuracy.log files inside their organization.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

submitter

The original files might be requested by mlperf during submission review so you need to store them.
Run the tool as follows, assuming <SOURCE> is your local subumission tree and <DEST> the location of the github submission repo:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

subumission -> submission


```
# from top of the inference source tree
python3 tools/submission/truncate_accuracy_log.py --input <SOURCE> --output <DEST>
```

### <system_desc_id>.json metadata

Expand Down Expand Up @@ -471,7 +480,7 @@ This section in progress [TODO].

#### Inference

Refer to the documentation found under https://github.com/mlperf/inference/tree/master/v0.7/compliance/nvidia
Refer to the documentation found under https://github.com/mlperf/inference/tree/master/compliance/nvidia


## Review
Expand Down