From 2b6bd64e6f036a0337f1c689a1669d35680d3f15 Mon Sep 17 00:00:00 2001 From: tjablin Date: Tue, 27 Oct 2020 09:14:17 -0700 Subject: [PATCH 1/2] Time Preprocessing --- inference_rules.adoc | 20 ++------------------ 1 file changed, 2 insertions(+), 18 deletions(-) diff --git a/inference_rules.adoc b/inference_rules.adoc index f029e13..5cd8d7a 100644 --- a/inference_rules.adoc +++ b/inference_rules.adoc @@ -444,7 +444,7 @@ using a checksum. The dataset must be unchanged at the start of each run. As input, before preprocessing: -* all imaging benchmarks take uncropped uncompressed bitmap +* all imaging benchmarks take uncropped compressed images * BERT takes text @@ -452,23 +452,7 @@ As input, before preprocessing: * DLRM takes a variable sized set of items, each a sequence of embedding indices -Sample-independent pre-processing that matches the reference model is -untimed. However, it must be pre-approved and added to the following list: - -* May resize to processed size (e.g. SSD-large) - -* May reorder channels / do arbitrary transpositions - -* May pad to arbitrary size (don’t be creative) - -* May do a single, consistent crop - -* Mean subtraction and normalization provided reference model expect those to be - done - -* May convert data among numerical formats - -Any other pre- and post-processing time is included in the wall-clock time for a +Pre- and post-processing time is included in the wall-clock time for a run result. === Test Data Traversal Order From b85e77b47d16be1fafd676ebbec6a5687ce63e55 Mon Sep 17 00:00:00 2001 From: tjablin Date: Tue, 27 Oct 2020 10:06:52 -0700 Subject: [PATCH 2/2] Datacenter only --- inference_rules.adoc | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-) diff --git a/inference_rules.adoc b/inference_rules.adoc index 5cd8d7a..5c4fe5c 100644 --- a/inference_rules.adoc +++ b/inference_rules.adoc @@ -444,7 +444,7 @@ using a checksum. The dataset must be unchanged at the start of each run. As input, before preprocessing: -* all imaging benchmarks take uncropped compressed images +* all imaging benchmarks take uncropped compressed images in for Datacenter submission and uncompressed images otherwise * BERT takes text @@ -452,8 +452,25 @@ As input, before preprocessing: * DLRM takes a variable sized set of items, each a sequence of embedding indices -Pre- and post-processing time is included in the wall-clock time for a -run result. +Sample-independent pre-processing that matches the reference model is +timed Datacenter submissions and untimed otherwise. Untimed preprocessing +must be pre-approved and added to the following list: + +* May resize to processed size (e.g. SSD-large) + +* May reorder channels / do arbitrary transpositions + +* May pad to arbitrary size (don’t be creative) + +* May do a single, consistent crop + +* Mean subtraction and normalization provided reference model expect those to be + done + +* May convert data among numerical formats + +Any other pre- and post-processing time is included in the wall-clock time for a +run result. === Test Data Traversal Order