Update docs for max-autotune usage (#1405)

Co-authored-by: Jack-Khuu <[email protected]>
pytorch · Dec 9, 2024 · aba0679 · aba0679
1 parent 326c1fe
commit aba0679
Show file tree

Hide file tree

Showing 2 changed files with 5 additions and 0 deletions.
diff --git a/docs/ADVANCED-USERS.md b/docs/ADVANCED-USERS.md
@@ -251,6 +251,8 @@ To improve performance, you can compile the model with `--compile`
 trading off the time to first token processed with time per token.  To
 improve performance further, you may also compile the prefill with
 `--compile-prefill`. This will increase further compilation times though. 
+For CPU, you can use `--max-autotune` to further improve the performance
+with `--compile` and `compile-prefill`. See [`max-autotune on CPU tutorial`](https://pytorch.org/tutorials/prototype/max_autotune_on_CPU_tutorial.html).
 
 Parallel prefill is not yet supported by exported models, and may be
 supported in a future release.

diff --git a/docs/model_customization.md b/docs/model_customization.md
@@ -34,6 +34,9 @@ prefill with `--compile_prefill`.
 
 To learn more about compilation, check out: https://pytorch.org/get-started/pytorch-2.0/
 
+For CPU, you can use `--max-autotune` to further improve the performance with `--compile` and `compile-prefill`.
+
+See [`max-autotune on CPU tutorial`](https://pytorch.org/tutorials/prototype/max_autotune_on_CPU_tutorial.html).
 
 ## Model Precision