Add note to install cudnn, re #8

nshepperd · Aug 27, 2019 · 50fa3b6 · 50fa3b6
1 parent a070f38
commit 50fa3b6
Showing 1 changed file with 2 additions and 0 deletions.
diff --git a/README.md b/README.md
@@ -48,6 +48,8 @@ PYTHONPATH=src ./encode.py <file|directory|glob> /path/to/encoded.npz
 PYTHONPATH=src ./train.py --dataset /path/to/encoded.npz
 ```
 
+Make sure `cudnn` is installed. [Some have reported](https://github.com/nshepperd/gpt-2/issues/8) that `train.py` runs without it but has worse memory usage and might OOM.
+
 ### Gradient Checkpointing
 
 https://github.com/openai/gradient-checkpointing is included to reduce the memory requirements of the model, and can be enabled by `--memory_saving_gradients`. The checkpoints are currently chosen manually (poorly) by just adding layer 10 to the 'checkpoints' collection in model.py. `--memory_saving_gradients` is enabled by default for training the 345M model.