llama-run : fix context size (ggerganov#11094)

Set `n_ctx` equal to `n_batch` in `Opt` class. Now context size is a more reasonable 2048. Signed-off-by: Eric Curtin <[email protected]>
teleprint-me · Jan 6, 2025 · dc7cef9 · dc7cef9
1 parent ecebbd2
commit dc7cef9
Showing 1 changed file with 1 addition and 0 deletions.
diff --git a/examples/run/run.cpp b/examples/run/run.cpp
@@ -83,6 +83,7 @@ class Opt {
         }
 
         ctx_params.n_batch        = context_size >= 0 ? context_size : context_size_default;
+        ctx_params.n_ctx          = ctx_params.n_batch;
         model_params.n_gpu_layers = ngl >= 0 ? ngl : ngl_default;
         temperature               = temperature >= 0 ? temperature : temperature_default;