Skip to content

Commit

Permalink
llama : on Metal, by default offload the full model
Browse files Browse the repository at this point in the history
ggml-ci
  • Loading branch information
ggerganov committed Jan 10, 2024
1 parent 3cb1c1f commit 07a1b05
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion llama.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -9069,7 +9069,8 @@ struct llama_model_params llama_model_default_params() {
};

#ifdef GGML_USE_METAL
result.n_gpu_layers = 1;
// note: we usually have plenty of VRAM, so by default offload all layers to the GPU
result.n_gpu_layers = 999;
#endif

return result;
Expand Down

0 comments on commit 07a1b05

Please sign in to comment.