cuda : refactor to remove global resources

[slaren]

Pools and other resources are tied to the ggml_backend instance and are freed along with it.

It should also be thread-safe.

[slaren]

There are now 220 221 GGML_UNUSED in ggml-cuda.cu.

[slaren]

Is there a way to run the server tests with -ngl 99?

[slaren]

The CUDA backend no longer uses ggml_tensor::backend. Other backends should also remove this and instead rely on the buffer type to identify the type of tensor, so that it can be removed from ggml_tensor.

[ggerganov]

Is there a way to run the server tests with -ngl 99?

If you rebase on master, should be able to run with:

cd examples/server/tests
N_GPU_LAYERS=99 LLAMA_SERVER_BIN_PATH=../../../build/bin/server ./tests.sh

[slaren]

Thanks. Unfortunately I am not able to complete all the server tests on my system because intermittent connection failures to huggingface cause the tests to abort. However, the tests that ran passed.

 llama_load_model_from_url: previous model file found ggml-model-f16.gguf.etag: "ee02c4b071444fa3c751868d68fb2676"
 llama_load_model_from_url: previous model file found ggml-model-f16.gguf.lastModified: Wed, 06 Mar 2024 16:54:46 GMT
 llama_load_model_from_url: curl_easy_perform() failed: SSL connect error
 llama_init_from_gpt_params: error: failed to load model 'ggml-model-f16.gguf'

[ggerganov]

The server tests pass on V100