Unable to make LLAMA_CUBLAS=1 Unknown option forward-unknown-to-host-compiler

[schwab]

Trying to do a fresh cuda based install on a ubuntu 20.04 machine with an RTX 3090 I get a nvcc fatal error
Nvidia driver: 530.30.02 Cuda driver: 12.1

This is the output.

make LLAMA_CUBLAS=1
I llama.cpp build info:
I UNAME_S: Linux
I UNAME_P: x86_64
I UNAME_M: x86_64
I CFLAGS: -I. -O3 -std=c11 -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -pthread -march=native -mtune=native -DGGML_USE_CUBLAS -I/usr/local/cuda/include -I/opt/cuda/include -I/targets/x86_64-linux/include
I CXXFLAGS: -I. -I./examples -O3 -std=c++11 -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native -DGGML_USE_CUBLAS -I/usr/local/cuda/include -I/opt/cuda/include -I/targets/x86_64-linux/include
I LDFLAGS: -lcublas -lculibos -lcudart -lcublasLt -lpthread -ldl -lrt -L/usr/local/cuda/lib64 -L/opt/cuda/lib64 -L/targets/x86_64-linux/lib
I CC: cc (Ubuntu 9.4.0-1ubuntu120.04.1) 9.4.0
I CXX: g++ (Ubuntu 9.4.0-1ubuntu120.04.1) 9.4.0

nvcc --forward-unknown-to-host-compiler -arch=native -I. -I./examples -O3 -std=c++11 -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native -DGGML_USE_CUBLAS -I/usr/local/cuda/include -I/opt/cuda/include -I/targets/x86_64-linux/include -Wno-pedantic -c ggml-cuda.cu -o ggml-cuda.o
nvcc fatal : Unknown option '-forward-unknown-to-host-compiler'

According to bard.google.com....
This is a known issue with CUDA 12.1. The -forward-unknown-to-host-compiler option was removed in CUDA 12.1. To work around this issue, you can use the following command:

nvcc -c -std=c++11 -Xcompiler -Wall myfile.cpp

Use code with caution. Learn more
This will compile the file without generating a warning.

[slaren]

I don't think that --forward-unknown-to-host-compiler has been removed in CUDA 12.1, bard is most likely hallucinating there. Make sure that your CUDA toolkit is actually 12.1, not just the driver.

[RodolfoCastanheira]

Got the same error. Slaren is right, Ubuntu 20.04 uses CUDA 10.1 by default. Got 12.1 from nvidia and that fixed the error. After some fight with apt, had only to tell make where the nvidia nvcc was with: make LLAMA_CUBLAS=1 PATH="/usr/local/cuda/bin/:$PATH"

[NathanBick]

@RodolfoCastanheira This worked for me as well, thank you. I am using Ubuntu 20.04 on WSL 2, specifically

[fhirfly]

@RodolfoCastanheira's solution worked for me

[Lucidology]

I know this is an old thread but it's the #1 result in google for this problem. I am trying to compile it on Linux Mint Unbuntu with cuda 12.2 and a Tesla P40. I am not sure how to resolve "nvcc fatal : Unknown option '-forward-unknown-to-host-compiler'"

Here is what I'm doing:

(/home/m/text-generation-webui-main/installer_files/env) m@m-Precision-Tower-7910-DBE:~/llama.cpp$ CUDA_VISIBLE_DEVICES=0 CUDA_DOCKER_ARCH=compute_61 make LLAMA_CUDA=1
I ccache not found. Consider installing it for faster compilation.
expr: syntax error: unexpected argument ‘070100’
expr: syntax error: unexpected argument ‘080100’
I llama.cpp build info:
I UNAME_S:   Linux
I UNAME_P:   x86_64
I UNAME_M:   x86_64
I CFLAGS:    -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_CUDA -I/usr/local/cuda/include -I/usr/local/cuda/targets/x86_64-linux/include  -std=c11   -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -pthread -march=native -mtune=native -Wdouble-promotion
I CXXFLAGS:  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -march=native -mtune=native -Wno-array-bounds -Wno-format-truncation -Wextra-semi -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_CUDA -I/usr/local/cuda/include -I/usr/local/cuda/targets/x86_64-linux/include
I NVCCFLAGS: -std=c++11 -O3 -use_fast_math --forward-unknown-to-host-compiler -Wno-deprecated-gpu-targets -arch=compute_61 -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_MMV_Y=1 -DK_QUANTS_PER_ITERATION=2 -DGGML_CUDA_PEER_MAX_BATCH_SIZE=128
I LDFLAGS:   -lcuda -lcublas -lculibos -lcudart -lcublasLt -lpthread -ldl -lrt -L/usr/local/cuda/lib64 -L/usr/lib64 -L/usr/local/cuda/targets/x86_64-linux/lib -L/usr/lib/wsl/lib
I CC:        cc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
I CXX:       g++ (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
I NVCC:      Cuda compilation tools, release 10.1, V10.1.243

nvcc -std=c++11 -O3 -use_fast_math --forward-unknown-to-host-compiler -Wno-deprecated-gpu-targets -arch=compute_61 -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_MMV_Y=1 -DK_QUANTS_PER_ITERATION=2 -DGGML_CUDA_PEER_MAX_BATCH_SIZE=128  -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_CUDA -I/usr/local/cuda/include -I/usr/local/cuda/targets/x86_64-linux/include  -Xcompiler "-std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-pedantic" -c ggml-cuda.cu -o ggml-cuda.o
nvcc fatal   : Unknown option '-forward-unknown-to-host-compiler'
make: *** [Makefile:482: ggml-cuda.o] Error 1
(/home/m/text-generation-webui-main/installer_files/env) m@m-Precision-Tower-7910-DBE:~/llama.cpp$ nvidia-smi
Sat Apr 20 23:29:38 2024
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.154.05             Driver Version: 535.154.05   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  Tesla P40                      Off | 00000000:02:00.0 Off |                    0 |
| N/A   46C    P0              48W / 250W |    157MiB / 23040MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
|   1  NVIDIA GeForce GTX 970         Off | 00000000:03:00.0  On |                  N/A |
| 24%   30C    P8              19W / 151W |    218MiB /  4096MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|    0   N/A  N/A      1273      C   /usr/local/bin/ollama                       146MiB |
|    0   N/A  N/A      1523      G   /usr/lib/xorg/Xorg                            4MiB |
|    0   N/A  N/A      3225      G   /usr/lib/xorg/Xorg                            4MiB |
|    1   N/A  N/A      1273      C   /usr/local/bin/ollama                        58MiB |
|    1   N/A  N/A      1523      G   /usr/lib/xorg/Xorg                           55MiB |
|    1   N/A  N/A      1785      G   xfwm4                                        45MiB |
|    1   N/A  N/A      3225      G   /usr/lib/xorg/Xorg                           52MiB |
+---------------------------------------------------------------------------------------+


$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Sun_Jul_28_19:07:16_PDT_2019
Cuda compilation tools, release 10.1, V10.1.243

[alangshur]

@Lucidology the solution above that includes PATH still works with the new LLAMA_CUDA argument. This fixed the problem for me:

make LLAMA_CUDA=1 PATH="/usr/local/cuda/bin/:$PATH"

[kien-vu-uet]

@Lucidology the solution above that includes PATH still works with the new LLAMA_CUDA argument. This fixed the problem for me:

make LLAMA_CUDA=1 PATH="/usr/local/cuda/bin/:$PATH"

It works for me. Thank bro!!!