-
Notifications
You must be signed in to change notification settings - Fork 10.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cuda: fix compile error in jetson platform #4975
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It sounds like ggml.h should #include <cuda_fp16.h>
, no? #includes shouldn't be sensitive to the order they are in.
I prefer u can check ggml-cuda.cu changes in #4766 |
Unrelated to what we prefer, if the header were to |
that's ok |
@slaren Hello, could you please check if these changes are okay, or if anything else is needed? |
I want to narrow down the GPU-releated stuff in the core Does Jetson platform not have diff --git a/ggml.h b/ggml.h
index 4c2ff6c6..9d165a86 100644
--- a/ggml.h
+++ b/ggml.h
@@ -305,9 +305,7 @@
extern "C" {
#endif
-#if defined(__ARM_NEON) && defined(__CUDACC__)
- typedef half ggml_fp16_t;
-#elif defined(__ARM_NEON) && !defined(_MSC_VER)
+#if defined(__ARM_NEON) && !defined(_MSC_VER)
typedef __fp16 ggml_fp16_t;
#else
typedef uint16_t ggml_fp16_t; I'm not sure when the |
this patch not work, nvcc dose not recognise
and if just fallback to |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am ok with this fix, but it would be good to add a comment explaining why these headers must be included last to prevent this from happening again.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since there are two approvals I'll avoid blocking this, but my two cents: If we're going to have a __CUDACC__
-specific typedef, why not have a __CUDACC__
-specific #include in the same file? It just seems wrong to me that we would intentionally reference a symbol without including the header that declares it - ggml.h or otherwise.
It would be one thing if this was a #define, but the compiler error is on the typedef itself in this case.
@KyL0N Please add a comment about why the headers are there and we can merge |
In my view, if |
Thanks for the discussion - IMO the fundamental issue is that |
* cuda: fix compile error in jetson platform * cuda: update comment in ggml-cuda.cu * cuda: update ggml-cuda.cu comment
* cuda: fix compile error in jetson platform * cuda: update comment in ggml-cuda.cu * cuda: update ggml-cuda.cu comment
this pr fix compile error in jetson platform
related issue: #4922
the error was introduced by #4766
#4766 add
#include "ggml-cuda.h"
before include
#include <cuda_fp16.h>
so the solution is move #include "ggml-cuda.h" after #include <cuda_fp16.h>