Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

upgrade to 1.5.0 #24

Closed
jwijffels opened this issue Nov 15, 2023 · 27 comments
Closed

upgrade to 1.5.0 #24

jwijffels opened this issue Nov 15, 2023 · 27 comments

Comments

@jwijffels
Copy link
Contributor

Stable release is now 1.5.0 and available at https://github.com/ggerganov/whisper.cpp/releases/tag/v1.5.0

@shikokuchuo
Copy link
Contributor

Would be great to run this with the new CUDA support!

@jwijffels
Copy link
Contributor Author

jwijffels commented Jan 25, 2024

Work on this is in branch upgrade-v1.5.4 at https://github.com/bnosac/audio.whisper/tree/upgrade-v1.5.4
Continuous integration tests #25 which run on CPU machines already seem to work.
Regarding CUDA. I don't have a CUDA machine currently but hopefully within some months.
Some work needs to be done on the Makevars and on making sure the downloading of the models point to the right url's.

@jwijffels
Copy link
Contributor Author

I've upgraded the R package to use whisper.cpp 1.5.4. A looot of backend changes.

@shikokuchuo
@rpodcast
@jmgirard

Feel free to test it out.
It should now be possible to user large-v1, large-v2, large-v3. I did some small checks that quantized models also worked, which it did.

For the CUDA support. I'll plan to look at it once I get my hands on a machine with CUDA on.
I only have access to Windows and Linux machines and the unit tests using github actions which run several platforms but it would be good if someone with a Mac could test this. Especially the compiler settings regarding the Accelerate framework.

@shikokuchuo
Copy link
Contributor

Yep tested briefly - the quantised models quite a bit faster. Do you know what it would take to enable CUDA? I can try testing it out if there are details somewhere. Or can the package link to system libs (that I can then build with CUDA support)?

@jmgirard
Copy link
Contributor

I can test on Mac and windows.

@jwijffels
Copy link
Contributor Author

Yep tested briefly - the quantised models quite a bit faster.

I also saw that.

Do you know what it would take to enable CUDA? I can try testing it out if there are details somewhere. Or can the package link to system libs (that I can then build with CUDA support)?

I'll move this question to #27

@jmgirard
Copy link
Contributor

Installation with default CFLAGS works on MacOS (M2 macbook).

Similarly with Sys.setenv(WHISPER_ACCELERATE = "1").

However, I get some warnings with the recommended CFLAGS:

Sys.setenv(WHISPER_CFLAGS = "-mavx -mavx2 -mfma -mf16c")
#> clang: warning: argument unused during compilation: '-mavx' [-Wunused-command-line-argument]
#> clang: warning: argument unused during compilation: '-mavx2' [-Wunused-command-line-argument]
#> clang: warning: argument unused during compilation: '-mfma' [-Wunused-command-line-argument]
#> clang: warning: argument unused during compilation: '-mf16c' [-Wunused-command-line-argument]

@jwijffels
Copy link
Contributor Author

I can test on Mac and windows.

Do you have the Mac Accelerate installed? That would help in testing.
And do you have Metal on Mac?
And CoreML on Mac?

@jmgirard
Copy link
Contributor

jmgirard commented Jan 27, 2024

I just installed accelerate via pip install accelerate and re-ran:

Sys.setenv(WHISPER_ACCELERATE = "1")
remotes::install_github("bnosac/audio.whisper", ref = "0.3", force = TRUE)

Seems to work but generates the following warnings:

whisper_cpp/ggml.c:9828:17: warning: 'cblas_sgemm' is deprecated: first deprecated in macOS 13.3 - An updated CBLAS interface supporting ILP64 is available.  Please compile with -DACCELERATE_NEW_LAPACK to access the new headers and -DACCELERATE_LAPACK_ILP64 for ILP64 support. [-Wdeprecated-declarations]
                cblas_sgemm(CblasRowMajor, CblasNoTrans, CblasTrans,
                ^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/vecLib.framework/Headers/cblas.h:610:6: note: 'cblas_sgemm' has been explicitly marked deprecated here
void cblas_sgemm(const enum CBLAS_ORDER __Order,
     ^
whisper_cpp/ggml.c:10238:9: warning: 'cblas_sgemm' is deprecated: first deprecated in macOS 13.3 - An updated CBLAS interface supporting ILP64 is available.  Please compile with -DACCELERATE_NEW_LAPACK to access the new headers and -DACCELERATE_LAPACK_ILP64 for ILP64 support. [-Wdeprecated-declarations]
        cblas_sgemm(CblasRowMajor, transposeA, CblasNoTrans, m, n, k, 1.0, a, lda, b, n, 0.0, c, n);
        ^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/vecLib.framework/Headers/cblas.h:610:6: note: 'cblas_sgemm' has been explicitly marked deprecated here
void cblas_sgemm(const enum CBLAS_ORDER __Order,
     ^
2 warnings generated.

Let me know if there are other things you want me to try on Mac.

@jwijffels
Copy link
Contributor Author

Seems to work but generates the following warnings:

Looks indeed ok. Does that speedup transcription?

@jmgirard
Copy link
Contributor

My macbook's system information says it supports Metal 3, but I'm not sure if I need to install anything to enable it.

I just installed CoreML via pip install coremltools. Do you need me to install any third-party packages?

@jmgirard
Copy link
Contributor

Looks indeed ok. Does that speedup transcription?

What do you recommend for systematically testing speed? Formal benchmarking or something like tictoc? Also, would the whisper model and/or duration of the audio file matter for getting a good estimate?

@jwijffels
Copy link
Contributor Author

jwijffels commented Jan 27, 2024

Just get a longer audio file en see how long it takes if you have the package installed with WHISPER_ACCELERATE and without - make sure you remove the environment variable Sys.unsetenv("WHISPER_ACCELERATE") or start a new session.

library(av)
download.file(url = "https://www.ubu.com/media/sound/dec_francis/Dec-Francis-E_rant1.mp3", 
              destfile = "rant1.mp3", mode = "wb")
av_audio_convert("rant1.mp3", output = "output.wav", format = "wav", sample_rate = 16000)
library(audio.whisper)
model <- whisper("medium")
trans <- predict(model, newdata = "output.wav", language = "en", n_threads = 1)
trans$timing

@jwijffels
Copy link
Contributor Author

jwijffels commented Jan 27, 2024

My macbook's system information says it supports Metal 3, but I'm not sure if I need to install anything to enable it.

I just installed CoreML via pip install coremltools. Do you need me to install any third-party packages?

I need to make changes in our Makevars file to enable Metal and CoreML. This will be for later.

@jmgirard
Copy link
Contributor

16.65 min with accelerate vs. 47.8 min without

@jwijffels
Copy link
Contributor Author

jwijffels commented Jan 28, 2024

Great so that accelerate option works on Mac. I will get back to this later for these coreml/metal, once I made the changes in the Makevars to enable these.

@jwijffels
Copy link
Contributor Author

Installation with default CFLAGS works on MacOS (M2 macbook).

Similarly with Sys.setenv(WHISPER_ACCELERATE = "1").

However, I get some warnings with the recommended CFLAGS:

Sys.setenv(WHISPER_CFLAGS = "-mavx -mavx2 -mfma -mf16c")
#> clang: warning: argument unused during compilation: '-mavx' [-Wunused-command-line-argument]
#> clang: warning: argument unused during compilation: '-mavx2' [-Wunused-command-line-argument]
#> clang: warning: argument unused during compilation: '-mfma' [-Wunused-command-line-argument]
#> clang: warning: argument unused during compilation: '-mf16c' [-Wunused-command-line-argument]

Theses -mavx -mavx2 -mfma -mf16c are non-Mac settings.

@jmgirard
Copy link
Contributor

Theses -mavx -mavx2 -mfma -mf16c are non-Mac settings.

Gotcha. I was confused because the README says:

"Common settings to set for WHISPER_CFLAGS on Mac/Linux/Windows are -mavx -mavx2 -mfma -mf16c and extra possible flags -msse3 and mssse3"

@jwijffels
Copy link
Contributor Author

I've enabled the use of Mac Metal (I think - I'm not sure - haven't got a Mac myself).
Would be great if you could test with this and see if that speeds up by doing this transcription for example: #24 (comment)

Sys.setenv(WHISPER_ACCELERATE = "1")
Sys.setenv(WHISPER_METAL = "1")
remotes::install_github("bnosac/audio.whisper", force = TRUE)

@jmgirard
Copy link
Contributor

Error when trying to build for METAL
── R CMD build ──────────────────────────────────────────────────
✔  checking for file ‘/private/var/folders/kr/tx86v16n5bx_djz_z2cpvfkc0000gq/T/Rtmpd2uqRK/remotes11bb064364aec/bnosac-audio.whisper-06e569c/DESCRIPTION’ ...
─  preparing ‘audio.whisper’:
✔  checking DESCRIPTION meta-information
─  cleaning src
─  checking for LF line-endings in source and make files and shell scripts
─  checking for empty or unneeded directories
─  building ‘audio.whisper_0.3.1.tar.gz’
   
Installing package into ‘/opt/homebrew/lib/R/4.3/site-library’
(as ‘lib’ is unspecified)
* installing *source* package ‘audio.whisper’ ...
** using staged installation
** libs
using C++ compiler: ‘Apple clang version 15.0.0 (clang-1500.1.0.2.5)’
using C++11
using SDK: ‘MacOSX14.2.sdk’
I whisper.cpp build info: 
I UNAME_S:  Darwin
I UNAME_P:  arm
I UNAME_M:  arm64
I PKG_CFLAGS:   -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread
I PKG_CPPFLAGS: -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread
I PKG_LIBS:   -framework Accelerate -framework Foundation -framework Metal -framework MetalKit

clang -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include   -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -fPIC  -g -O2  -c whisper_cpp/ggml-quants.c -o whisper_cpp/ggml-quants.o
clang -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include   -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -fPIC  -g -O2  -c whisper_cpp/ggml-backend.c -o whisper_cpp/ggml-backend.o
clang -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include   -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -fPIC  -g -O2  -c whisper_cpp/ggml-alloc.c -o whisper_cpp/ggml-alloc.o
clang -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include   -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -fPIC  -g -O2  -c whisper_cpp/ggml.c -o whisper_cpp/ggml.o
whisper_cpp/ggml.c:9828:17: warning: 'cblas_sgemm' is deprecated: first deprecated in macOS 13.3 - An updated CBLAS interface supporting ILP64 is available.  Please compile with -DACCELERATE_NEW_LAPACK to access the new headers and -DACCELERATE_LAPACK_ILP64 for ILP64 support. [-Wdeprecated-declarations]
                cblas_sgemm(CblasRowMajor, CblasNoTrans, CblasTrans,
                ^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/vecLib.framework/Headers/cblas.h:610:6: note: 'cblas_sgemm' has been explicitly marked deprecated here
void cblas_sgemm(const enum CBLAS_ORDER __Order,
     ^
whisper_cpp/ggml.c:10238:9: warning: 'cblas_sgemm' is deprecated: first deprecated in macOS 13.3 - An updated CBLAS interface supporting ILP64 is available.  Please compile with -DACCELERATE_NEW_LAPACK to access the new headers and -DACCELERATE_LAPACK_ILP64 for ILP64 support. [-Wdeprecated-declarations]
        cblas_sgemm(CblasRowMajor, transposeA, CblasNoTrans, m, n, k, 1.0, a, lda, b, n, 0.0, c, n);
        ^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/vecLib.framework/Headers/cblas.h:610:6: note: 'cblas_sgemm' has been explicitly marked deprecated here
void cblas_sgemm(const enum CBLAS_ORDER __Order,
     ^
2 warnings generated.
clang++ -std=gnu++11 -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include    -fPIC  -g -O2  -c whisper_cpp/whisper.cpp -o whisper_cpp/whisper.o
clang++ -std=gnu++11 -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include    -fPIC  -g -O2  -c whisper_cpp/common-ggml.cpp -o whisper_cpp/common-ggml.o
clang++ -std=gnu++11 -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include    -fPIC  -g -O2  -c whisper_cpp/common.cpp -o whisper_cpp/common.o
clang++ -std=gnu++11 -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include    -fPIC  -g -O2  -c rcpp_whisper.cpp -o rcpp_whisper.o
clang++ -std=gnu++11 -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include    -fPIC  -g -O2  -c RcppExports.cpp -o RcppExports.o
make: *** No rule to make target `+', needed by `audio.whisper.so'.  Stop.
ERROR: compilation failed for package ‘audio.whisper’
* removing ‘/opt/homebrew/lib/R/4.3/site-library/audio.whisper’
* restoring previous ‘/opt/homebrew/lib/R/4.3/site-library/audio.whisper’
Warning message:
In i.p(...) :
  installation of package ‘/var/folders/kr/tx86v16n5bx_djz_z2cpvfkc0000gq/T//Rtmpd2uqRK/file11bb02eb10497/audio.whisper_0.3.1.tar.gz’ had non-zero exit status

@jwijffels
Copy link
Contributor Author

jwijffels commented Jan 30, 2024

Right, sorry about your time, probably this metal is fixed with this commit 32758b8
If it doesn't, I'll probably have to look for spinning up some Metal in AWS. These metal optimisations are new to me, I hope it is as simple at that commit but it might not be.

@jmgirard
Copy link
Contributor

jmgirard commented Jan 30, 2024

Not a big deal to try this on my end.

Different error
── R CMD build ──────────────────────────────────────────────────
✔  checking for file ‘/private/var/folders/kr/tx86v16n5bx_djz_z2cpvfkc0000gq/T/Rtmp0xWYAY/remotes12e63549897e2/bnosac-audio.whisper-32758b8/DESCRIPTION’ ...
─  preparing ‘audio.whisper’:
✔  checking DESCRIPTION meta-information
─  cleaning src
─  checking for LF line-endings in source and make files and shell scripts
─  checking for empty or unneeded directories
─  building ‘audio.whisper_0.3.1.tar.gz’
   
Installing package into ‘/opt/homebrew/lib/R/4.3/site-library’
(as ‘lib’ is unspecified)
* installing *source* package ‘audio.whisper’ ...
** using staged installation
** libs
using C++ compiler: ‘Apple clang version 15.0.0 (clang-1500.1.0.2.5)’
using C++11
using SDK: ‘MacOSX14.2.sdk’
I whisper.cpp build info: 
I UNAME_S:  Darwin
I UNAME_P:  arm
I UNAME_M:  arm64
I PKG_CFLAGS:   -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread
I PKG_CPPFLAGS: -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread
I PKG_LIBS:   -framework Accelerate -framework Foundation -framework Metal -framework MetalKit

clang -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include   -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -fPIC  -g -O2  -c whisper_cpp/ggml-quants.c -o whisper_cpp/ggml-quants.o
clang -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include   -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -fPIC  -g -O2  -c whisper_cpp/ggml-backend.c -o whisper_cpp/ggml-backend.o
clang -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include   -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -fPIC  -g -O2  -c whisper_cpp/ggml-alloc.c -o whisper_cpp/ggml-alloc.o
clang -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include   -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -fPIC  -g -O2  -c whisper_cpp/ggml.c -o whisper_cpp/ggml.o
whisper_cpp/ggml.c:9828:17: warning: 'cblas_sgemm' is deprecated: first deprecated in macOS 13.3 - An updated CBLAS interface supporting ILP64 is available.  Please compile with -DACCELERATE_NEW_LAPACK to access the new headers and -DACCELERATE_LAPACK_ILP64 for ILP64 support. [-Wdeprecated-declarations]
                cblas_sgemm(CblasRowMajor, CblasNoTrans, CblasTrans,
                ^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/vecLib.framework/Headers/cblas.h:610:6: note: 'cblas_sgemm' has been explicitly marked deprecated here
void cblas_sgemm(const enum CBLAS_ORDER __Order,
     ^
whisper_cpp/ggml.c:10238:9: warning: 'cblas_sgemm' is deprecated: first deprecated in macOS 13.3 - An updated CBLAS interface supporting ILP64 is available.  Please compile with -DACCELERATE_NEW_LAPACK to access the new headers and -DACCELERATE_LAPACK_ILP64 for ILP64 support. [-Wdeprecated-declarations]
        cblas_sgemm(CblasRowMajor, transposeA, CblasNoTrans, m, n, k, 1.0, a, lda, b, n, 0.0, c, n);
        ^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/vecLib.framework/Headers/cblas.h:610:6: note: 'cblas_sgemm' has been explicitly marked deprecated here
void cblas_sgemm(const enum CBLAS_ORDER __Order,
     ^
2 warnings generated.
clang++ -std=gnu++11 -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include    -fPIC  -g -O2  -c whisper_cpp/whisper.cpp -o whisper_cpp/whisper.o
clang++ -std=gnu++11 -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include    -fPIC  -g -O2  -c whisper_cpp/common-ggml.cpp -o whisper_cpp/common-ggml.o
clang++ -std=gnu++11 -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include    -fPIC  -g -O2  -c whisper_cpp/common.cpp -o whisper_cpp/common.o
clang++ -std=gnu++11 -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include    -fPIC  -g -O2  -c rcpp_whisper.cpp -o rcpp_whisper.o
clang++ -std=gnu++11 -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include    -fPIC  -g -O2  -c RcppExports.cpp -o RcppExports.o
make: *** No rule to make target `ggml-metal.o', needed by `audio.whisper.so'.  Stop.
ERROR: compilation failed for package ‘audio.whisper’
* removing ‘/opt/homebrew/lib/R/4.3/site-library/audio.whisper’
* restoring previous ‘/opt/homebrew/lib/R/4.3/site-library/audio.whisper’
Warning message:
In i.p(...) :
  installation of package ‘/var/folders/kr/tx86v16n5bx_djz_z2cpvfkc0000gq/T//Rtmp0xWYAY/file12e634c64562c/audio.whisper_0.3.1.tar.gz’ had non-zero exit status

@jwijffels
Copy link
Contributor Author

Thanks for your time, I'll move this Metal integration to issue #34

@jwijffels
Copy link
Contributor Author

My macbook's system information says it supports Metal 3, but I'm not sure if I need to install anything to enable it.

I just installed CoreML via pip install coremltools. Do you need me to install any third-party packages?

Now that we have Accelerate/Metal working, I also enables coreml (#35)

@jwijffels
Copy link
Contributor Author

@shikokuchuo Feel free to test auto the CUDA integration - it's enabled on the master branch (see #27).

@shikokuchuo
Copy link
Contributor

Fantastic - I will take it for a spin when I get some time!

@shikokuchuo
Copy link
Contributor

Wow the sample file translation was instantaneous. May be able to actually make use of this now - I've put it on my list for closer inspection. Thanks for letting me know about the update.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants