-
Notifications
You must be signed in to change notification settings - Fork 13
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
upgrade to 1.5.0 #24
Comments
Would be great to run this with the new CUDA support! |
Work on this is in branch upgrade-v1.5.4 at https://github.com/bnosac/audio.whisper/tree/upgrade-v1.5.4 |
I've upgraded the R package to use whisper.cpp 1.5.4. A looot of backend changes. @shikokuchuo Feel free to test it out. For the CUDA support. I'll plan to look at it once I get my hands on a machine with CUDA on. |
Yep tested briefly - the quantised models quite a bit faster. Do you know what it would take to enable CUDA? I can try testing it out if there are details somewhere. Or can the package link to system libs (that I can then build with CUDA support)? |
I can test on Mac and windows. |
I also saw that.
I'll move this question to #27 |
Installation with default CFLAGS works on MacOS (M2 macbook). Similarly with However, I get some warnings with the recommended CFLAGS:
|
Do you have the Mac Accelerate installed? That would help in testing. |
I just installed accelerate via Sys.setenv(WHISPER_ACCELERATE = "1")
remotes::install_github("bnosac/audio.whisper", ref = "0.3", force = TRUE) Seems to work but generates the following warnings:
Let me know if there are other things you want me to try on Mac. |
Looks indeed ok. Does that speedup transcription? |
My macbook's system information says it supports Metal 3, but I'm not sure if I need to install anything to enable it. I just installed CoreML via |
What do you recommend for systematically testing speed? Formal benchmarking or something like tictoc? Also, would the whisper model and/or duration of the audio file matter for getting a good estimate? |
Just get a longer audio file en see how long it takes if you have the package installed with WHISPER_ACCELERATE and without - make sure you remove the environment variable
|
I need to make changes in our Makevars file to enable Metal and CoreML. This will be for later. |
16.65 min with accelerate vs. 47.8 min without |
Great so that accelerate option works on Mac. I will get back to this later for these coreml/metal, once I made the changes in the Makevars to enable these. |
Theses -mavx -mavx2 -mfma -mf16c are non-Mac settings. |
Gotcha. I was confused because the README says: "Common settings to set for WHISPER_CFLAGS on Mac/Linux/Windows are -mavx -mavx2 -mfma -mf16c and extra possible flags -msse3 and mssse3" |
I've enabled the use of Mac Metal (I think - I'm not sure - haven't got a Mac myself).
|
Error when trying to build for METAL
|
Right, sorry about your time, probably this metal is fixed with this commit 32758b8 |
Not a big deal to try this on my end. Different error
|
Thanks for your time, I'll move this Metal integration to issue #34 |
Now that we have Accelerate/Metal working, I also enables coreml (#35) |
@shikokuchuo Feel free to test auto the CUDA integration - it's enabled on the master branch (see #27). |
Fantastic - I will take it for a spin when I get some time! |
Wow the sample file translation was instantaneous. May be able to actually make use of this now - I've put it on my list for closer inspection. Thanks for letting me know about the update. |
Stable release is now 1.5.0 and available at https://github.com/ggerganov/whisper.cpp/releases/tag/v1.5.0
The text was updated successfully, but these errors were encountered: