-
Notifications
You must be signed in to change notification settings - Fork 536
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adding CMake and VS2019/2022 support #1398
Comments
could you please explain to me how to crosscompile a windows static binary of essentia with gaia? |
@Lovecraft-github To me, Gaia and the related machine learning models look abandoned. That's why I haven't done any tests besides checking that the Gaia cmake project at https://github.com/wo80/gaia/tree/cmake compiles and links ok with the Essentia cmake project. If you haven't tried yet, I suggest you use the code from that branch. Please also read the guide - after compiling and installing Gaia, you will have to configure Essentia with
I haven't tried to cross-compile the cmake project. The whole point of this, was to get it working with Visual Studio. So if you are on Linux trying to compile for Windows, I'm probably not much of a help. |
Firstly, thank you very much @wo80 for the PR!!! We'll be looking at it soon. To clarify the situation, we are planning to get rid of dependency on Gaia. It is there for the SVM classification models support (GaiaTransform algorithm), however those are outdated by now. Our current TensorFlow models have a significantly better performance in terms of accuracy and generalization, so we need to remove Gaia and add support for TensorFlow on Windows instead. |
@wo80 I managed to build it for linux, but I didn't find a way to do it for windows. It was a long shot to ask you. I'm with you that seems abandoned. Thanks anyway. @dbogdanov So we could say that gaia2 is officially gone. Well, that at least solves the problem of continuing trying :-P. It is planned to release static binaries or installer for windows in a near future? or make it possible to cross-compiling it?. And last, any estimation when it will happen? |
@Lovecraft-github I've updated some of the CMake modules which should also fix problems finding the Gaia package. The latest code is in the msvc-shared-lib branch - in case you still want to use the old models. @dbogdanov I've also fixed issues with the Tensorflow support. I have linked with recent Tensorflow v2.15.0 and everything builds fine. I've successfully run the
I have to say, it's a bit unclear which models are supposed to work with which examples... |
Hi everyone, first of all thank you for the joint efforts and time spent creating and maintaining this wonder! I started using the lib successfully on macOS and Linux with python bindings, and now I've been curious to make it work also on Windows. It's not working out of the box with classic
Going to describe what I've done in reproducible detail. Every shell command has been issued in a A. Build environment setup from scratch
B. Get essentia sources and build dependenciesNavigate to your favorite working directory, then Run # clone essentia
git clone --recurse-submodules https://github.com/wo80/essentia.git -b cmake
cd essentia
# install vcpkg locally, to resolve C++ lib dependencies
git clone https://github.com/microsoft/vcpkg
.\vcpkg\bootstrap-vcpkg.bat
# get dependencies with vcpkg
.\vcpkg\vcpkg install eigen3
.\vcpkg\vcpkg install fftw3
.\vcpkg\vcpkg install libyaml
# this one is veery long, as it also compiles ffmpeg (may take up to 40 mins)
.\vcpkg\vcpkg install chromaprint
.\vcpkg\vcpkg install libsamplerate
.\vcpkg\vcpkg install taglib
pip install numpy C. Build essentia with python bindingsFrom the same # configure essentia build
cmake -B build -D BUILD_PYTHON_BINDINGS=ON -S . "-DCMAKE_TOOLCHAIN_FILE=vcpkg/scripts/buildsystems/vcpkg.cmake"
# build and test essentia
cmake --build build --config Release
ctest --test-dir build --output-on-failure -C Release D. OutcomeThe build is successful and the tests are passing. I see also the compiled wheel inside the
I'm getting:
the output from
Where am I doing it wrong? How to make it compile for Win64 instead? Any hint is appreciated. |
@gotev You could try specifying the target platform explictily, like And can you post the complete output of the CMake configure step? |
By keeping everything else as my previous comment, except for the commands run at paragraph if (Test-Path .\build) { Remove-Item -Path .\build -Recurse -Force }
cmake -G "Visual Studio 17 2022" -A x64 -B build -D BUILD_PYTHON_BINDINGS=ON -S . "-DCMAKE_TOOLCHAIN_FILE=vcpkg/scripts/buildsystems/vcpkg.cmake"
# build and test essentia
cmake --build build --config Release
ctest --test-dir build --output-on-failure -C Release I'm still getting During the final steps of the build, I intercepted this from the console output: Creating library C:/Users/Admin/Desktop/essentia/build/src/python/Release/_essentia.cp312-win_amd64.lib and object
C:/Users/Admin/Desktop/essentia/build/src/python/Release/_essentia.cp312-win_amd64.exp
_essentia.vcxproj -> C:\Users\Admin\Desktop\essentia\build\src\python\Release\_essentia.cp312-win_amd64.pyd
Building Custom Rule C:/Users/Admin/Desktop/essentia/src/python/CMakeLists.txt
Processing c:\users\admin\desktop\essentia\build\src\python
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Installing backend dependencies: started
Installing backend dependencies: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'done'
Building wheels for collected packages: essentia
Building wheel for essentia (pyproject.toml): started
Building wheel for essentia (pyproject.toml): finished with status 'done'
Created wheel for essentia: filename=essentia-2.1b6.dev0-cp312-cp312-win32.whl size=1978664 sha256=430f279d9b4d2be2
ebe9c974d0525a7e46d2a6130a5fa0875236ac25143ba19e
Stored in directory: C:\Users\Admin\AppData\Local\Temp\pip-ephem-wheel-cache-odjj24xg\wheels\2b\11\7b\0b358554c85e0
36a6917648cc3dceefc2379e67ac2793b5169
Successfully built essentia Full output from: cmake -G "Visual Studio 17 2022" -A x64 -B build -D BUILD_PYTHON_BINDINGS=ON -S . "-DCMAKE_TOOLCHAIN_FILE=vcpkg/scripts/buildsystems/vcpkg.cmake" is the following: -- Selecting Windows SDK version 10.0.22621.0 to target Windows 10.0.19045.
-- The C compiler identification is MSVC 19.39.33521.0
-- The CXX compiler identification is MSVC 19.39.33521.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/VC/Tools/MSVC/14.39.33519/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/VC/Tools/MSVC/14.39.33519/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found FFmpeg: C:/Users/Admin/Desktop/essentia/vcpkg/installed/x64-windows/debug/lib/avcodec.lib;C:/Users/Admin/Desktop/essentia/vcpkg/installed/x64-windows/debug/lib/avformat.lib;C:/Users/Admin/Desktop/essentia/vcpkg/installed/x64-windows/debug/lib/avutil.lib;C:/Users/Admin/Desktop/essentia/vcpkg/installed/x64-windows/debug/lib/swresample.lib
-- Found SampleRate: C:/Users/Admin/Desktop/essentia/vcpkg/installed/x64-windows/debug/lib/samplerate.lib (found version "0.2.2")
-- Found Taglib: C:/Users/Admin/Desktop/essentia/vcpkg/installed/x64-windows/debug/lib/tag.lib (found version "2.0")
-- Found Chromaprint: C:/Users/Admin/Desktop/essentia/vcpkg/installed/x64-windows/debug/lib/chromaprint.lib (found version "1.5.1")
-- Found YAML: C:/Users/Admin/Desktop/essentia/vcpkg/installed/x64-windows/debug/lib/yaml.lib (found version "0.1")
-- Found Python3: C:/Python312/libs/python312.lib (found version "3.12.2") found components: Development NumPy Interpreter Development.Module Development.Embed
-- Found FFTW3f: C:/Users/Admin/Desktop/essentia/vcpkg/installed/x64-windows/debug/lib/fftw3f.lib (found version "3.3.9")
-- Found Python3: C:/Python312/python.exe (found version "3.12.2") found components: Interpreter Development Development.Module Development.Embed
-- Found Python3: C:/Python312/python.exe (found version "3.12.2") found components: Interpreter
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - not found
-- Found Threads: TRUE
-- Configuring done (31.0s)
-- Generating done (0.5s)
-- Build files have been written to: C:/Users/Admin/Desktop/essentia/build |
Got it one step further. Executed steps A and B from my first comment and then: if (Test-Path .\build) { Remove-Item -Path .\build -Recurse -Force }
cmake -G "Visual Studio 17 2022" -A x64 -B build -D BUILD_PYTHON_BINDINGS=ON -S . "-DCMAKE_TOOLCHAIN_FILE=vcpkg/scripts/buildsystems/vcpkg.cmake" Then I manually edited: code .\build\src\python\setup.py by adding this line: options={'bdist_wheel': {'plat_name': 'win_amd64'}}, after line 86, so the setup in the file looks like this: setup(
..
cmdclass = {
'bdist': bdist,
'sdist': sdist,
'install_lib': install_lib,
'build_ext': cmake_build_ext,
},
options={'bdist_wheel': {'plat_name': 'win_amd64'}},
) Then, by running: cmake --build build --config Release
ctest --test-dir build --output-on-failure -C Release the lib compiles, tests passes and the 64 bit wheel gets generated: I'm able to install it successfully in the virtualenv: .\venv\Scripts\activate
pip install ..\essentia\build\wheel\essentia-2.1b6.dev0-cp312-cp312-win_amd64.whl Processing c:\users\admin\desktop\essentia\build\wheel\essentia-2.1b6.dev0-cp312-cp312-win_amd64.whl
Collecting numpy>=1.8.2 (from essentia==2.1b6.dev0)
Using cached numpy-1.26.4-cp312-cp312-win_amd64.whl.metadata (61 kB)
Collecting pyyaml (from essentia==2.1b6.dev0)
Using cached PyYAML-6.0.1-cp312-cp312-win_amd64.whl.metadata (2.1 kB)
Collecting six (from essentia==2.1b6.dev0)
Using cached six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB)
Using cached numpy-1.26.4-cp312-cp312-win_amd64.whl (15.5 MB)
Using cached PyYAML-6.0.1-cp312-cp312-win_amd64.whl (138 kB)
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Installing collected packages: six, pyyaml, numpy, essentia
Successfully installed essentia-2.1b6.dev0 numpy-1.26.4 pyyaml-6.0.1 six-1.16.0 But then when I try to import it: python Python 3.12.2 (tags/v3.12.2:6abddd9, Feb 6 2024, 21:26:36) [MSC v.1937 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import essentia
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\Admin\Desktop\myscript\venv\Lib\site-packages\essentia\__init__.py", line 1, in <module>
from . import _essentia
ImportError: DLL load failed while importing _essentia: The specified module could not be found. It fails with this error 🤔 |
That manual editing of The error you are getting is probably due to vcpkg building shared libs, and python does not know where to find them. You need to add that path manually (the path used below is just a guess from the output you provided): import os
os.add_dll_directory("C:/Users/Admin/Desktop/essentia/vcpkg/installed/x64-windows/debug/bin")
import essentia
... |
@wo80 great! Yeah the path is correct. I can now import the library:
but I wonder if there's a way to install those libs in standard dirs wth vcpkg (inside the virtualenv), so I don't have to write this platform specific python code for Windows. This line caught my attention:
that |
I used the installer provided on python.org, and I'm quite confident, that chocolatey does the same.
First, take a look at https://docs.python.org/3/whatsnew/3.8.html#bpo-36085-whatsnew . That means, shared dependencies have to be installed in a system path or the wheel directory. I'm not a python guy, but my guess is, even in case this is done, some manual loading might be necessary in I have made improvements linking with static dependencies in wo80@61d8947 , but there are still problems - for example linking the FFmpeg libraries statically, you also need to add Windows' EDIT: you can try building static libraries with vcpkg using |
Retried again all the steps, by also recreating the VM from scratch and explicitly specifying x64 architecture when installing python from chocolatey (it's using the installer from python.org) and also dependencies with vcpkg. To sum it up, those are all the updated steps from scratch: A. Build environment setup from scratch
B. Get essentia sources and build dependenciesNavigate to your favorite working directory, then Run # clone essentia
git clone --recurse-submodules https://github.com/wo80/essentia.git -b cmake
cd essentia
# install vcpkg locally, to resolve C++ lib dependencies
git clone https://github.com/microsoft/vcpkg
.\vcpkg\bootstrap-vcpkg.bat
# get dependencies with vcpkg
.\vcpkg\vcpkg install eigen3:x64-windows
.\vcpkg\vcpkg install fftw3:x64-windows
.\vcpkg\vcpkg install libyaml:x64-windows
.\vcpkg\vcpkg install libsamplerate:x64-windows
.\vcpkg\vcpkg install taglib:x64-windows
# this one is veery long, as it also compiles ffmpeg (may take up to 40 mins)
.\vcpkg\vcpkg install chromaprint:x64-windows
pip install numpy C. Build essentia with python bindingsFrom the same if (Test-Path .\build) { Remove-Item -Path .\build -Recurse -Force }
cmake -G "Visual Studio 17 2022" -A x64 -B build -D BUILD_PYTHON_BINDINGS=ON -S . "-DCMAKE_TOOLCHAIN_FILE=vcpkg/scripts/buildsystems/vcpkg.cmake"
# build and test essentia
cmake --build build --config Release
ctest --test-dir build --output-on-failure -C Release After the build completes: cmake --build build --config Release I'm still getting only To workaround this (haven't found what's causing the issue yet, but replicated it the same from scratch 2 times) and prepare a 64 bit wheel with the needed DLLs bundled in, without manually editing any file, I found this way: pip install wheel setuptools delvewheel
# from inside the essentia sources root directory, after the build is completed
cd .\build\src\python
# manually build the 64 bit wheel
python setup.py bdist_wheel --plat-name win_amd64
# get to the wheel output directory
cd ..\..\wheel
# find the exact name of the 64 bit wheel
$wheel_name = (Get-ChildItem -Path . -Filter "*amd64.whl" | Select-Object -First 1).Name
# show missing DLLs
delvewheel show $wheel_name --add-path ..\..\vcpkg\installed\x64-windows\bin
# integrate missing DLLs in the wheel and repackage it
delvewheel repair $wheel_name --add-path ..\..\vcpkg\installed\x64-windows\bin -w . Then, to test it out: python -m venv venv
.\venv\Scripts\activate
pip install .\essentia-2.1b6.dev0-cp312-cp312-win_amd64.whl install output
and boom, it's now imported without any additional statements! Now it's very late, so I'll leave further refinements and tests for another time! Further tunings will certainly be needed to avoid workarounds, enable all the features and be able to use the full fledged library as on macOS and Linux. |
@wo80 @dbogdanov managed to get a completely working 64 bit wheel for Windows (with DLLs bundled in the wheel, so no static libs needed) with TensorFlow support. There are some warnings here and there, but it's working 🎉 This is the working recipe from scratch to finish: A. Build environment setup from scratch
B. Get essentia sources and build dependenciesNavigate to your favorite working directory, then Run # clone essentia
git clone --recurse-submodules https://github.com/wo80/essentia.git -b cmake
cd essentia
# install vcpkg locally, to resolve C++ lib dependencies
git clone https://github.com/microsoft/vcpkg
.\vcpkg\bootstrap-vcpkg.bat
# get dependencies with vcpkg
.\vcpkg\vcpkg install eigen3:x64-windows
.\vcpkg\vcpkg install fftw3:x64-windows
.\vcpkg\vcpkg install libyaml:x64-windows
.\vcpkg\vcpkg install libsamplerate:x64-windows
.\vcpkg\vcpkg install taglib:x64-windows
# this one is veery long, as it also compiles ffmpeg (may take up to 40 mins)
.\vcpkg\vcpkg install chromaprint:x64-windows
pip install numpy Get TensorFlow. Save this in $tfRemoteUrl = "https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-cpu-windows-x86_64-2.15.0.zip"
$tfPatchHeadersUrl = "https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-cpu-linux-x86_64-2.15.0.tar.gz"
$tfTempPath = ".\tf-temp"
$zipFile = ".\tf.zip"
$zipPatchHeadersFile = ".\tf-patch-headers.tar.gz"
if (Test-Path $tfTempPath) {
Remove-Item -Path $tfTempPath -Recurse -Force
}
mkdir $tfTempPath
Set-Location $tfTempPath
$ProgressPreference = 'SilentlyContinue'
Write-Host "Downloading $tfRemoteUrl ..."
Invoke-WebRequest -Uri $tfRemoteUrl -OutFile $zipFile
Write-Host "Downloading $tfPatchHeadersUrl ..."
Invoke-WebRequest -Uri $tfPatchHeadersUrl -OutFile $zipPatchHeadersFile
$ProgressPreference = 'Continue'
Write-Host "Extracting $tfRemoteUrl ..."
Expand-Archive -Path $zipFile -DestinationPath .
Write-Host "Extracting $zipPatchHeadersFile ..."
mkdir .\tf-patch-headers
tar -xvzf $zipPatchHeadersFile -C .\tf-patch-headers
Remove-Item $zipFile
Remove-Item $zipPatchHeadersFile
Write-Host "Copying files ..."
Copy-Item .\lib\tensorflow.dll ..\vcpkg\installed\x64-windows\bin -Force
Copy-Item .\lib\tensorflow.lib ..\vcpkg\installed\x64-windows\lib -Force
Copy-Item .\tf-patch-headers\include\* -Destination ..\vcpkg\installed\x64-windows\include -Recurse -Force
Copy-Item .\include\* -Destination ..\vcpkg\installed\x64-windows\include -Recurse -Force
Set-Location .. C. Build essentia with TensorFlow and python bindingsFrom the same if (Test-Path .\build) { Remove-Item -Path .\build -Recurse -Force }
cmake -G "Visual Studio 17 2022" -A x64 -B build -D BUILD_PYTHON_BINDINGS=ON -D USE_TENSORFLOW=ON -S . "-DCMAKE_TOOLCHAIN_FILE=vcpkg/scripts/buildsystems/vcpkg.cmake"
# build and test essentia
cmake --build build --config Release
ctest --test-dir build --output-on-failure -C Release D. Package 64 bit wheel with all needed DLLsFrom the same pip install wheel setuptools delvewheel
# from inside the essentia sources root directory, after the build is completed
cd .\build\src\python
# manually build the 64 bit wheel
python setup.py bdist_wheel --plat-name win_amd64
# get to the wheel output directory
cd ..\..\wheel
# find the exact name of the 64 bit wheel
$wheel_name = (Get-ChildItem -Path . -Filter "*amd64.whl" | Select-Object -First 1).Name
# show missing DLLs
delvewheel show $wheel_name --add-path ..\..\vcpkg\installed\x64-windows\bin
# integrate missing DLLs in the wheel and repackage it
delvewheel repair $wheel_name --add-path ..\..\vcpkg\installed\x64-windows\bin -w . The resulting wheel will be in E. Install the wheel and import essentiapython -m venv venv
.\venv\Scripts\activate
pip install path\to\wheel.whl
|
@gotev That's great! You're the first one reporting back, so thank you for your efforts! Can you test the following in PowerShell: cd build/wheel
tar -xf essentia-2.1b6.dev0-cp312-cp312-win32.whl
Get-ChildItem -Path essentia -Recurse -Filter "*.pyd"
Select-String -Path essentia-2.1b6.dev0.dist-info/WHEEL -Pattern "Tag:" If it prints
then the weird EDIT: import sysconfig
sysconfig.get_platform() I still want to know where that |
cd build/wheel
tar -xf essentia-2.1b6.dev0-cp312-cp312-win32.whl
Get-ChildItem -Path essentia -Recurse -Filter "*.pyd"
Select-String -Path essentia-2.1b6.dev0.dist-info/WHEEL -Pattern "Tag:" outputs: PS C:\Users\Admin\Desktop\essentia\build\wheel> tar -xf essentia-2.1b6.dev0-cp312-cp312-win32.whl
PS C:\Users\Admin\Desktop\essentia\build\wheel> Get-ChildItem -Path essentia -Recurse -Filter "*.pyd"
Directory: C:\Users\Admin\Desktop\essentia\build\wheel\essentia
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 3/12/2024 2:03 PM 6030848 _essentia.cp312-win_amd64.pyd
PS C:\Users\Admin\Desktop\essentia\build\wheel> Select-String -Path essentia-2.1b6.dev0.dist-info/WHEEL -Pattern "Tag:"
essentia-2.1b6.dev0.dist-info\WHEEL:4:Tag: cp312-cp312-win32 Python 3.12.2 (tags/v3.12.2:6abddd9, Feb 6 2024, 21:26:36) [MSC v.1937 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sysconfig
>>> sysconfig.get_platform()
'win-amd64' Try replicating my whole procedure using your favorite VM solution and let me know if you have the same outcome. I've read somewhere that MS kept |
Yeah, very strange. I'm running Windows 10 22H2 fully updated on bare metal with VS 2022. The only difference being
Anyways, it's good to have the detailed info available here and I might add a dedicated Python section to my guide. |
@wo80 if you post here the exact commands you're using for the dependencies, I can try them on my VM instead of using vspkg and see if the outcome changes. Also, as I'm running the whole build env in a VM on an Intel macOS Sonoma 14.4, if there's a way to run the build in a windows docker instead of a VM, we'll have the CI pipeline ready for win builds and it will be a lot faster and portable, getting rid of all the manual operations. I've noticed that vspkg packages are not always aligned to latest versions and although it's convenient to use because it's straightforward, you have less control on what gets built, for example I couldn't tune ffmpeg. |
@gotev I basically do what's in the GitHub workflow: It doesn't build the Python wheel, though. This might be a good opportunity to add that to the workflow. |
I tested building the wheel in the Github workflow and the right platform is picked automatically, so still a mystery where that The Python setup is a bit hacky since |
Hi, I installed essentia following this procedure. Thanks @gotev, I couldn't have done it without this very detailed step-by-step.
And it worked. Details not important
I made a small test with tensorflow models and I have slightly different results between this installation, and another one in WSL (Windows Subsystem for Linux) installed with the command
I get these messages when I run a script:
The calculation is faster in Windows than in WSL. |
I have modified the Windows workflow job to use a batch script for downloading and installing the dependencies: This should be straightforward for users to run on a Windows machine. I'm also playing around with an audio-only FFmpeg workflow for Windows / MSVC: Shared and static build artifacts are available, but I haven't had time to test those yet. Using the audio-only libraries could reduce the size of a self-contained Essentia package considerably (like, for example a Python wheel), with TensorFlow remaining the only huge dependency. The plan is to make official releases, which could then be used instead of github.com/GyanD/codexffmpeg/releases in the batch script. |
@wo80 great job! We should be just fine with the shared libs (DLLs) for the wheel, but I think we should find a way to get rid of
to the |
A bit late in the game, I followed all the steps from you @gotev to build the python_wheel for amd64 on a Windows machine. Everything works fine, I can install the wheel but.... then:
Any ideas what could have gone wrong? |
@FelixKarlheinz I've seen this error a lot when I started work on building the Python Windows package with CMake. It just means that the native pyd binary could not be loaded, but it's not very helpful when trying to find out what went wrong. Please check which version of Python was used in your build (by examining the CMake configure output) I recently configured the python-wheel branch to upload build artifacts, see for example
All dependencies are linked statically. Tensorflow algorithms are disabled, but I could modify the workflow to include them. You could either test that wheel or just download the artifact and compare the contents to your installation and see if anything is different (edit: anything not related to the pyd binary). |
@wo80 thank you for your fast answer. The thing is I mainly use the genre classification with the discogs-effnet model as a pre-process in my pipeline so tensorflow is the crucial thing I want to get to run on Windows :). I will try your tips, thank you! Regarding the static linking, what would be the steps I need to do to include the tensorflow algorithms? Much appreciate your work! |
@FelixKarlheinz Regarding the Essentia build, only the Tensorflow for Windows is distributed as a shared library only, in the latest iteration of packaging/build-dependencies-msvc.bat I use version 2.16.1. Since there is no static Tensorflow library to link, you will need to take further steps to make the Python wheel work. You can do one of the following:
The So, to wrap up, you can use
and then try one of the three options above to handle the missing Tensorflow dll (which will be available in |
@wo80 Starting with the good news, I can install your wheel that you provided in the artifacts folder and I can import essentia. I tried to basically replicate build-cmake_yml. Unfortunately, cmake is not finding any libs (even though prefix path is set) so I had to point to them individually which worked except for FFTW3. After the build, I build the python_wheel but I still end up with the circular import error. I will keep investigating why this is not working on my machine with the same commands as in your build job. |
@wo80 Do you happen to have time to include tensorflow into your python_wheel? |
@FelixKarlheinz The artifact is available at https://github.com/wo80/essentia/actions/runs/9783114926 . It does not contain the Tensorflow dll, so you still have to fix that choosing one of the options explained above. The wheel is built from the new I've also added a section regarding Python in my build instructions. |
@dbogdanov Any news? Even if you decide not to have a second, competing build system, I guess it would make sense to integrate some of the changes listed in the 8 points of my opening post. Should I just start doing pull requests? |
Voicing support for this. A first-class CMake build implementation would make it easier for new users to navigate around existing deficiencies in build-related documentation and behavior. Thanks for the excellent work starting this effort @wo80! |
This is not an issue. I opened it here, because I wasn't sure, if other issues were notified correctly if this was opened in the discussions section.
I've been working on Visual Studio compatibility lately (well, picked up work from a year ago), see https://github.com/wo80/essentia/
Here's what's in it:
There's a GitHub workflow and a short guide available showing how to compile the project.
If you are interested in merging this, I think the changes in the cmake branch should be split up into smaller pull requests. I'd suggest the following changes to go into separate PRs and that detailed discussion being done there:
using namespace std
from headers, see CppCoreGuidelines#Rs-using-directive (wo80@b510967)Regarding point 6: having the Visual Studio project files doesn't make sense with the CMake script in place. The win32 3rd party code is completely out of date, so I'd suggest, if you want to keep the option to cross-compile for Windows, all dependencies should be pulled from the online repos (like done in my GitHub workflow) and the versions being updated on a regular basis. I might update the shell scripts, if I find the time.
CMake build tested on
WAF build
./waf --with-examples --with-cpptests --std c++17
tested onUnit tests are passing on all platforms. I have tested the Python wheel only on Debian. The
essentia_python_tutorial.ipynb
notebook runs without problems. I've also been able to build the wheel for Windows, but haven't done much testing. The following code runs fine in the Python prompt:I'm compiling against shared library dependencies (FFmpeg etc.), so the location where those are residing has to be added with
os.add_dll_directory
on Windows.I've tried to find the affected issues and pull requests. No guarantee for completeness, though.
The text was updated successfully, but these errors were encountered: