-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[R-package] [gpu] Unable to properly pass boost directories to Rbuild on Windows 10 #5135
Comments
This does not work either:
And adding |
Thanks for opening this issue @aadler, and for all the details!
blegh sorry about that. I suspect that the argument parsing in that script is not handling Windows paths with I'll put up a pull request shortly to fix that.
This is very encouraging! I think then it the problem is limited to "[R-package] cannot build GPU version with MSVC and Rtools 4.x", right?
Thank you for trying the installation with just |
Thank you, @jameslamb, for the quick response. What I mean by "doesn't work" is that I pass a boost root directory "C:/local/boost_1_78_0" that is not registered by the script: "found Boost: C:/rtools42/x86_64-w64-mingw32.static.posix/lib/cmake/Boost-1.78.0/BoostConfig.cmake" Run with what I believe should be the proper invocation, I get this: D:\LightGBM\build>cmake -A x64 -DUSE_GPU=1 -DBOOST_ROOT=C:/local/boost_1_78_0 -DBOOST_LIBRARYDIR=C:/local/boost_1_78_0/lib64-msvc-14.3 ..
-- Building for: Visual Studio 17 2022
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.19043.
-- The C compiler identification is MSVC 19.31.31105.0
-- The CXX compiler identification is MSVC 19.31.31105.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.31.31103/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.31.31103/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found OpenMP_C: -openmp (found version "2.0")
-- Found OpenMP_CXX: -openmp (found version "2.0")
-- Found OpenMP: TRUE (found version "2.0")
-- Looking for CL_VERSION_2_2
-- Looking for CL_VERSION_2_2 - found
-- Found OpenCL: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.6/lib/x64/OpenCL.lib (found version "2.2")
-- OpenCL include directory: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.6/include
-- Boost toolset is unknown (compiler MSVC 19.31.31105.0)
-- Boost toolset is unknown (compiler MSVC 19.31.31105.0)
-- Boost toolset is unknown (compiler MSVC 19.31.31105.0)
-- Found Boost: C:/rtools42/x86_64-w64-mingw32.static.posix/lib/cmake/Boost-1.78.0/BoostConfig.cmake (found suitable version "1.78.0", minimum required is "1.56.0") found components: filesystem system
-- Performing Test MM_PREFETCH
-- Performing Test MM_PREFETCH - Failed
-- Performing Test MM_MALLOC
-- Performing Test MM_MALLOC - Failed
-- Configuring done
-- Generating done
CMake Warning:
Manually-specified variables were not used by the project:
BOOST_LIBRARYDIR
-- Build files have been written to: D:/LightGBM/build Note that it does not find the boost binaries which I passed. Now running the actual build will return a gazillion and two errors:
Will return a huge log file chock full of errors. The entire near 3K line logfile is here: LGBM.log Start (Lines 1–33):
End (Lines 2899–2910):
|
UpdateLooking at another issue, I found that despite the script not registering -DBOOST_HOME etc, if I changed the name of New ProblemThis leads to another issue; I think they are related but would be happy to split out if needed. Using the advice given in #5140, I checked out 3.3.2 and tried to build it. The CLI seems to complete, but I need the R package. Running Error MessageD:\LightGBM>Rscript build_r.R --use-gpu
* checking for file 'D:/LightGBM/lightgbm_r/DESCRIPTION' ... OK
* preparing 'lightgbm':
* checking DESCRIPTION meta-information ... OK
* cleaning src
* checking for LF line-endings in source and make files and shell scripts
* checking for empty or unneeded directories
WARNING: directory 'lightgbm/src/external_libs/compute/test' is empty
* looking to see if a 'data/datalist' file should be added
* building 'lightgbm_3.3.2.tar.gz'
Warning in utils::tar(filepath, pkgname, compression = compression, compression_level = 9L, :
storing paths of more than 100 bytes is not portable:
'lightgbm/src/external_libs/compute/include/boost/compute/algorithm/detail/find_extrema_with_atomics.hpp'
Warning in utils::tar(filepath, pkgname, compression = compression, compression_level = 9L, :
storing paths of more than 100 bytes is not portable:
'lightgbm/src/external_libs/compute/include/boost/compute/algorithm/detail/find_extrema_with_reduce.hpp'
Warning in utils::tar(filepath, pkgname, compression = compression, compression_level = 9L, :
storing paths of more than 100 bytes is not portable:
'lightgbm/src/external_libs/compute/include/boost/compute/algorithm/detail/reduce_by_key_with_scan.hpp'
Warning in utils::tar(filepath, pkgname, compression = compression, compression_level = 9L, :
storing paths of more than 100 bytes is not portable:
'lightgbm/src/external_libs/compute/include/boost/compute/iterator/detail/get_base_iterator_buffer.hpp'
* installing to library 'C:/R/RCurrent/R-4.2.0alpha/library'
* installing *source* package 'lightgbm' ...
** using staged installation
** libs
installing via 'install.libs.R' to C:/R/RCurrent/R-4.2.0alpha/library/00LOCK-lightgbm/00new/lightgbm
Trying 'Visual Studio 16 2019'
-- Configuring incomplete, errors occurred!
CMake Error at CMakeLists.txt:32 (PROJECT):
See also "D:/Rtemp/RtmpoZLwxV/R.INSTALL12dc7fb76921/lightgbm/src/build/CMakeFiles/CMakeOutput.log".
Generator
Visual Studio 16 2019
could not find any instance of Visual Studio.
Trying 'Visual Studio 15 2017'
CMake Error at CMakeLists.txt:32 (PROJECT):
-- Configuring incomplete, errors occurred!
Generator
Visual Studio 15 2017
could not find any instance of Visual Studio.
See also "D:/Rtemp/RtmpoZLwxV/R.INSTALL12dc7fb76921/lightgbm/src/build/CMakeFiles/CMakeOutput.log".
Trying 'Visual Studio 14 2015'
CMake Error at CMakeLists.txt:32 (PROJECT):
-- Configuring incomplete, errors occurred!
Failed to run MSBuild command:
MSBuild.exe
to get the value of VCTargetsPath:
The system cannot find the file specified
See also "D:/Rtemp/RtmpoZLwxV/R.INSTALL12dc7fb76921/lightgbm/src/build/CMakeFiles/CMakeOutput.log".
Warning in eval(ei, envir) :
Building with Visual Studio failed. Attempting with MSYS2
CMake Error: Error: generator : MSYS Makefiles
Does not match the generator used previously: Visual Studio 14 2015
Either remove the CMakeCache.txt file and CMakeFiles directory or choose a different binary directory.
CMake Error: Error: generator : MSYS Makefiles
Does not match the generator used previously: Visual Studio 14 2015
Either remove the CMakeCache.txt file and CMakeFiles directory or choose a different binary directory.
Error in .run_shell_command("cmake", c(cmake_args, "..")) :
Command failed with exit code: 1
* removing 'C:/R/RCurrent/R-4.2.0alpha/library/lightgbm'
* restoring previous 'C:/R/RCurrent/R-4.2.0alpha/library/lightgbm'
Error in .run_shell_command(install_cmd, install_args) :
Command failed with exit code: 1
Execution halted |
Visual Studio 17 2022 support was added in #4889. If you're building from source on LightGBM/R-package/src/install.libs.R Line 68 in b462d0a
|
Thanks. That worked for registering VS 17/2022, but we're back to the parsing issue. When nothing is passed other than :\LightGBM>Rscript build_r.R --use-gpu
* checking for file 'D:/LightGBM/lightgbm_r/DESCRIPTION' ... OK
* preparing 'lightgbm':
* checking DESCRIPTION meta-information ... OK
* cleaning src
* checking for LF line-endings in source and make files and shell scripts
* checking for empty or unneeded directories
WARNING: directory 'lightgbm/src/external_libs/compute/test' is empty
* looking to see if a 'data/datalist' file should be added
* building 'lightgbm_3.3.2.tar.gz'
Warning in utils::tar(filepath, pkgname, compression = compression, compression_level = 9L, :
storing paths of more than 100 bytes is not portable:
'lightgbm/src/external_libs/compute/include/boost/compute/algorithm/detail/find_extrema_with_atomics.hpp'
Warning in utils::tar(filepath, pkgname, compression = compression, compression_level = 9L, :
storing paths of more than 100 bytes is not portable:
'lightgbm/src/external_libs/compute/include/boost/compute/algorithm/detail/find_extrema_with_reduce.hpp'
Warning in utils::tar(filepath, pkgname, compression = compression, compression_level = 9L, :
storing paths of more than 100 bytes is not portable:
'lightgbm/src/external_libs/compute/include/boost/compute/algorithm/detail/reduce_by_key_with_scan.hpp'
Warning in utils::tar(filepath, pkgname, compression = compression, compression_level = 9L, :
storing paths of more than 100 bytes is not portable:
'lightgbm/src/external_libs/compute/include/boost/compute/iterator/detail/get_base_iterator_buffer.hpp'
* installing to library 'C:/R/RCurrent/R-4.2.0alpha/library'
* installing *source* package 'lightgbm' ...
** using staged installation
** libs
installing via 'install.libs.R' to C:/R/RCurrent/R-4.2.0alpha/library/00LOCK-lightgbm/00new/lightgbm
Trying 'Visual Studio 17 2022'
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 6.2.9200.
-- The C compiler identification is MSVC 19.31.31105.0
-- The CXX compiler identification is MSVC 19.31.31105.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.31.31103/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.31.31103/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- R version passed into FindLibR.cmake: 4.2.0
-- Creating R.lib and R.def
Creating 'D:/Rtemp/RtmpGqfbKQ/R.INSTALL51ccda8528b/lightgbm/src/build/R.def' from 'C:/R/RCurrent/R-4.2.0alpha/bin/x64/R.dll'
Successfully created 'D:/Rtemp/RtmpGqfbKQ/R.INSTALL51ccda8528b/lightgbm/src/build/R.def'
-- Found LibR: C:/R/RCurrent/R-4.2.0alpha
-- LIBR_EXECUTABLE: C:/R/RCurrent/R-4.2.0alpha/bin/x64/R.exe
-- LIBR_INCLUDE_DIRS: C:/R/RCurrent/R-4.2.0alpha/include
-- LIBR_CORE_LIBRARY: C:/R/RCurrent/R-4.2.0alpha/bin/x64/R.dll
-- Found OpenMP_C: -openmp (found version "2.0")
-- Found OpenMP_CXX: -openmp (found version "2.0")
-- Found OpenMP: TRUE (found version "2.0")
-- Looking for CL_VERSION_2_2
-- Looking for CL_VERSION_2_2 - found
-- Found OpenCL: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.6/lib/x64/OpenCL.lib (found version "2.2")
-- OpenCL include directory: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.6/include
-- Boost toolset is unknown (compiler MSVC 19.31.31105.0)
-- Boost toolset is unknown (compiler MSVC 19.31.31105.0)
-- Boost toolset is unknown (compiler MSVC 19.31.31105.0)
-- Found Boost: C:/rtools42/x86_64-w64-mingw32.static.posix/lib/cmake/Boost-1.78.0/BoostConfig.cmake (found suitable version "1.78.0", minimum required is "1.56.0") found components: filesystem system
-- Performing Test MM_PREFETCH
-- Performing Test MM_PREFETCH - Failed
-- Performing Test MM_MALLOC
-- Performing Test MM_MALLOC - Failed
-- Configuring done
-- Generating done
-- Build files have been written to: D:/Rtemp/RtmpGqfbKQ/R.INSTALL51ccda8528b/lightgbm/src/build This then leads to a failed build (excerpt below): Successfully created build files for 'Visual Studio 17 2022'
Building lib_lightgbm
Microsoft (R) Build Engine version 17.1.0+ae57d105c for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.
Checking Build System
Building Custom Rule D:/Rtemp/RtmpGqfbKQ/R.INSTALL51ccda8528b/lightgbm/src/CMakeLists.txt
boosting.cpp
gbdt.cpp
gbdt_model_text.cpp
gbdt_prediction.cpp
prediction_early_stop.cpp
bin.cpp
config.cpp
config_auto.cpp
dataset.cpp
dataset_loader.cpp
file_io.cpp
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(61,26): warning C4083: expected ')'; found identifier '_VCRUNTIME_DISABLED_WARNINGS' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\boosting.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
json11.cpp
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(111,1): warning C4005: '_CRT_STRINGIZE': macro redefinition (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\boosting.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\_mingw.h(287): message : see previous definition of '_CRT_STRINGIZE' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\boosting.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(114,1): warning C4005: '_CRT_WIDE': macro redefinition (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\boosting.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\_mingw.h(292): message : see previous definition of '_CRT_WIDE' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\boosting.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(155,1): warning C4005: '__CRTDECL': macro redefinition (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\boosting.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\_mingw.h(399): message : see previous definition of '__CRTDECL' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\boosting.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(325,1): warning C4005: '_CRT_DEPRECATE_TEXT': macro redefinition (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\boosting.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\_mingw.h(350): message : see previous definition of '_CRT_DEPRECATE_TEXT' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\boosting.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(366,51): error C2065: 'uintptr_t': undeclared identifier (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\boosting.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(366,61): error C2146: syntax error: missing ')' before identifier '_StackCookie' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\boosting.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(367,67): error C2065: 'uintptr_t': undeclared identifier (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\boosting.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(367,77): error C2146: syntax error: missing ')' before identifier '_StackCookie' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\boosting.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(61,26): warning C4083: expected ')'; found identifier '_VCRUNTIME_DISABLED_WARNINGS' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\gbdt.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(371,35): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\boosting.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(111,1): warning C4005: '_CRT_STRINGIZE': macro redefinition (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\gbdt.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\_mingw.h(287): message : see previous definition of '_CRT_STRINGIZE' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\gbdt.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(371,18): error C2146: syntax error: missing ';' before identifier '__security_cookie' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\boosting.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(61,26): warning C4083: expected ')'; found identifier '_VCRUNTIME_DISABLED_WARNINGS' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\gbdt_model_text.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(61,26): warning C4083: expected ')'; found identifier '_VCRUNTIME_DISABLED_WARNINGS' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\gbdt_prediction.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(61,26): warning C4083: expected ')'; found identifier '_VCRUNTIME_DISABLED_WARNINGS' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\prediction_early_stop.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(61,26): warning C4083: expected ')'; found identifier '_VCRUNTIME_DISABLED_WARNINGS' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\io\bin.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(114,1): warning C4005: '_CRT_WIDE': macro redefinition (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\gbdt.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\_mingw.h(292): message : see previous definition of '_CRT_WIDE' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\gbdt.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(111,1): warning C4005: '_CRT_STRINGIZE': macro redefinition (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\gbdt_model_text.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\_mingw.h(287): message : see previous definition of '_CRT_STRINGIZE' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\gbdt_model_text.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(111,1): warning C4005: '_CRT_STRINGIZE': macro redefinition (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\gbdt_prediction.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\_mingw.h(287): message : see previous definition of '_CRT_STRINGIZE' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\gbdt_prediction.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(111,1): warning C4005: '_CRT_STRINGIZE': macro redefinition (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\prediction_early_stop.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\_mingw.h(287): message : see previous definition of '_CRT_STRINGIZE' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\prediction_early_stop.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(111,1): warning C4005: '_CRT_STRINGIZE': macro redefinition (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\io\bin.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\_mingw.h(287): message : see previous definition of '_CRT_STRINGIZE' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\io\bin.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj] It goes on and on. When I try passing the boost directory, I get the original issue: :\LightGBM>Rscript build_r.R --use-gpu --boost-librarydir=C:\local\boost_1_78_0\lib
* checking for file 'D:/LightGBM/lightgbm_r/DESCRIPTION' ... OK
* preparing 'lightgbm':
* checking DESCRIPTION meta-information ... OK
* cleaning src
* checking for LF line-endings in source and make files and shell scripts
* checking for empty or unneeded directories
WARNING: directory 'lightgbm/src/external_libs/compute/test' is empty
* looking to see if a 'data/datalist' file should be added
* building 'lightgbm_3.3.2.tar.gz'
Warning in utils::tar(filepath, pkgname, compression = compression, compression_level = 9L, :
storing paths of more than 100 bytes is not portable:
'lightgbm/src/external_libs/compute/include/boost/compute/algorithm/detail/find_extrema_with_atomics.hpp'
Warning in utils::tar(filepath, pkgname, compression = compression, compression_level = 9L, :
storing paths of more than 100 bytes is not portable:
'lightgbm/src/external_libs/compute/include/boost/compute/algorithm/detail/find_extrema_with_reduce.hpp'
Warning in utils::tar(filepath, pkgname, compression = compression, compression_level = 9L, :
storing paths of more than 100 bytes is not portable:
'lightgbm/src/external_libs/compute/include/boost/compute/algorithm/detail/reduce_by_key_with_scan.hpp'
Warning in utils::tar(filepath, pkgname, compression = compression, compression_level = 9L, :
storing paths of more than 100 bytes is not portable:
'lightgbm/src/external_libs/compute/include/boost/compute/iterator/detail/get_base_iterator_buffer.hpp'
* installing to library 'C:/R/RCurrent/R-4.2.0alpha/library'
* installing *source* package 'lightgbm' ...
** using staged installation
** libs
installing via 'install.libs.R' to C:/R/RCurrent/R-4.2.0alpha/library/00LOCK-lightgbm/00new/lightgbm
Error in source("install.libs.R", local = local.env) :
install.libs.R:141:45: unexpected symbol
140: # NOTE: build_r.R replaces the line below
141: command_line_args <- c("-DBOOST_LIBRARYDIR="C
^
* removing 'C:/R/RCurrent/R-4.2.0alpha/library/lightgbm'
* restoring previous 'C:/R/RCurrent/R-4.2.0alpha/library/lightgbm'
Error in .run_shell_command(install_cmd, install_args) :
Command failed with exit code: 1
Execution halted Or Error in source("install.libs.R", local = local.env) :
install.libs.R:141:70: unexpected string constant
140: # NOTE: build_r.R replaces the line below
141: command_line_args <- c("-DBOOST_LIBRARYDIR="/c/local/boost_1_78_0/lib""
^ Or Error in source("install.libs.R", local = local.env) :
install.libs.R:141:45: unexpected symbol
140: # NOTE: build_r.R replaces the line below
141: command_line_args <- c("-DBOOST_LIBRARYDIR="c
^
* removing 'C:/R/RCurrent/R-4.2.0alpha/library/lightgbm'
* restoring previous 'C:/R/RCurrent/R-4.2.0alpha/library/lightgbm' etc. I appreciate your patience, @jameslamb. At this point, I'll stick with building the gpu-enabled version 3.3.2 from source using MSYS2, which works and can use the GPU (althogh for my problem, right now, it seems the CPU is faster :D ). Thank you!! |
Hi, @jameslamb. Just confirming this is still an issue with 3.3.2.99 version 9489f87. Thanks |
Thanks for the nudge and sorry for the delay @aadler ! I also really appreciate "hey even as of the most recent commit this is still a bug" more than those "any updates?" types of comments people leave sometimes 😍 It's hard for me to test this (and possible fixes) because I don't have easy access to a Windows environment, but will try to get to it soon if no other maintainers do. |
Hi @jameslamb. New version so I thought I'd check the BOOST issue. Still not working unless I am doing something wrong. Note where I say the libraries should be and note where CMake finds them :( Thank you. D:\LightGBM\build>cmake -A x64 -DUSE_GPU=1 -DBOOST_ROOT=D:/local/boost_1_80_0 -DBOOST_LIBRARYDIR=D:/local/boost_1_80_0/lib64-msvc-14.3 ..
-- Building for: Visual Studio 17 2022
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.19043.
-- The C compiler identification is MSVC 19.33.31629.0
-- The CXX compiler identification is MSVC 19.33.31629.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.33.31629/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.33.31629/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found OpenMP_C: -openmp (found version "2.0")
-- Found OpenMP_CXX: -openmp (found version "2.0")
-- Found OpenMP: TRUE (found version "2.0")
-- Looking for CL_VERSION_3_0
-- Looking for CL_VERSION_3_0 - found
-- Found OpenCL: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.7/lib/x64/OpenCL.lib (found version "3.0")
-- OpenCL include directory: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.7/include
-- Boost toolset is unknown (compiler MSVC 19.33.31629.0)
-- Boost toolset is unknown (compiler MSVC 19.33.31629.0)
-- Boost toolset is unknown (compiler MSVC 19.33.31629.0)
-- Found Boost: C:/rtools42/x86_64-w64-mingw32.static.posix/lib/cmake/Boost-1.78.0/BoostConfig.cmake (found suitable version "1.78.0", minimum required is "1.56.0") found components: filesystem system
-- Performing Test MM_PREFETCH
-- Performing Test MM_PREFETCH - Failed
-- Performing Test MM_MALLOC
-- Performing Test MM_MALLOC - Failed
-- Looking for inet_pton
-- Looking for inet_pton - found
-- Configuring done
-- Generating done
CMake Warning:
Manually-specified variables were not used by the project:
BOOST_LIBRARYDIR
-- Build files have been written to: D:/LightGBM/build |
Hey @aadler could you try pulling the branch from #5607? It doesn't fix the "LightGBM uses RTools Boost instead of the one you asked for" issue yet, but I think it at least fixes the parsing of paths so the errors like You could pull it like this git remote add upstream https://github.com/microsoft/LightGBM.git
git fetch upstream fix/r-gpu
git checkout fix/r-gpu
git submodule update --recursive |
Hello. @jameslamb I get the following error: D:\LightGBM>git checkout fix/r-gpu
hint: If you meant to check out a remote tracking branch on, e.g. 'origin',
hint: you can do so by fully qualifying the name with the --track option:
hint:
hint: git checkout --track origin/<name>
hint:
hint: If you'd like to always have checkouts of an ambiguous <name> prefer
hint: one remote, e.g. the 'origin' remote, consider setting
hint: checkout.defaultRemote=origin in your config.
fatal: 'fix/r-gpu' matched multiple (2) remote tracking branches |
Ok try git checkout --track upstream/fix/r-gpu |
True. None of those errors, but it is still finding the wrong Boost. D:\LightGBM>Rscript build_r.R --use-gpu --boost-root=D:/local/boost_1_80_0/
* checking for file 'D:/LightGBM/lightgbm_r/DESCRIPTION' ... OK
* preparing 'lightgbm':
* checking DESCRIPTION meta-information ... OK
* cleaning src
* installing the package to build vignettes
-----------------------------------
* installing *source* package 'lightgbm' ...
** using staged installation
** libs
installing via 'install.libs.R' to D:/Rtemp/RtmpcFC2dN/Rinst765423f358f8/00LOCK-lightgbm/00new/lightgbm
Trying 'Visual Studio 17 2022'
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 6.2.9200.
-- The C compiler identification is MSVC 19.33.31629.0
-- The CXX compiler identification is MSVC 19.33.31629.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.33.31629/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.33.31629/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- R version passed into FindLibR.cmake: 4.2.2
-- Creating R.lib and R.def
Creating 'D:/Rtemp/RtmpcFC2dN/Rbuild765417941dd1/lightgbm/src/build/R.def' from 'C:/R/RCurrent/R-4.2.2patched/bin/x64/R.dll'
Successfully created 'D:/Rtemp/RtmpcFC2dN/Rbuild765417941dd1/lightgbm/src/build/R.def'
-- Found LibR: C:/R/RCurrent/R-4.2.2patched
-- LIBR_EXECUTABLE: C:/R/RCurrent/R-4.2.2patched/bin/x64/R.exe
-- LIBR_INCLUDE_DIRS: C:/R/RCurrent/R-4.2.2patched/include
-- LIBR_CORE_LIBRARY: C:/R/RCurrent/R-4.2.2patched/bin/x64/R.dll
-- Found OpenMP_C: -openmp (found version "2.0")
-- Found OpenMP_CXX: -openmp (found version "2.0")
-- Found OpenMP: TRUE (found version "2.0")
-- Looking for CL_VERSION_3_0
-- Looking for CL_VERSION_3_0 - found
-- Found OpenCL: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.7/lib/x64/OpenCL.lib (found version "3.0")
-- OpenCL include directory: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.7/include
-- Found Boost: C:/rtools42/x86_64-w64-mingw32.static.posix/lib/cmake/Boost-1.80.0/BoostConfig.cmake (found suitable version "1.80.0", minimum required is "1.56.0") found components: filesystem system
-- Performing Test MM_PREFETCH
-- Performing Test MM_PREFETCH - Failed
-- Performing Test MM_MALLOC
-- Performing Test MM_MALLOC - Failed
-- Looking for inet_pton
-- Looking for inet_pton - found
-- Configuring done
-- Generating done
-- Build files have been written to: D:/Rtemp/RtmpcFC2dN/Rbuild765417941dd1/lightgbm/src/build
Successfully created build files for 'Visual Studio 17 2022'
Building lib_lightgbm
MSBuild version 17.3.1+2badb37d1 for .NET Framework
Checking Build System
Building Custom Rule D:/Rtemp/RtmpcFC2dN/Rbuild765417941dd1/lightgbm/src/CMakeLists.txt
c_api.cpp
lightgbm_R.cpp
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.33.31629\include\vcruntime.h(61,26): warning C4083: expected ')'; found identifier '_VCRUNTIME_DISABLED_WARNINGS' (compiling source file D:\Rtemp\RtmpcFC2dN\Rbuild765417941dd1\lightgbm\src\src\c_api.cpp) [D:\Rtemp\RtmpcFC2dN\Rbuild765417941dd1\lightgbm\src\build\lightgbm_capi_objs.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.33.31629\include\vcruntime.h(111,1): warning C4005: '_CRT_STRINGIZE': macro redefinition (compiling source file D:\Rtemp\RtmpcFC2dN\Rbuild765417941dd1\lightgbm\src\src\c_api.cpp) [D:\Rtemp\RtmpcFC2dN\Rbuild765417941dd1\lightgbm\src\build\lightgbm_capi_objs.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\_mingw.h(287): message : see previous definition of '_CRT_STRINGIZE' (compiling source file D:\Rtemp\RtmpcFC2dN\Rbuild765417941dd1\lightgbm\src\src\c_api.cpp) [D:\Rtemp\RtmpcFC2dN\Rbuild765417941dd1\lightgbm\src\build\lightgbm_capi_objs.vcxproj] and a zillion errors |
Nice, thanks for checking that! Totally understood. Like I mentioned above, I didn't expect #5607 to fix the "wrong Boost" issue yet. Just trying to fix one issue at a time 😅 |
You may find the last line here of interest: D:\LightGBM>Rscript build_r.R --use-gpu --boost-librarydir=D:/local/boost_1_80_0/lib64-msvc-14.3/
* checking for file 'D:/LightGBM/lightgbm_r/DESCRIPTION' ... OK
* preparing 'lightgbm':
* checking DESCRIPTION meta-information ... OK
* cleaning src
* installing the package to build vignettes
-----------------------------------
* installing *source* package 'lightgbm' ...
** using staged installation
** libs
installing via 'install.libs.R' to D:/Rtemp/RtmpodIZGs/Rinst8a386da85e09/00LOCK-lightgbm/00new/lightgbm
Trying 'Visual Studio 17 2022'
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 6.2.9200.
-- The C compiler identification is MSVC 19.33.31629.0
-- The CXX compiler identification is MSVC 19.33.31629.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.33.31629/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.33.31629/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- R version passed into FindLibR.cmake: 4.2.2
-- Creating R.lib and R.def
Creating 'D:/Rtemp/RtmpodIZGs/Rbuild8a383a3b7b7c/lightgbm/src/build/R.def' from 'C:/R/RCurrent/R-4.2.2patched/bin/x64/R.dll'
Successfully created 'D:/Rtemp/RtmpodIZGs/Rbuild8a383a3b7b7c/lightgbm/src/build/R.def'
-- Found LibR: C:/R/RCurrent/R-4.2.2patched
-- LIBR_EXECUTABLE: C:/R/RCurrent/R-4.2.2patched/bin/x64/R.exe
-- LIBR_INCLUDE_DIRS: C:/R/RCurrent/R-4.2.2patched/include
-- LIBR_CORE_LIBRARY: C:/R/RCurrent/R-4.2.2patched/bin/x64/R.dll
-- Found OpenMP_C: -openmp (found version "2.0")
-- Found OpenMP_CXX: -openmp (found version "2.0")
-- Found OpenMP: TRUE (found version "2.0")
-- Looking for CL_VERSION_3_0
-- Looking for CL_VERSION_3_0 - found
-- Found OpenCL: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.7/lib/x64/OpenCL.lib (found version "3.0")
-- OpenCL include directory: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.7/include
-- Found Boost: C:/rtools42/x86_64-w64-mingw32.static.posix/lib/cmake/Boost-1.80.0/BoostConfig.cmake (found suitable version "1.80.0", minimum required is "1.56.0") found components: filesystem system
-- Performing Test MM_PREFETCH
-- Performing Test MM_PREFETCH - Failed
-- Performing Test MM_MALLOC
-- Performing Test MM_MALLOC - Failed
-- Looking for inet_pton
-- Looking for inet_pton - found
-- Configuring done
-- Generating done
CMake Warning:
Manually-specified variables were not used by the project:
BOOST_LIBRARYDIR |
Hi. Is this issue solved? I'm having the same problem.
|
Not to my knowledge. |
@ruddnr has posted another reproducible example showing this issue on v4.3.0: #6289 I have not personally had time to investigate this further, and might not for a while. We're still struggling with a significant lack of maintainer availability in this project, and there are some other issues occupying my time. I'm happy to answer questions and help with reviews, but can't devote time to testing a fix for this at the moment. We'd greatly appreciate if someone watching this thread would volunteer to try to fix this in a way that works for both Windows and Unix-likes (#5960). As a starting point... the fix probably requires changes to some subset of these 3 files: |
Description
I am trying to build LightGBM for R on Windows 10 with GPU support. I have the Boost libraries for MS VS 2022 installed under
C:\local\boost_1_78_0
as recommended, but Rtools42 comes with its own version of Boost which the Cmake finds first. When I pass--msys2
, the build works properly. However, when I do not pass it, It fails with a multitude of errors such as:I believe this is due to the build expecting VS as the compiler but finding the wrong boost libraries as per
Reproducible example
I have tried various invocations to pass the proper locations and all fail
What am I doing wrong?
Environment info
Windows 10 Home 64
Intel i7-8700K
NVIDIA RTX 3080
NVIDIA CUDA 11.6
CMake 3.23
VS 2022
Boost 1.7.8 for lib64-msvc-14.3
R 4.2.0 alpha
Rtools42
The text was updated successfully, but these errors were encountered: