Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(bazel): support building C++ libs on windows platform #1873

Merged
merged 10 commits into from
Oct 9, 2024

Conversation

An-DJ
Copy link
Contributor

@An-DJ An-DJ commented Oct 7, 2024

What does this PR do?

Make bazel happy on Windows OS.

  1. Rewrite a little codes in row.cc, then it could be compiled by MSVC (VLA is not supported)
  2. Add MSVC flag /Zc:preprocessor to enable C99/C11
  3. Add MSVC flag /utf-8 to set source and execution character sets to UTF-8.
  4. Load arrow(arrow_python) interface libraries *.lib to the linker
  5. Add a new CI env windows-2022 to build fury cpp libraries

Related issues

#798

Does this PR introduce any user-facing change?

  • Does this PR introduce any public API change?
  • Does this PR introduce any binary protocol compatibility change?

Benchmark

@An-DJ An-DJ changed the title feat(bazel): support building on windows platform feat(bazel): support C++ libs building on windows platform Oct 7, 2024
@An-DJ An-DJ changed the title feat(bazel): support C++ libs building on windows platform feat(bazel): support building C++ libs on windows platform Oct 7, 2024
@An-DJ An-DJ marked this pull request as draft October 7, 2024 12:59
@An-DJ
Copy link
Contributor Author

An-DJ commented Oct 7, 2024

Have tried my best already : ( but there are still strange compiling errors using MSVC toolchain. It looks like some micro(s) cannot be compiled by MSVC correctly, right?

2024-10-07T11:48:59.2650949Z 2024-10-07 11:48:59,246 - INFO - running command: bazel test //:_format //:_serialization //:_util //:cp_fury_so //:mmh3 //:python/pyfury/_serialization.pyx_cython_translation //:python/pyfury/_serialization.so //:python/pyfury/_util.pyx_cython_translation //:python/pyfury/_util.so //:python/pyfury/format/_format.pyx_cython_translation //:python/pyfury/format/_format.so //:python/pyfury/lib/mmh3/__init__.py_cython_translation //:python/pyfury/lib/mmh3/__init__.so //:python/pyfury/lib/mmh3/mmh3.pyx_cython_translation //:python/pyfury/lib/mmh3/mmh3.so //cpp/fury:fury //cpp/fury/columnar:arrow_writer_test //cpp/fury/columnar:convert_test //cpp/fury/columnar:fury_columnar_format //cpp/fury/encoder:fury_encoder //cpp/fury/encoder:row_encoder_test //cpp/fury/meta:field_info_test //cpp/fury/meta:fury_meta //cpp/fury/meta:preprocessor_test //cpp/fury/meta:type_traits_test //cpp/fury/row:fury_row_format //cpp/fury/row:row_test //cpp/fury/thirdparty:libmmh3 //cpp/fury/util:buffer_test //cpp/fury/util:fury_util //cpp/fury/util:logging_test //cpp/fury/util:status_test //cpp/fury/util:string_util_test //cpp/fury/util:time_util_test 
2024-10-07T11:48:59.4852555Z Loading: 
2024-10-07T11:48:59.5573023Z Loading: 
2024-10-07T11:48:59.5614037Z Loading: 0 packages loaded
2024-10-07T11:48:59.6769211Z Analyzing: 12 targets (0 packages loaded, 0 targets configured)
2024-10-07T11:49:01.4841282Z Analyzing: 12 targets (29 packages loaded, 6 targets configured)
2024-10-07T11:49:06.3579821Z Analyzing: 12 targets (29 packages loaded, 6 targets configured)
2024-10-07T11:49:07.4906093Z Analyzing: 12 targets (33 packages loaded, 131 targets configured)
2024-10-07T11:49:08.4914727Z Analyzing: 12 targets (34 packages loaded, 131 targets configured)
2024-10-07T11:49:18.7559121Z Analyzing: 12 targets (34 packages loaded, 131 targets configured)
2024-10-07T11:49:20.4990366Z Analyzing: 12 targets (38 packages loaded, 353 targets configured)
2024-10-07T11:49:21.4995926Z Analyzing: 12 targets (38 packages loaded, 353 targets configured)
2024-10-07T11:49:22.5529236Z Analyzing: 12 targets (47 packages loaded, 566 targets configured)
2024-10-07T11:49:22.5903226Z INFO: Analyzed 12 targets (47 packages loaded, 570 targets configured).
2024-10-07T11:49:22.5988159Z INFO: Found 12 test targets...
2024-10-07T11:49:22.7919799Z [0 / 45] [Prepa] BazelWorkspaceStatusAction stable-status.txt
2024-10-07T11:49:23.7624906Z [29 / 85] Compiling googletest/src/gtest.cc; 0s local ... (5 actions, 4 running)
2024-10-07T11:49:25.5557383Z [30 / 85] Compiling googletest/src/gtest.cc; 2s local ... (5 actions, 4 running)
2024-10-07T11:49:27.6007418Z [30 / 85] Compiling googletest/src/gtest.cc; 4s local ... (5 actions running)
2024-10-07T11:49:27.8156109Z ERROR: D:/a/fury/fury/cpp/fury/meta/BUILD:11:8: Compiling cpp/fury/meta/preprocessor_test.cc failed: (Exit 2): cl.exe failed: error executing command (from target //cpp/fury/meta:preprocessor_test) C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.41.34120\bin\HostX64\x64\cl.exe /nologo /DCOMPILER_MSVC /DNOMINMAX /D_WIN32_WINNT=0x0601 /D_CRT_SECURE_NO_DEPRECATE ... (remaining 39 arguments skipped)
2024-10-07T11:49:27.8912021Z INFO: Elapsed time: 28.604s, Critical Path: 5.08s
2024-10-07T11:49:27.8917492Z INFO: 35 processes: 35 internal.
2024-10-07T11:49:27.8920784Z FAILED: Build did NOT complete successfully
2024-10-07T11:49:27.9328969Z 2024-10-07 11:49:27,917 - ERROR - cpp/fury/meta/preprocessor_test.cc(27): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
2024-10-07T11:49:27.9332429Z cpp/fury/meta/preprocessor_test.cc(27): error C2059: syntax error: '=='
2024-10-07T11:49:27.9336551Z cpp/fury/meta/preprocessor_test.cc(28): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
2024-10-07T11:49:27.9338650Z cpp/fury/meta/preprocessor_test.cc(28): error C2059: syntax error: '=='
2024-10-07T11:49:27.9340991Z cpp/fury/meta/preprocessor_test.cc(29): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
2024-10-07T11:49:27.9342757Z cpp/fury/meta/preprocessor_test.cc(29): error C2059: syntax error: '=='
2024-10-07T11:49:27.9344530Z cpp/fury/meta/preprocessor_test.cc(30): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
2024-10-07T11:49:27.9346163Z cpp/fury/meta/preprocessor_test.cc(30): error C2059: syntax error: '=='
2024-10-07T11:49:27.9348190Z cpp/fury/meta/preprocessor_test.cc(31): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
2024-10-07T11:49:27.9349910Z cpp/fury/meta/preprocessor_test.cc(31): error C2059: syntax error: '=='
2024-10-07T11:49:27.9351634Z cpp/fury/meta/preprocessor_test.cc(32): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
2024-10-07T11:49:27.9354637Z cpp/fury/meta/preprocessor_test.cc(32): error C2059: syntax error: '=='
2024-10-07T11:49:27.9356158Z cpp/fury/meta/preprocessor_test.cc(36): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
2024-10-07T11:49:27.9357812Z cpp/fury/meta/preprocessor_test.cc(36): error C2059: syntax error: '=='
2024-10-07T11:49:27.9359260Z cpp/fury/meta/preprocessor_test.cc(40): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
2024-10-07T11:49:27.9360650Z cpp/fury/meta/preprocessor_test.cc(40): error C2059: syntax error: '=='
2024-10-07T11:49:27.9362106Z cpp/fury/meta/preprocessor_test.cc(47): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
2024-10-07T11:49:27.9364017Z cpp/fury/meta/preprocessor_test.cc(48): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
2024-10-07T11:49:27.9365891Z cpp/fury/meta/preprocessor_test.cc(49): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
2024-10-07T11:49:27.9367738Z cpp/fury/meta/preprocessor_test.cc(50): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
2024-10-07T11:49:27.9369598Z cpp/fury/meta/preprocessor_test.cc(51): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
2024-10-07T11:49:27.9371125Z cpp/fury/meta/preprocessor_test.cc(47): error C2065: 'PLUS': undeclared identifier
2024-10-07T11:49:27.9372448Z cpp/fury/meta/preprocessor_test.cc(47): error C3861: 'FURY_PP_FOREACH_IMPL_': identifier not found
2024-10-07T11:49:27.9373747Z cpp/fury/meta/preprocessor_test.cc(48): error C2065: 'PLUS': undeclared identifier
2024-10-07T11:49:27.9375035Z cpp/fury/meta/preprocessor_test.cc(48): error C3861: 'FURY_PP_FOREACH_IMPL_': identifier not found
2024-10-07T11:49:27.9376389Z cpp/fury/meta/preprocessor_test.cc(49): error C2065: 'PLUS': undeclared identifier
2024-10-07T11:49:27.9377701Z cpp/fury/meta/preprocessor_test.cc(49): error C3861: 'FURY_PP_FOREACH_IMPL_': identifier not found
2024-10-07T11:49:27.9379021Z cpp/fury/meta/preprocessor_test.cc(50): error C2065: 'PLUS': undeclared identifier
2024-10-07T11:49:27.9380310Z cpp/fury/meta/preprocessor_test.cc(50): error C3861: 'FURY_PP_FOREACH_IMPL_': identifier not found
2024-10-07T11:49:27.9381579Z cpp/fury/meta/preprocessor_test.cc(51): error C2065: 'PLUS': undeclared identifier
2024-10-07T11:49:27.9382867Z cpp/fury/meta/preprocessor_test.cc(51): error C3861: 'FURY_PP_FOREACH_IMPL_': identifier not found
2024-10-07T11:49:27.9384162Z cpp/fury/meta/preprocessor_test.cc(53): error C2131: expression did not evaluate to a constant
2024-10-07T11:49:27.9385561Z cpp/fury/meta/preprocessor_test.cc(47): note: a non-constant (sub-)expression was encountered
2024-10-07T11:49:27.9386877Z cpp/fury/meta/preprocessor_test.cc(53): note: the call stack of the evaluation (the oldest call first) is
2024-10-07T11:49:27.9389053Z cpp/fury/meta/preprocessor_test.cc(53): note: while evaluating function 'int fury::Preprocessor_ForEach_Test::TestBody::<lambda_3e1d113607ce18e4f82d08cb1698e1cf>::operator ()(void) const'
2024-10-07T11:49:27.9391249Z cpp/fury/meta/preprocessor_test.cc(56): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
2024-10-07T11:49:27.9392809Z cpp/fury/meta/preprocessor_test.cc(56): error C2065: 'STR': undeclared identifier
2024-10-07T11:49:27.9393879Z cpp/fury/meta/preprocessor_test.cc(56): error C2065: 'a': undeclared identifier
2024-10-07T11:49:27.9395104Z cpp/fury/meta/preprocessor_test.cc(56): error C2065: 'bc': undeclared identifier
2024-10-07T11:49:27.9396389Z cpp/fury/meta/preprocessor_test.cc(56): error C2065: 'def': undeclared identifier
2024-10-07T11:49:27.9397748Z cpp/fury/meta/preprocessor_test.cc(56): error C3861: 'FURY_PP_FOREACH_IMPL_': identifier not found
2024-10-07T11:49:27.9399299Z cpp/fury/meta/preprocessor_test.cc(56): error C2131: expression did not evaluate to a constant
2024-10-07T11:49:27.9400725Z cpp/fury/meta/preprocessor_test.cc(56): note: a non-constant (sub-)expression was encountered
2024-10-07T11:49:27.9402037Z cpp/fury/meta/preprocessor_test.cc(57): error C2131: expression did not evaluate to a constant
2024-10-07T11:49:27.9403691Z cpp/fury/meta/preprocessor_test.cc(57): note: failure was caused by a read of an uninitialized symbol
2024-10-07T11:49:27.9405158Z cpp/fury/meta/preprocessor_test.cc(57): note: see usage of 'strings'
2024-10-07T11:49:27.9406404Z cpp/fury/meta/preprocessor_test.cc(58): error C2131: expression did not evaluate to a constant
2024-10-07T11:49:27.9408007Z cpp/fury/meta/preprocessor_test.cc(58): note: failure was caused by out of range index 1; allowed range is 0 <= index < 0
2024-10-07T11:49:27.9409760Z cpp/fury/meta/preprocessor_test.cc(59): error C2131: expression did not evaluate to a constant
2024-10-07T11:49:27.9411215Z cpp/fury/meta/preprocessor_test.cc(59): note: failure was caused by out of range index 2; allowed range is 0 <= index < 0
2024-10-07T11:49:27.9412560Z //cpp/fury/columnar:arrow_writer_test                                 NO STATUS
2024-10-07T11:49:27.9413629Z //cpp/fury/columnar:convert_test                                      NO STATUS
2024-10-07T11:49:27.9414566Z //cpp/fury/encoder:row_encoder_test                                   NO STATUS
2024-10-07T11:49:27.9415539Z //cpp/fury/meta:field_info_test                                       NO STATUS
2024-10-07T11:49:27.9416535Z //cpp/fury/meta:type_traits_test                                      NO STATUS
2024-10-07T11:49:27.9417532Z //cpp/fury/row:row_test                                               NO STATUS
2024-10-07T11:49:27.9418524Z //cpp/fury/util:buffer_test                                           NO STATUS
2024-10-07T11:49:27.9419439Z //cpp/fury/util:logging_test                                          NO STATUS
2024-10-07T11:49:27.9420301Z //cpp/fury/util:status_test                                           NO STATUS
2024-10-07T11:49:27.9421130Z //cpp/fury/util:string_util_test                                      NO STATUS
2024-10-07T11:49:27.9422076Z //cpp/fury/util:time_util_test                                        NO STATUS
2024-10-07T11:49:27.9423066Z //cpp/fury/meta:preprocessor_test                               FAILED TO BUILD
2024-10-07T11:49:27.9423684Z 
2024-10-07T11:49:27.9424119Z Executed 0 out of 12 tests: 1 fails to build and 11 were skipped.
2024-10-07T11:49:27.9424650Z 
2024-10-07T11:49:27.9424822Z Traceback (most recent call last):
2024-10-07T11:49:27.9433310Z   File "D:\a\fury\fury\ci\run_ci.py", line 187, in <module>
2024-10-07T11:49:27.9434055Z     _parse_args()
2024-10-07T11:49:27.9434658Z   File "D:\a\fury\fury\ci\run_ci.py", line 183, in _parse_args
2024-10-07T11:49:27.9435344Z     args.func(**arg_dict)
2024-10-07T11:49:27.9435948Z   File "D:\a\fury\fury\ci\run_ci.py", line 85, in _run_cpp
2024-10-07T11:49:27.9436611Z     _exec_cmd(
2024-10-07T11:49:27.9437176Z   File "D:\a\fury\fury\ci\run_ci.py", line 43, in _exec_cmd
2024-10-07T11:49:27.9438533Z     result = subprocess.check_output(cmd, shell=True, universal_newlines=True)
2024-10-07T11:49:27.9439424Z              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-07T11:49:27.9440808Z   File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\subprocess.py", line 466, in check_output
2024-10-07T11:49:27.9442286Z     return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
2024-10-07T11:49:27.9451061Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-07T11:49:27.9452391Z   File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\subprocess.py", line 571, in run
2024-10-07T11:49:27.9453725Z     raise CalledProcessError(retcode, process.args,
2024-10-07T11:49:27.9464501Z subprocess.CalledProcessError: Command 'bazel test //:_format //:_serialization //:_util //:cp_fury_so //:mmh3 //:python/pyfury/_serialization.pyx_cython_translation //:python/pyfury/_serialization.so //:python/pyfury/_util.pyx_cython_translation //:python/pyfury/_util.so //:python/pyfury/format/_format.pyx_cython_translation //:python/pyfury/format/_format.so //:python/pyfury/lib/mmh3/__init__.py_cython_translation //:python/pyfury/lib/mmh3/__init__.so //:python/pyfury/lib/mmh3/mmh3.pyx_cython_translation //:python/pyfury/lib/mmh3/mmh3.so //cpp/fury:fury //cpp/fury/columnar:arrow_writer_test //cpp/fury/columnar:convert_test //cpp/fury/columnar:fury_columnar_format //cpp/fury/encoder:fury_encoder //cpp/fury/encoder:row_encoder_test //cpp/fury/meta:field_info_test //cpp/fury/meta:fury_meta //cpp/fury/meta:preprocessor_test //cpp/fury/meta:type_traits_test //cpp/fury/row:fury_row_format //cpp/fury/row:row_test //cpp/fury/thirdparty:libmmh3 //cpp/fury/util:buffer_test //cpp/fury/util:fury_util //cpp/fury/util:logging_test //cpp/fury/util:status_test //cpp/fury/util:string_util_test //cpp/fury/util:time_util_test ' returned non-zero exit status 1.
2024-10-07T11:49:28.0952778Z ##[error]Process completed with exit code 1.
2024-10-07T11:49:28.1122428Z Post job cleanup.
2024-10-07T11:49:28.3256650Z [command]"C:\Program Files\Git\bin\git.exe" version
2024-10-07T11:49:28.3469718Z git version 2.46.2.windows.1
2024-10-07T11:49:28.3534153Z Temporarily overriding HOME='D:\a\_temp\b85d7ab1-dde8-4df7-a086-5fbb294d0caf' before making global git config changes
2024-10-07T11:49:28.3535220Z Adding repository directory to the temporary git global config as a safe directory
2024-10-07T11:49:28.3545549Z [command]"C:\Program Files\Git\bin\git.exe" config --global --add safe.directory D:\a\fury\fury
2024-10-07T11:49:28.3798583Z [command]"C:\Program Files\Git\bin\git.exe" config --local --name-only --get-regexp core\.sshCommand
2024-10-07T11:49:28.4043736Z [command]"C:\Program Files\Git\bin\git.exe" submodule foreach --recursive "sh -c \"git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :\""
2024-10-07T11:49:28.9377435Z [command]"C:\Program Files\Git\bin\git.exe" config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
2024-10-07T11:49:28.9630620Z http.https://github.com/.extraheader
2024-10-07T11:49:28.9686176Z [command]"C:\Program Files\Git\bin\git.exe" config --local --unset-all http.https://github.com/.extraheader
2024-10-07T11:49:28.9948637Z [command]"C:\Program Files\Git\bin\git.exe" submodule foreach --recursive "sh -c \"git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :\""
2024-10-07T11:49:29.4915221Z Cleaning up orphan processes
2024-10-07T11:49:29.5286854Z Terminate orphan process: pid (6488) (java)

@An-DJ
Copy link
Contributor Author

An-DJ commented Oct 7, 2024

Another approach is replacing MSVC with Clang (related commit) but failed, either.

2024-10-07T12:49:39.0395930Z ERROR: D:/a/fury/fury/cpp/fury/meta/BUILD:11:8: Compiling cpp/fury/meta/preprocessor_test.cc failed: undeclared inclusion(s) in rule '//cpp/fury/meta:preprocessor_test':
2024-10-07T12:49:39.0400436Z this rule is missing dependency declarations for the following files included by 'cpp/fury/meta/preprocessor_test.cc':
2024-10-07T12:49:39.0446231Z   'C:/Program Files/LLVM/lib/clang/18/include/vadefs.h'
2024-10-07T12:49:39.0478260Z   'C:/Program Files/LLVM/lib/clang/18/include/stddef.h'
2024-10-07T12:49:39.0514092Z   'C:/Program Files/LLVM/lib/clang/18/include/__stddef_ptrdiff_t.h'
2024-10-07T12:49:39.0515513Z   'C:/Program Files/LLVM/lib/clang/18/include/__stddef_size_t.h'
2024-10-07T12:49:39.0516882Z   'C:/Program Files/LLVM/lib/clang/18/include/__stddef_wchar_t.h'
2024-10-07T12:49:39.0517980Z   'C:/Program Files/LLVM/lib/clang/18/include/__stddef_null.h'
2024-10-07T12:49:39.0519396Z   'C:/Program Files/LLVM/lib/clang/18/include/__stddef_nullptr_t.h'
2024-10-07T12:49:39.0520771Z   'C:/Program Files/LLVM/lib/clang/18/include/__stddef_max_align_t.h'
2024-10-07T12:49:39.0521800Z   'C:/Program Files/LLVM/lib/clang/18/include/__stddef_offsetof.h'
2024-10-07T12:49:39.0522699Z   'C:/Program Files/LLVM/lib/clang/18/include/float.h'
2024-10-07T12:49:39.0523519Z   'C:/Program Files/LLVM/lib/clang/18/include/limits.h'
2024-10-07T12:49:39.0524307Z   'C:/Program Files/LLVM/lib/clang/18/include/intrin.h'
2024-10-07T12:49:39.0525122Z   'C:/Program Files/LLVM/lib/clang/18/include/x86intrin.h'
2024-10-07T12:49:39.0525964Z   'C:/Program Files/LLVM/lib/clang/18/include/ia32intrin.h'
2024-10-07T12:49:39.0526813Z   'C:/Program Files/LLVM/lib/clang/18/include/immintrin.h'
2024-10-07T12:49:39.0527691Z   'C:/Program Files/LLVM/lib/clang/18/include/x86gprintrin.h'
2024-10-07T12:49:39.0528515Z   'C:/Program Files/LLVM/lib/clang/18/include/mmintrin.h'
2024-10-07T12:49:39.0529347Z   'C:/Program Files/LLVM/lib/clang/18/include/xmmintrin.h'
2024-10-07T12:49:39.0530170Z   'C:/Program Files/LLVM/lib/clang/18/include/mm_malloc.h'
2024-10-07T12:49:39.0530970Z   'C:/Program Files/LLVM/lib/clang/18/include/stddef.h'
2024-10-07T12:49:39.0531762Z   'C:/Program Files/LLVM/lib/clang/18/include/emmintrin.h'
2024-10-07T12:49:39.0532589Z   'C:/Program Files/LLVM/lib/clang/18/include/xmmintrin.h'
2024-10-07T12:49:39.0534625Z   'C:/Program Files/LLVM/lib/clang/18/include/bmiintrin.h'
2024-10-07T12:49:39.0535464Z   'C:/Program Files/LLVM/lib/clang/18/include/fxsrintrin.h'
2024-10-07T12:49:39.0536340Z   'C:/Program Files/LLVM/lib/clang/18/include/xsaveintrin.h'
2024-10-07T12:49:39.0537224Z   'C:/Program Files/LLVM/lib/clang/18/include/adcintrin.h'
2024-10-07T12:49:39.0538230Z   'C:/Program Files/LLVM/lib/clang/18/include/stdint.h'
2024-10-07T12:49:39.0539045Z   'C:/Program Files/LLVM/lib/clang/18/include/vadefs.h'
2024-10-07T12:49:39.0539822Z   'C:/Program Files/LLVM/lib/clang/18/include/intrin.h'
2024-10-07T12:49:39.0540572Z   'C:/Program Files/LLVM/lib/clang/18/include/intrin.h'
2024-10-07T12:49:39.0541341Z   'C:/Program Files/LLVM/lib/clang/18/include/stddef.h'
2024-10-07T12:49:39.0542135Z   'C:/Program Files/LLVM/lib/clang/18/include/intrin.h'

Maybe we need configure more details about the clang toolchain. : (

@An-DJ
Copy link
Contributor Author

An-DJ commented Oct 7, 2024

Based on the committed work, we could build the fury lib manually. (That is, commands in Build Fury C++ can be executed successfully)

@chaokunyang Looking forward to your advice..

@chaokunyang
Copy link
Collaborator

Have tried my best already : ( but there are still strange compiling errors using MSVC toolchain. It looks like some micro(s) cannot be compiled by MSVC correctly, right?

2024-10-07T11:48:59.2650949Z 2024-10-07 11:48:59,246 - INFO - running command: bazel test //:_format //:_serialization //:_util //:cp_fury_so //:mmh3 //:python/pyfury/_serialization.pyx_cython_translation //:python/pyfury/_serialization.so //:python/pyfury/_util.pyx_cython_translation //:python/pyfury/_util.so //:python/pyfury/format/_format.pyx_cython_translation //:python/pyfury/format/_format.so //:python/pyfury/lib/mmh3/__init__.py_cython_translation //:python/pyfury/lib/mmh3/__init__.so //:python/pyfury/lib/mmh3/mmh3.pyx_cython_translation //:python/pyfury/lib/mmh3/mmh3.so //cpp/fury:fury //cpp/fury/columnar:arrow_writer_test //cpp/fury/columnar:convert_test //cpp/fury/columnar:fury_columnar_format //cpp/fury/encoder:fury_encoder //cpp/fury/encoder:row_encoder_test //cpp/fury/meta:field_info_test //cpp/fury/meta:fury_meta //cpp/fury/meta:preprocessor_test //cpp/fury/meta:type_traits_test //cpp/fury/row:fury_row_format //cpp/fury/row:row_test //cpp/fury/thirdparty:libmmh3 //cpp/fury/util:buffer_test //cpp/fury/util:fury_util //cpp/fury/util:logging_test //cpp/fury/util:status_test //cpp/fury/util:string_util_test //cpp/fury/util:time_util_test 
2024-10-07T11:48:59.4852555Z Loading: 
2024-10-07T11:48:59.5573023Z Loading: 
2024-10-07T11:48:59.5614037Z Loading: 0 packages loaded
2024-10-07T11:48:59.6769211Z Analyzing: 12 targets (0 packages loaded, 0 targets configured)
2024-10-07T11:49:01.4841282Z Analyzing: 12 targets (29 packages loaded, 6 targets configured)
2024-10-07T11:49:06.3579821Z Analyzing: 12 targets (29 packages loaded, 6 targets configured)
2024-10-07T11:49:07.4906093Z Analyzing: 12 targets (33 packages loaded, 131 targets configured)
2024-10-07T11:49:08.4914727Z Analyzing: 12 targets (34 packages loaded, 131 targets configured)
2024-10-07T11:49:18.7559121Z Analyzing: 12 targets (34 packages loaded, 131 targets configured)
2024-10-07T11:49:20.4990366Z Analyzing: 12 targets (38 packages loaded, 353 targets configured)
2024-10-07T11:49:21.4995926Z Analyzing: 12 targets (38 packages loaded, 353 targets configured)
2024-10-07T11:49:22.5529236Z Analyzing: 12 targets (47 packages loaded, 566 targets configured)
2024-10-07T11:49:22.5903226Z INFO: Analyzed 12 targets (47 packages loaded, 570 targets configured).
2024-10-07T11:49:22.5988159Z INFO: Found 12 test targets...
2024-10-07T11:49:22.7919799Z [0 / 45] [Prepa] BazelWorkspaceStatusAction stable-status.txt
2024-10-07T11:49:23.7624906Z [29 / 85] Compiling googletest/src/gtest.cc; 0s local ... (5 actions, 4 running)
2024-10-07T11:49:25.5557383Z [30 / 85] Compiling googletest/src/gtest.cc; 2s local ... (5 actions, 4 running)
2024-10-07T11:49:27.6007418Z [30 / 85] Compiling googletest/src/gtest.cc; 4s local ... (5 actions running)
2024-10-07T11:49:27.8156109Z ERROR: D:/a/fury/fury/cpp/fury/meta/BUILD:11:8: Compiling cpp/fury/meta/preprocessor_test.cc failed: (Exit 2): cl.exe failed: error executing command (from target //cpp/fury/meta:preprocessor_test) C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.41.34120\bin\HostX64\x64\cl.exe /nologo /DCOMPILER_MSVC /DNOMINMAX /D_WIN32_WINNT=0x0601 /D_CRT_SECURE_NO_DEPRECATE ... (remaining 39 arguments skipped)
2024-10-07T11:49:27.8912021Z INFO: Elapsed time: 28.604s, Critical Path: 5.08s
2024-10-07T11:49:27.8917492Z INFO: 35 processes: 35 internal.
2024-10-07T11:49:27.8920784Z FAILED: Build did NOT complete successfully
2024-10-07T11:49:27.9328969Z 2024-10-07 11:49:27,917 - ERROR - cpp/fury/meta/preprocessor_test.cc(27): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
2024-10-07T11:49:27.9332429Z cpp/fury/meta/preprocessor_test.cc(27): error C2059: syntax error: '=='
2024-10-07T11:49:27.9336551Z cpp/fury/meta/preprocessor_test.cc(28): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
2024-10-07T11:49:27.9338650Z cpp/fury/meta/preprocessor_test.cc(28): error C2059: syntax error: '=='
2024-10-07T11:49:27.9340991Z cpp/fury/meta/preprocessor_test.cc(29): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
2024-10-07T11:49:27.9342757Z cpp/fury/meta/preprocessor_test.cc(29): error C2059: syntax error: '=='
2024-10-07T11:49:27.9344530Z cpp/fury/meta/preprocessor_test.cc(30): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
2024-10-07T11:49:27.9346163Z cpp/fury/meta/preprocessor_test.cc(30): error C2059: syntax error: '=='
2024-10-07T11:49:27.9348190Z cpp/fury/meta/preprocessor_test.cc(31): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
2024-10-07T11:49:27.9349910Z cpp/fury/meta/preprocessor_test.cc(31): error C2059: syntax error: '=='
2024-10-07T11:49:27.9351634Z cpp/fury/meta/preprocessor_test.cc(32): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
2024-10-07T11:49:27.9354637Z cpp/fury/meta/preprocessor_test.cc(32): error C2059: syntax error: '=='
2024-10-07T11:49:27.9356158Z cpp/fury/meta/preprocessor_test.cc(36): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
2024-10-07T11:49:27.9357812Z cpp/fury/meta/preprocessor_test.cc(36): error C2059: syntax error: '=='
2024-10-07T11:49:27.9359260Z cpp/fury/meta/preprocessor_test.cc(40): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
2024-10-07T11:49:27.9360650Z cpp/fury/meta/preprocessor_test.cc(40): error C2059: syntax error: '=='
2024-10-07T11:49:27.9362106Z cpp/fury/meta/preprocessor_test.cc(47): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
2024-10-07T11:49:27.9364017Z cpp/fury/meta/preprocessor_test.cc(48): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
2024-10-07T11:49:27.9365891Z cpp/fury/meta/preprocessor_test.cc(49): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
2024-10-07T11:49:27.9367738Z cpp/fury/meta/preprocessor_test.cc(50): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
2024-10-07T11:49:27.9369598Z cpp/fury/meta/preprocessor_test.cc(51): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
2024-10-07T11:49:27.9371125Z cpp/fury/meta/preprocessor_test.cc(47): error C2065: 'PLUS': undeclared identifier
2024-10-07T11:49:27.9372448Z cpp/fury/meta/preprocessor_test.cc(47): error C3861: 'FURY_PP_FOREACH_IMPL_': identifier not found
2024-10-07T11:49:27.9373747Z cpp/fury/meta/preprocessor_test.cc(48): error C2065: 'PLUS': undeclared identifier
2024-10-07T11:49:27.9375035Z cpp/fury/meta/preprocessor_test.cc(48): error C3861: 'FURY_PP_FOREACH_IMPL_': identifier not found
2024-10-07T11:49:27.9376389Z cpp/fury/meta/preprocessor_test.cc(49): error C2065: 'PLUS': undeclared identifier
2024-10-07T11:49:27.9377701Z cpp/fury/meta/preprocessor_test.cc(49): error C3861: 'FURY_PP_FOREACH_IMPL_': identifier not found
2024-10-07T11:49:27.9379021Z cpp/fury/meta/preprocessor_test.cc(50): error C2065: 'PLUS': undeclared identifier
2024-10-07T11:49:27.9380310Z cpp/fury/meta/preprocessor_test.cc(50): error C3861: 'FURY_PP_FOREACH_IMPL_': identifier not found
2024-10-07T11:49:27.9381579Z cpp/fury/meta/preprocessor_test.cc(51): error C2065: 'PLUS': undeclared identifier
2024-10-07T11:49:27.9382867Z cpp/fury/meta/preprocessor_test.cc(51): error C3861: 'FURY_PP_FOREACH_IMPL_': identifier not found
2024-10-07T11:49:27.9384162Z cpp/fury/meta/preprocessor_test.cc(53): error C2131: expression did not evaluate to a constant
2024-10-07T11:49:27.9385561Z cpp/fury/meta/preprocessor_test.cc(47): note: a non-constant (sub-)expression was encountered
2024-10-07T11:49:27.9386877Z cpp/fury/meta/preprocessor_test.cc(53): note: the call stack of the evaluation (the oldest call first) is
2024-10-07T11:49:27.9389053Z cpp/fury/meta/preprocessor_test.cc(53): note: while evaluating function 'int fury::Preprocessor_ForEach_Test::TestBody::<lambda_3e1d113607ce18e4f82d08cb1698e1cf>::operator ()(void) const'
2024-10-07T11:49:27.9391249Z cpp/fury/meta/preprocessor_test.cc(56): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
2024-10-07T11:49:27.9392809Z cpp/fury/meta/preprocessor_test.cc(56): error C2065: 'STR': undeclared identifier
2024-10-07T11:49:27.9393879Z cpp/fury/meta/preprocessor_test.cc(56): error C2065: 'a': undeclared identifier
2024-10-07T11:49:27.9395104Z cpp/fury/meta/preprocessor_test.cc(56): error C2065: 'bc': undeclared identifier
2024-10-07T11:49:27.9396389Z cpp/fury/meta/preprocessor_test.cc(56): error C2065: 'def': undeclared identifier
2024-10-07T11:49:27.9397748Z cpp/fury/meta/preprocessor_test.cc(56): error C3861: 'FURY_PP_FOREACH_IMPL_': identifier not found
2024-10-07T11:49:27.9399299Z cpp/fury/meta/preprocessor_test.cc(56): error C2131: expression did not evaluate to a constant
2024-10-07T11:49:27.9400725Z cpp/fury/meta/preprocessor_test.cc(56): note: a non-constant (sub-)expression was encountered
2024-10-07T11:49:27.9402037Z cpp/fury/meta/preprocessor_test.cc(57): error C2131: expression did not evaluate to a constant
2024-10-07T11:49:27.9403691Z cpp/fury/meta/preprocessor_test.cc(57): note: failure was caused by a read of an uninitialized symbol
2024-10-07T11:49:27.9405158Z cpp/fury/meta/preprocessor_test.cc(57): note: see usage of 'strings'
2024-10-07T11:49:27.9406404Z cpp/fury/meta/preprocessor_test.cc(58): error C2131: expression did not evaluate to a constant
2024-10-07T11:49:27.9408007Z cpp/fury/meta/preprocessor_test.cc(58): note: failure was caused by out of range index 1; allowed range is 0 <= index < 0
2024-10-07T11:49:27.9409760Z cpp/fury/meta/preprocessor_test.cc(59): error C2131: expression did not evaluate to a constant
2024-10-07T11:49:27.9411215Z cpp/fury/meta/preprocessor_test.cc(59): note: failure was caused by out of range index 2; allowed range is 0 <= index < 0
2024-10-07T11:49:27.9412560Z //cpp/fury/columnar:arrow_writer_test                                 NO STATUS
2024-10-07T11:49:27.9413629Z //cpp/fury/columnar:convert_test                                      NO STATUS
2024-10-07T11:49:27.9414566Z //cpp/fury/encoder:row_encoder_test                                   NO STATUS
2024-10-07T11:49:27.9415539Z //cpp/fury/meta:field_info_test                                       NO STATUS
2024-10-07T11:49:27.9416535Z //cpp/fury/meta:type_traits_test                                      NO STATUS
2024-10-07T11:49:27.9417532Z //cpp/fury/row:row_test                                               NO STATUS
2024-10-07T11:49:27.9418524Z //cpp/fury/util:buffer_test                                           NO STATUS
2024-10-07T11:49:27.9419439Z //cpp/fury/util:logging_test                                          NO STATUS
2024-10-07T11:49:27.9420301Z //cpp/fury/util:status_test                                           NO STATUS
2024-10-07T11:49:27.9421130Z //cpp/fury/util:string_util_test                                      NO STATUS
2024-10-07T11:49:27.9422076Z //cpp/fury/util:time_util_test                                        NO STATUS
2024-10-07T11:49:27.9423066Z //cpp/fury/meta:preprocessor_test                               FAILED TO BUILD
2024-10-07T11:49:27.9423684Z 
2024-10-07T11:49:27.9424119Z Executed 0 out of 12 tests: 1 fails to build and 11 were skipped.
2024-10-07T11:49:27.9424650Z 
2024-10-07T11:49:27.9424822Z Traceback (most recent call last):
2024-10-07T11:49:27.9433310Z   File "D:\a\fury\fury\ci\run_ci.py", line 187, in <module>
2024-10-07T11:49:27.9434055Z     _parse_args()
2024-10-07T11:49:27.9434658Z   File "D:\a\fury\fury\ci\run_ci.py", line 183, in _parse_args
2024-10-07T11:49:27.9435344Z     args.func(**arg_dict)
2024-10-07T11:49:27.9435948Z   File "D:\a\fury\fury\ci\run_ci.py", line 85, in _run_cpp
2024-10-07T11:49:27.9436611Z     _exec_cmd(
2024-10-07T11:49:27.9437176Z   File "D:\a\fury\fury\ci\run_ci.py", line 43, in _exec_cmd
2024-10-07T11:49:27.9438533Z     result = subprocess.check_output(cmd, shell=True, universal_newlines=True)
2024-10-07T11:49:27.9439424Z              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-07T11:49:27.9440808Z   File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\subprocess.py", line 466, in check_output
2024-10-07T11:49:27.9442286Z     return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
2024-10-07T11:49:27.9451061Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-07T11:49:27.9452391Z   File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\subprocess.py", line 571, in run
2024-10-07T11:49:27.9453725Z     raise CalledProcessError(retcode, process.args,
2024-10-07T11:49:27.9464501Z subprocess.CalledProcessError: Command 'bazel test //:_format //:_serialization //:_util //:cp_fury_so //:mmh3 //:python/pyfury/_serialization.pyx_cython_translation //:python/pyfury/_serialization.so //:python/pyfury/_util.pyx_cython_translation //:python/pyfury/_util.so //:python/pyfury/format/_format.pyx_cython_translation //:python/pyfury/format/_format.so //:python/pyfury/lib/mmh3/__init__.py_cython_translation //:python/pyfury/lib/mmh3/__init__.so //:python/pyfury/lib/mmh3/mmh3.pyx_cython_translation //:python/pyfury/lib/mmh3/mmh3.so //cpp/fury:fury //cpp/fury/columnar:arrow_writer_test //cpp/fury/columnar:convert_test //cpp/fury/columnar:fury_columnar_format //cpp/fury/encoder:fury_encoder //cpp/fury/encoder:row_encoder_test //cpp/fury/meta:field_info_test //cpp/fury/meta:fury_meta //cpp/fury/meta:preprocessor_test //cpp/fury/meta:type_traits_test //cpp/fury/row:fury_row_format //cpp/fury/row:row_test //cpp/fury/thirdparty:libmmh3 //cpp/fury/util:buffer_test //cpp/fury/util:fury_util //cpp/fury/util:logging_test //cpp/fury/util:status_test //cpp/fury/util:string_util_test //cpp/fury/util:time_util_test ' returned non-zero exit status 1.
2024-10-07T11:49:28.0952778Z ##[error]Process completed with exit code 1.
2024-10-07T11:49:28.1122428Z Post job cleanup.
2024-10-07T11:49:28.3256650Z [command]"C:\Program Files\Git\bin\git.exe" version
2024-10-07T11:49:28.3469718Z git version 2.46.2.windows.1
2024-10-07T11:49:28.3534153Z Temporarily overriding HOME='D:\a\_temp\b85d7ab1-dde8-4df7-a086-5fbb294d0caf' before making global git config changes
2024-10-07T11:49:28.3535220Z Adding repository directory to the temporary git global config as a safe directory
2024-10-07T11:49:28.3545549Z [command]"C:\Program Files\Git\bin\git.exe" config --global --add safe.directory D:\a\fury\fury
2024-10-07T11:49:28.3798583Z [command]"C:\Program Files\Git\bin\git.exe" config --local --name-only --get-regexp core\.sshCommand
2024-10-07T11:49:28.4043736Z [command]"C:\Program Files\Git\bin\git.exe" submodule foreach --recursive "sh -c \"git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :\""
2024-10-07T11:49:28.9377435Z [command]"C:\Program Files\Git\bin\git.exe" config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
2024-10-07T11:49:28.9630620Z http.https://github.com/.extraheader
2024-10-07T11:49:28.9686176Z [command]"C:\Program Files\Git\bin\git.exe" config --local --unset-all http.https://github.com/.extraheader
2024-10-07T11:49:28.9948637Z [command]"C:\Program Files\Git\bin\git.exe" submodule foreach --recursive "sh -c \"git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :\""
2024-10-07T11:49:29.4915221Z Cleaning up orphan processes
2024-10-07T11:49:29.5286854Z Terminate orphan process: pid (6488) (java)

Looks strange. @PragmaTwice do you know where goes wrong?

@chaokunyang
Copy link
Collaborator

Build Fury C++

I don't have a windows machine for try this out, I will see whether I can reproduce this error by ssh into github windows CI machine

@chaokunyang
Copy link
Collaborator

You can use bazel build //:cp_fury_so for test, I run it in github ci machine, it failed with:

ERROR: D:/a/fury/fury/cpp/fury/thirdparty/BUILD:3:11: Compiling cpp/fury/thirdparty/MurmurHash3.cc failed: (Exit 2): cl.exe failed: error executing CppCompile command (from target //cpp/fury/thirdparty:libmmh3) C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.41.34120\bin\HostX64\x64\cl.exe @bazel-out/x64_windows-opt/bin/cpp/fury/thirdparty/_objs/libmmh3/MurmurHash3.obj.params
cpp/fury/thirdparty/MurmurHash3.cc(129): error C3861: '_rotl': identifier not found
cpp/fury/thirdparty/MurmurHash3.cc(133): error C3861: '_rotl': identifier not found
cpp/fury/thirdparty/MurmurHash3.cc(152): error C3861: '_rotl': identifier not found
cpp/fury/thirdparty/MurmurHash3.cc(196): error C3861: '_rotl': identifier not found
cpp/fury/thirdparty/MurmurHash3.cc(200): error C3861: '_rotl': identifier not found
cpp/fury/thirdparty/MurmurHash3.cc(205): error C3861: '_rotl': identifier not found
cpp/fury/thirdparty/MurmurHash3.cc(209): error C3861: '_rotl': identifier not found
cpp/fury/thirdparty/MurmurHash3.cc(214): error C3861: '_rotl': identifier not found
cpp/fury/thirdparty/MurmurHash3.cc(218): error C3861: '_rotl': identifier not found
cpp/fury/thirdparty/MurmurHash3.cc(223): error C3861: '_rotl': identifier not found
cpp/fury/thirdparty/MurmurHash3.cc(227): error C3861: '_rotl': identifier not found
cpp/fury/thirdparty/MurmurHash3.cc(250): error C3861: '_rotl': identifier not found
cpp/fury/thirdparty/MurmurHash3.cc(263): error C3861: '_rotl': identifier not found
cpp/fury/thirdparty/MurmurHash3.cc(276): error C3861: '_rotl': identifier not found
cpp/fury/thirdparty/MurmurHash3.cc(289): error C3861: '_rotl': identifier not found
cpp/fury/thirdparty/MurmurHash3.cc(350): error C3861: '_rotl64': identifier not found
cpp/fury/thirdparty/MurmurHash3.cc(354): error C3861: '_rotl64': identifier not found
cpp/fury/thirdparty/MurmurHash3.cc(359): error C3861: '_rotl64': identifier not found
cpp/fury/thirdparty/MurmurHash3.cc(363): error C3861: '_rotl64': identifier not found
cpp/fury/thirdparty/MurmurHash3.cc(392): error C3861: '_rotl64': identifier not found
cpp/fury/thirdparty/MurmurHash3.cc(413): error C3861: '_rotl64': identifier not found

I added #include <intrin.h> to make it work, then I got similar error as you:

ERROR: D:/a/fury/fury/cpp/fury/meta/BUILD:11:8: Compiling cpp/fury/meta/preprocessor_test.cc failed: (Exit 2): cl.exe failed: error executing CppCompile command (from target //cpp/fury/meta:preprocessor_test) C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.41.34120\bin\HostX64\x64\cl.exe ... (remaining 1 argument skipped)
cpp/fury/meta/preprocessor_test.cc(27): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
cpp/fury/meta/preprocessor_test.cc(27): error C2059: syntax error: '=='
cpp/fury/meta/preprocessor_test.cc(28): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
cpp/fury/meta/preprocessor_test.cc(28): error C2059: syntax error: '=='
cpp/fury/meta/preprocessor_test.cc(29): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
cpp/fury/meta/preprocessor_test.cc(29): error C2059: syntax error: '=='
cpp/fury/meta/preprocessor_test.cc(30): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
cpp/fury/meta/preprocessor_test.cc(30): error C2059: syntax error: '=='
cpp/fury/meta/preprocessor_test.cc(31): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
cpp/fury/meta/preprocessor_test.cc(31): error C2059: syntax error: '=='
cpp/fury/meta/preprocessor_test.cc(32): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
cpp/fury/meta/preprocessor_test.cc(32): error C2059: syntax error: '=='
cpp/fury/meta/preprocessor_test.cc(36): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
cpp/fury/meta/preprocessor_test.cc(36): error C2059: syntax error: '=='
cpp/fury/meta/preprocessor_test.cc(40): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
cpp/fury/meta/preprocessor_test.cc(40): error C2059: syntax error: '=='
cpp/fury/meta/preprocessor_test.cc(47): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
cpp/fury/meta/preprocessor_test.cc(48): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
cpp/fury/meta/preprocessor_test.cc(49): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
cpp/fury/meta/preprocessor_test.cc(50): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
cpp/fury/meta/preprocessor_test.cc(51): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
cpp/fury/meta/preprocessor_test.cc(47): error C2065: 'PLUS': undeclared identifier
cpp/fury/meta/preprocessor_test.cc(47): error C3861: 'FURY_PP_FOREACH_IMPL_': identifier not found
cpp/fury/meta/preprocessor_test.cc(48): error C2065: 'PLUS': undeclared identifier
cpp/fury/meta/preprocessor_test.cc(48): error C3861: 'FURY_PP_FOREACH_IMPL_': identifier not found
cpp/fury/meta/preprocessor_test.cc(49): error C2065: 'PLUS': undeclared identifier
cpp/fury/meta/preprocessor_test.cc(49): error C3861: 'FURY_PP_FOREACH_IMPL_': identifier not found
cpp/fury/meta/preprocessor_test.cc(50): error C2065: 'PLUS': undeclared identifier
cpp/fury/meta/preprocessor_test.cc(50): error C3861: 'FURY_PP_FOREACH_IMPL_': identifier not found
cpp/fury/meta/preprocessor_test.cc(51): error C2065: 'PLUS': undeclared identifier
cpp/fury/meta/preprocessor_test.cc(51): error C3861: 'FURY_PP_FOREACH_IMPL_': identifier not found
cpp/fury/meta/preprocessor_test.cc(53): error C2131: expression did not evaluate to a constant
cpp/fury/meta/preprocessor_test.cc(47): note: a non-constant (sub-)expression was encountered
cpp/fury/meta/preprocessor_test.cc(53): note: the call stack of the evaluation (the oldest call first) is
cpp/fury/meta/preprocessor_test.cc(53): note: while evaluating function 'int fury::Preprocessor_ForEach_Test::TestBody::<lambda_3e1d113607ce18e4f82d08cb1698e1cf>::operator ()(void) const'
cpp/fury/meta/preprocessor_test.cc(56): warning C4003: not enough arguments for function-like macro invocation 'FURY_PP_NARG_CALC'
cpp/fury/meta/preprocessor_test.cc(56): error C2065: 'STR': undeclared identifier
cpp/fury/meta/preprocessor_test.cc(56): error C2065: 'a': undeclared identifier
cpp/fury/meta/preprocessor_test.cc(56): error C2065: 'bc': undeclared identifier
cpp/fury/meta/preprocessor_test.cc(56): error C2065: 'def': undeclared identifier
cpp/fury/meta/preprocessor_test.cc(56): error C3861: 'FURY_PP_FOREACH_IMPL_': identifier not found
cpp/fury/meta/preprocessor_test.cc(56): error C2131: expression did not evaluate to a constant
cpp/fury/meta/preprocessor_test.cc(56): note: a non-constant (sub-)expression was encountered
cpp/fury/meta/preprocessor_test.cc(57): error C2131: expression did not evaluate to a constant
cpp/fury/meta/preprocessor_test.cc(57): note: failure was caused by a read of an uninitialized symbol
cpp/fury/meta/preprocessor_test.cc(57): note: see usage of 'strings'
cpp/fury/meta/preprocessor_test.cc(58): error C2131: expression did not evaluate to a constant
cpp/fury/meta/preprocessor_test.cc(58): note: failure was caused by out of range index 1; allowed range is 0 <= index < 0
cpp/fury/meta/preprocessor_test.cc(59): error C2131: expression did not evaluate to a constant
cpp/fury/meta/preprocessor_test.cc(59): note: failure was caused by out of range index 2; allowed range is 0 <= index < 0
Target //cpp/fury/meta:preprocessor_test failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 1.434s, Critical Path: 1.11s
INFO: 6 processes: 6 internal.
ERROR: Build did NOT complete successfully

@chaokunyang
Copy link
Collaborator

chaokunyang commented Oct 9, 2024

I don't think our current macro definition works for windows:

#define FURY_PP_NARG_IMPL(...) FURY_PP_NARG_CALC(__VA_ARGS__)

#define FURY_PP_NARG_CALC(                                                     \
    _1, _2, _3, _4, _5, _6, _7, _8, _9, _10, _11, _12, _13, _14, _15, _16,     \
    _17, _18, _19, _20, _21, _22, _23, _24, _25, _26, _27, _28, _29, _30, _31, \
    _32, _33, _34, _35, _36, _37, _38, _39, _40, _41, _42, _43, _44, _45, _46, \
    _47, _48, _49, _50, _51, _52, _53, _54, _55, _56, _57, _58, _59, _60, _61, \
    _62, _63, N, ...)                                                          \
  N

may need to change to(MSVC needs additional expansion pass) :

#define FURY_PP_EXPAND(x) x

#define FURY_PP_NARG_IMPL(...) \
    FURY_PP_EXPAND(FURY_PP_NARG_CALC(__VA_ARGS__, \
        63, 62, 61, 60, 59, 58, 57, 56, 55, 54, 53, 52, 51, 50, \
        49, 48, 47, 46, 45, 44, 43, 42, 41, 40, 39, 38, 37, 36, \
        35, 34, 33, 32, 31, 30, 29, 28, 27, 26, 25, 24, 23, 22, \
        21, 20, 19, 18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, \
        7, 6, 5, 4, 3, 2, 1, 0))

#define FURY_PP_NARG_CALC( \
    _1, _2, _3, _4, _5, _6, _7, _8, _9, _10, \
    _11, _12, _13, _14, _15, _16, _17, _18, _19, _20, \
    _21, _22, _23, _24, _25, _26, _27, _28, _29, _30, \
    _31, _32, _33, _34, _35, _36, _37, _38, _39, _40, \
    _41, _42, _43, _44, _45, _46, _47, _48, _49, _50, \
    _51, _52, _53, _54, _55, _56, _57, _58, _59, _60, \
    _61, _62, _63, N, ...) N

Could you try to make cpp/fury/meta/preprocessor.h support windows? @An-DJ

@An-DJ
Copy link
Contributor Author

An-DJ commented Oct 9, 2024

@chaokunyang __VA_ARGS__ is a standard macro syntax in C99/C11, which is not be supported in MSVC by default.

According to related documents, we can add /Zc:preprocessor to cxx flags to enable C99. (available starting in Visual Studio 2019 version 16.5)

Thanks to Microsoft : )

@An-DJ An-DJ marked this pull request as ready for review October 9, 2024 17:21
Copy link
Collaborator

@chaokunyang chaokunyang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks awesome, seems we can support pyfury for Windows now. Thank you @An-DJ

@chaokunyang chaokunyang merged commit 0e20458 into apache:main Oct 9, 2024
38 checks passed
@An-DJ An-DJ deleted the feat/support-windows-build branch October 10, 2024 01:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants