Build errors: error: unknown type name 'StringPiece' #7692

mbasmanova · 2023-11-22T12:11:02Z

Problem description

Seeing build errors on latest main. CC: @pedroerp @majetideepak @kgpai

In file included from /Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-build/src/planner/binder/tableref/ub_duckdb_bind_tableref.cpp:8:
In file included from /Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-src/src/planner/binder/tableref/bind_table_function.cpp:22:
In file included from /Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-src/src/include/duckdb/function/table/read_csv.hpp:13:
In file included from /Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-src/src/include/duckdb/execution/operator/persistent/csv_reader_options.hpp:18:
In file included from /Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-src/src/include/duckdb/common/multi_file_reader_options.hpp:12:
/Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-src/src/include/duckdb/common/hive_partitioning.hpp:28:75: error: use of undeclared identifier 'duckdb_re2'; did you mean 'duckdb_hll'?
        DUCKDB_API static std::map<string, string> Parse(const string &filename, duckdb_re2::RE2 &regex);
                                                                                 ^~~~~~~~~~
                                                                                 duckdb_hll
/Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-src/src/include/duckdb/common/types/hyperloglog.hpp:15:11: note: 'duckdb_hll' declared here
namespace duckdb_hll {
          ^
In file included from /Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-build/src/planner/binder/tableref/ub_duckdb_bind_tableref.cpp:8:
In file included from /Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-src/src/planner/binder/tableref/bind_table_function.cpp:22:
In file included from /Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-src/src/include/duckdb/function/table/read_csv.hpp:13:
In file included from /Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-src/src/include/duckdb/execution/operator/persistent/csv_reader_options.hpp:18:
In file included from /Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-src/src/include/duckdb/common/multi_file_reader_options.hpp:12:
/Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-src/src/include/duckdb/common/hive_partitioning.hpp:28:75: error: no type named 'RE2' in namespace 'duckdb_hll'; did you mean simply 'RE2'?
        DUCKDB_API static std::map<string, string> Parse(const string &filename, duckdb_re2::RE2 &regex);
                                                                                 ^~~~~~~~~~~~~~~
                                                                                 RE2
/usr/local/include/re2/re2.h:1011:12: note: 'RE2' declared here
using re2::RE2;
           ^
2 errors generated.

System information

Velox System Info v0.0.2
Commit: 6d4b6eb
CMake Version: 3.26.4
System: Darwin-23.1.0
Arch: x86_64
C++ Compiler: /Library/Developer/CommandLineTools/usr/bin/c++
C++ Compiler Version: 15.0.0.15000040
C Compiler: /Library/Developer/CommandLineTools/usr/bin/cc
C Compiler Version: 15.0.0.15000040
CMake Prefix Path: /Library/Developer/CommandLineTools/SDKs/MacOSX14.0.sdk/usr;/usr/local;/usr;/;/usr/local/Cellar/cmake/3.26.4;/usr/local;/usr/X11R6;/usr/pkg;/opt;/sw;/opt/local

Conda Env

# packages in environment at /usr/local/Caskroom/miniconda/base:
#
# Name                    Version                   Build  Channel
brotlipy                  0.7.0           py310hca72f7f_1002
bzip2                     1.0.8                h1de35cc_0
ca-certificates           2023.01.10           hecd8cb5_0
certifi                   2022.12.7       py310hecd8cb5_0
cffi                      1.15.1          py310h6c40b1e_3
charset-normalizer        2.0.4              pyhd3eb1b0_0
chevron                   0.14.0                   pypi_0    pypi
conda                     23.1.0          py310hecd8cb5_0
conda-content-trust       0.1.3           py310hecd8cb5_0
conda-package-handling    2.0.2           py310hecd8cb5_0
conda-package-streaming   0.7.0           py310hecd8cb5_0
cryptography              38.0.4          py310hf6deb26_0
idna                      3.4             py310hecd8cb5_0
libffi                    3.4.2                hecd8cb5_6
ncurses                   6.4                  hcec6c5f_0
openssl                   1.1.1s               hca72f7f_0
pip                       22.3.1          py310hecd8cb5_0
pluggy                    1.0.0           py310hecd8cb5_1
pycosat                   0.6.4           py310hca72f7f_0
pycparser                 2.21               pyhd3eb1b0_0
pyopenssl                 22.0.0             pyhd3eb1b0_0
pysocks                   1.7.1           py310hecd8cb5_0
python                    3.10.9               h218abb5_0
python.app                3               py310hca72f7f_0
pyyaml                    6.0                      pypi_0    pypi
readline                  8.2                  hca72f7f_0
regex                     2022.10.31               pypi_0    pypi
requests                  2.28.1          py310hecd8cb5_0
ruamel.yaml               0.17.21         py310hca72f7f_0
ruamel.yaml.clib          0.2.6           py310hca72f7f_1
setuptools                65.6.3          py310hecd8cb5_0
six                       1.16.0             pyhd3eb1b0_1
sqlite                    3.40.1               h880c91c_0
tk                        8.6.12               h5d9f67b_0
toolz                     0.12.0          py310hecd8cb5_0
tqdm                      4.64.1          py310hecd8cb5_0
tzdata                    2022g                h04d1e81_0
urllib3                   1.26.14         py310hecd8cb5_0
wheel                     0.37.1             pyhd3eb1b0_0
xz                        5.2.10               h6c40b1e_1
zlib                      1.2.13               h4dc903c_0
zstandard                 0.18.0          py310hca72f7f_0

CMake log

-- Build files have been written to: /Users/mbasmanova/cpp/velox-1/_build/debug
/Library/Developer/CommandLineTools/usr/bin/make build BUILD_DIR=debug -j 16
cmake --build _build/debug -j 16
[7/1484] Building CXX object _deps/duckdb-build/src/optimizer/rule/CMakeFiles/duckdb_optimizer_rules.dir/ub_duckdb_optimizer_rules.cpp.o
FAILED: _deps/duckdb-build/src/optimizer/rule/CMakeFiles/duckdb_optimizer_rules.dir/ub_duckdb_optimizer_rules.cpp.o
ccache /Library/Developer/CommandLineTools/usr/bin/c++ -DDUCKDB -DDUCKDB_MAIN_LIBRARY -DFOLLY_HAVE_INT128_T=1 -I/usr/local/include -I/Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-src/src/include -I/Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-src/third_party/fsst -I/Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-src/third_party/fmt/include -I/Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-src/third_party/hyperloglog -I/Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-src/third_party/fastpforlib -I/Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-src/third_party/fast_float -I/Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-src/third_party/re2 -I/Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-src/third_party/miniz -I/Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-src/third_party/utf8proc/include -I/Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-src/third_party/miniparquet -I/Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-src/third_party/concurrentqueue -I/Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-src/third_party/pcg -I/Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-src/third_party/tdigest -I/Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-src/third_party/mbedtls/include -I/Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-src/third_party/jaro_winkler -mavx2 -mfma -mavx -mf16c -mlzcnt -std=c++17 -mbmi2 -fvisibility=hidden -fvisibility-inlines-hidden -D USE_VELOX_COMMON_BASE -D HAS_UNCAUGHT_EXCEPTIONS -Wall -Wextra -Wno-unused        -Wno-unused-parameter        -Wno-sign-compare        -Wno-ignored-qualifiers        -Wno-range-loop-analysis          -Wno-mismatched-tags          -Wno-nullability-completeness -g -g -O0 -DDEBUG -Wall   -Werror  -Wunused -Werror=vla -Wnarrowing -pedantic -Wextra -Wno-unused-parameter -Wno-redundant-move -Wexit-time-destructors -std=c++11 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.0.sdk -fPIC -fcolor-diagnostics -MD -MT _deps/duckdb-build/src/optimizer/rule/CMakeFiles/duckdb_optimizer_rules.dir/ub_duckdb_optimizer_rules.cpp.o -MF _deps/duckdb-build/src/optimizer/rule/CMakeFiles/duckdb_optimizer_rules.dir/ub_duckdb_optimizer_rules.cpp.o.d -o _deps/duckdb-build/src/optimizer/rule/CMakeFiles/duckdb_optimizer_rules.dir/ub_duckdb_optimizer_rules.cpp.o -c /Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-build/src/optimizer/rule/ub_duckdb_optimizer_rules.cpp
In file included from /Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-build/src/optimizer/rule/ub_duckdb_optimizer_rules.cpp:16:
In file included from /Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-src/src/optimizer/rule/regex_optimizations.cpp:8:
/Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-src/third_party/re2/re2/regexp.h:196:28: error: unknown type name 'StringPiece'; did you mean 're2::StringPiece'?
  void set_error_arg(const StringPiece& error_arg) { error_arg_ = error_arg; }
                           ^~~~~~~~~~~
                           re2::StringPiece
/usr/local/include/re2/stringpiece.h:39:7: note: 're2::StringPiece' declared here
class StringPiece {
      ^
In file included from /Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-build/src/optimizer/rule/ub_duckdb_optimizer_rules.cpp:16:
In file included from /Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-src/src/optimizer/rule/regex_optimizations.cpp:8:
/Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-src/third_party/re2/re2/regexp.h:199:9: error: unknown type name 'StringPiece'; did you mean 're2::StringPiece'?
  const StringPiece& error_arg() const { return error_arg_; }
        ^~~~~~~~~~~
        re2::StringPiece
/usr/local/include/re2/stringpiece.h:39:7: note: 're2::StringPiece' declared here
class StringPiece {
      ^
In file included from /Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-build/src/optimizer/rule/ub_duckdb_optimizer_rules.cpp:16:
In file included from /Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-src/src/optimizer/rule/regex_optimizations.cpp:8:
/Users/mbasmanova/cpp/velox-1/_build/debug/_deps/duckdb-src/third_party/re2/re2/regexp.h:215:3: error: unknown type name 'StringPiece'; did you mean 're2::StringPiece'?
  StringPiece error_arg_;  // Piece of regexp containing syntax error.
  ^~~~~~~~~~~
  re2::StringPiece
/usr/local/include/re2/stringpiece.h:39:7: note: 're2::StringPiece' declared here
class StringPiece {
      ^

The text was updated successfully, but these errors were encountered:

mbasmanova · 2023-11-22T12:11:14Z

CC: @assignUser

mbasmanova · 2023-11-22T12:12:32Z

I wonder if this is due to #6725

CC: @majetideepak

mbasmanova · 2023-11-22T12:57:54Z

Tried a clean build. Still seeing errors.

In file included from /Users/mbasmanova/cpp/velox-1/velox/common/process/ProcessBase.cpp:25:
In file included from /Users/mbasmanova/cpp/velox-1/cmake-build-debug/_deps/folly-src/folly/FileUtil.h:26:
/Users/mbasmanova/cpp/velox-1/cmake-build-debug/_deps/folly-src/folly/Range.h:197:12: error: 'char_traits<unsigned char>' is deprecated: char_traits<T> for T not equal to char, wchar_t, char8_t, char16_t or char32_t is non-standard and is provided for a temporary period. It will be removed in LLVM 18, so please migrate off of it. [-Werror,-Wdeprecated-declarations]
      std::char_traits<typename std::remove_const<value_type>::type>;
           ^
/Users/mbasmanova/cpp/velox-1/cmake-build-debug/_deps/folly-src/folly/Range.h:110:20: note: in instantiation of template class 'folly::Range<const unsigned char *>' requested here
    const typename Range<Iter>::value_type& needle);
                   ^
/Users/mbasmanova/cpp/velox-1/cmake-build-debug/_deps/folly-src/folly/Range.h:1495:15: note: while substituting deduced template arguments into function template 'qfind' [with Iter = const unsigned char *]
inline size_t qfind(
              ^
/Library/Developer/CommandLineTools/SDKs/MacOSX14.0.sdk/usr/include/c++/v1/__string/char_traits.h:79:8: note: 'char_traits<unsigned char>' has been explicitly marked deprecated here
struct _LIBCPP_DEPRECATED_("char_traits<T> for T not equal to char, wchar_t, char8_t, char16_t or char32_t is non-standard and is provided for a temporary period. It will be removed in LLVM 18, so please migrate off of it.")
       ^
/Library/Developer/CommandLineTools/SDKs/MacOSX14.0.sdk/usr/include/c++/v1/__config:836:53: note: expanded from macro '_LIBCPP_DEPRECATED_'
#      define _LIBCPP_DEPRECATED_(m) __attribute__((deprecated(m)))
                                                    ^
1 error generated.
make[3]: *** [velox/common/process/CMakeFiles/velox_process.dir/ProcessBase.cpp.o] Error 1
make[3]: *** Waiting for unfinished jobs....

mbasmanova · 2023-11-22T12:58:51Z

Actually, the latest errors come after changing cmake to

#if(${VELOX_ENABLE_DUCKDB})
#  set_source(DuckDB)
#  resolve_dependency(DuckDB)
#endif()

include_directories(SYSTEM velox)
include_directories(SYSTEM velox/external)
include_directories(SYSTEM velox/external/duckdb)
include_directories(SYSTEM velox/external/duckdb/tpch/dbgen/include)

mbasmanova · 2023-11-22T13:03:43Z

Also seeing these errors:

/Users/mbasmanova/cpp/velox-1/cmake-build-debug/_deps/duckdb-src/third_party/fmt/format.cc:40:36: error: no template named 'buffer' in namespace 'fmt::internal'; did you mean 'detail::buffer'?
char* sprintf_format(Double value, internal::buffer<char>& buf,
                                   ^~~~~~~~~~~~~~~~
                                   detail::buffer
/usr/local/include/fmt/core.h:749:29: note: 'detail::buffer' declared here
template <typename T> class buffer {
                            ^
/Users/mbasmanova/cpp/velox-1/cmake-build-debug/_deps/duckdb-src/third_party/fmt/format.cc:81:20: error: no member named 'to_unsigned' in namespace 'fmt::internal'; did you mean 'detail::to_unsigned'?
      unsigned n = internal::to_unsigned(result);
                   ^~~~~~~~~~~~~~~~~~~~~
                   detail::to_unsigned
/usr/local/include/fmt/core.h:412:20: note: 'detail::to_unsigned' declared here
FMT_CONSTEXPR auto to_unsigned(Int value) ->
                   ^
/Users/mbasmanova/cpp/velox-1/cmake-build-debug/_deps/duckdb-src/third_party/fmt/format.cc:98:54: error: use of undeclared identifier 'to_unsigned'; did you mean 'detail::to_unsigned'?
                if (p != end) std::memmove(where, p, to_unsigned(end - p));
                                                     ^~~~~~~~~~~
                                                     detail::to_unsigned
/usr/local/include/fmt/core.h:412:20: note: 'detail::to_unsigned' declared here
FMT_CONSTEXPR auto to_unsigned(Int value) ->
                   ^
/Users/mbasmanova/cpp/velox-1/cmake-build-debug/_deps/duckdb-src/third_party/fmt/format.cc:104:13: error: no member named 'resize' in 'fmt::detail::buffer<char>'
        buf.resize(n);
        ~~~ ^
/Users/mbasmanova/cpp/velox-1/cmake-build-debug/_deps/duckdb-src/third_party/fmt/format.cc:107:11: error: no member named 'reserve' in 'fmt::detail::buffer<char>'
      buf.reserve(n + 1);
      ~~~ ^
/Users/mbasmanova/cpp/velox-1/cmake-build-debug/_deps/duckdb-src/third_party/fmt/format.cc:111:11: error: no member named 'reserve' in 'fmt::detail::buffer<char>'
      buf.reserve(buf.capacity() + 1);
      ~~~ ^
/Users/mbasmanova/cpp/velox-1/cmake-build-debug/_deps/duckdb-src/third_party/fmt/format.cc:118:57: error: no template named 'buffer' in namespace 'fmt::internal'; did you mean 'detail::buffer'?
template FMT_API char* internal::sprintf_format(double, internal::buffer<char>&,
                                                        ^~~~~~~~~~~~~~~~
                                                        detail::buffer

majetideepak · 2023-11-22T14:01:06Z

I am taking a look as well.

majetideepak · 2023-11-22T14:26:49Z

@mbasmanova can you delete your cmake-build-debug folder and try again?
I can build locally.
CI is okay as well.
https://app.circleci.com/pipelines/github/facebookincubator/velox/39083/workflows/66785eb4-ebfa-40e4-99b7-b199f457e4d8/jobs/264693

majetideepak · 2023-11-22T14:53:53Z

Actually, the latest errors come after changing cmake to

#if(${VELOX_ENABLE_DUCKDB})
#  set_source(DuckDB)
#  resolve_dependency(DuckDB)
#endif()

include_directories(SYSTEM velox)
include_directories(SYSTEM velox/external)
include_directories(SYSTEM velox/external/duckdb)
include_directories(SYSTEM velox/external/duckdb/tpch/dbgen/include)

Why are these changes needed?

mbasmanova · 2023-11-22T14:58:08Z

Why are these changes needed?

Was just trying to get unblocked. I tried deleting cmake-build-debug already. Still can't build.

majetideepak · 2023-11-22T15:00:00Z

Was just trying to get unblocked

Got it. DuckDB inside the external folder is now fully removed.
include_directories(SYSTEM velox/external/duckdb)

mbasmanova · 2023-11-22T15:01:16Z

@majetideepak Yes... but the problem is I can no longer build the project.

mbasmanova · 2023-11-22T15:02:55Z

I'm seeing changes in .circleci/dist_compile.yml to make CI happy. I wonder if something similar is needed in local setup.

mbasmanova · 2023-11-22T15:05:17Z

if(${VELOX_ENABLE_DUCKDB})
  set_source(DuckDB)
  resolve_dependency(DuckDB)
endif()

What does this do? Is this supposed to checkout DuckDB code from GitHub and build it locally? Is this resulting in the following errors?

In file included from /Users/mbasmanova/cpp/velox-1/cmake-build-debug/_deps/duckdb-build/src/optimizer/rule/ub_duckdb_optimizer_rules.cpp:16:
In file included from /Users/mbasmanova/cpp/velox-1/cmake-build-debug/_deps/duckdb-src/src/optimizer/rule/regex_optimizations.cpp:8:
/Users/mbasmanova/cpp/velox-1/cmake-build-debug/_deps/duckdb-src/third_party/re2/re2/regexp.h:196:28: error: unknown type name 'StringPiece'; did you mean 're2::StringPiece'?
  void set_error_arg(const StringPiece& error_arg) { error_arg_ = error_arg; }
                           ^~~~~~~~~~~
                           re2::StringPiece

mbasmanova · 2023-11-22T15:06:33Z

Looks like I have fresh code in cmake-build-debug/_deps/duckdb-src, but it fails to build...

% ls -lt cmake-build-debug/_deps/duckdb-src
total 384
-rw-r--r--   1 mbasmanova  staff   37138 Nov 22 10:03 CMakeLists.txt
drwxr-xr-x  15 mbasmanova  staff     480 Nov 22 10:03 tools
drwxr-xr-x  27 mbasmanova  staff     864 Nov 22 10:03 third_party
drwxr-xr-x  33 mbasmanova  staff    1056 Nov 22 10:03 test
drwxr-xr-x  18 mbasmanova  staff     576 Nov 22 10:03 src
drwxr-xr-x  71 mbasmanova  staff    2272 Nov 22 10:03 scripts
drwxr-xr-x   8 mbasmanova  staff     256 Nov 22 10:03 logo
drwxr-xr-x  16 mbasmanova  staff     512 Nov 22 10:03 extension
drwxr-xr-x   9 mbasmanova  staff     288 Nov 22 10:03 examples
drwxr-xr-x   5 mbasmanova  staff     160 Nov 22 10:03 data
drwxr-xr-x  18 mbasmanova  staff     576 Nov 22 10:03 benchmark
-rw-r--r--   1 mbasmanova  staff    2582 Nov 22 10:03 README.md
-rw-r--r--   1 mbasmanova  staff   11103 Nov 22 10:03 Makefile
-rw-r--r--   1 mbasmanova  staff    1072 Nov 22 10:03 LICENSE
-rw-r--r--   1 mbasmanova  staff     326 Nov 22 10:03 DuckDBConfigVersion.cmake.in
-rw-r--r--   1 mbasmanova  staff     546 Nov 22 10:03 DuckDBConfig.cmake.in
-rw-r--r--   1 mbasmanova  staff  104247 Nov 22 10:03 Doxyfile
-rw-r--r--   1 mbasmanova  staff    8339 Nov 22 10:03 CONTRIBUTING.md
-rw-r--r--   1 mbasmanova  staff    1354 Nov 22 10:03 CODE_OF_CONDUCT.md
-rw-r--r--   1 mbasmanova  staff     347 Nov 22 10:03 CITATION.cff

mbasmanova · 2023-11-22T15:07:36Z

Should I be able to build using 'make debug' from inside cmake-build-debug/_deps/duckdb-src ?

majetideepak · 2023-11-22T15:08:58Z

My hunch is that your system re2 /usr/local/include/re2/re2.h is conflicting with DuckDB's build. Can you remove your system re2 (headers and library). Velox can build re2 via bundling.

mbasmanova · 2023-11-22T15:09:29Z

This is the command used by CLion to build:

/Applications/CLion.app/Contents/bin/cmake/mac/bin/cmake --build /Users/mbasmanova/cpp/velox-1/cmake-build-debug --target velox_aggregation_fuzzer_test -- -j 12

mbasmanova · 2023-11-22T15:10:37Z

Can you remove your system re2 (headers and library). Velox can build re2 via bundling.

If I remove it... will it not cause problems building something other than Velox? Or are you saying Velox will re-create these files during buid?

mbasmanova · 2023-11-22T15:11:03Z

Is there a safer way to tell Velox not to use system's RE2?

mbasmanova · 2023-11-22T15:13:01Z

Interestingly make debug from cmake-build-debug/_deps/duckdb-src is making progress.

majetideepak · 2023-11-22T15:16:17Z

There is a safe way: You can specify re2_SOURCE=BUNDLED in the environment and run CMake again.

majetideepak · 2023-11-22T15:17:11Z

You should see Using BUNDLED re2 in the CMake output.

majetideepak · 2023-11-22T15:18:06Z

But /usr/local/include/ is a standard system path. So I am afraid the conflict could still exist if DuckDB's build system picks this up.

mbasmanova · 2023-11-22T15:20:20Z

Hmm... doesn't seem to work

-- Setting re2 source to AUTO
-- Found re2: /usr/local/lib/libre2.a  
-- Using SYSTEM re2

majetideepak · 2023-11-22T15:23:49Z

Can you try deleting the cmake-build-debug/CMakeCache.txt?

mbasmanova · 2023-11-22T15:23:50Z

I'm also seeing

[ 0%] Building CXX object _deps/re2-build/CMakeFiles/re2.dir/re2/bitstate.cc.o

which seems to be in conflict with

Using SYSTEM re2

mbasmanova · 2023-11-22T15:28:51Z

Deleted cmake-build-debug again. Seeing this now.

-- Setting re2 source to BUNDLED
-- Building re2 from source
...
-- Building DuckDB from source
-- Found Git: /usr/local/bin/git (found version "2.31.1") 
fatal: No names found, cannot describe anything.
fatal: No names found, cannot describe anything.
-- git hash 6d4b6ebec, version v..1-dev
Treating warnings as errors.
-- Using BUNDLED DuckDB

majetideepak · 2023-11-22T15:32:30Z

Does CMake fail?

mbasmanova · 2023-11-22T15:33:20Z

Still build fails.

In file included from /Users/mbasmanova/cpp/velox-1/cmake-build-debug/_deps/duckdb-build/src/optimizer/rule/ub_duckdb_optimizer_rules.cpp:16:
In file included from /Users/mbasmanova/cpp/velox-1/cmake-build-debug/_deps/duckdb-src/src/optimizer/rule/regex_optimizations.cpp:8:
/Users/mbasmanova/cpp/velox-1/cmake-build-debug/_deps/duckdb-src/third_party/re2/re2/regexp.h:196:28: error: unknown type name 'StringPiece'; did you mean 're2::StringPiece'?
  void set_error_arg(const StringPiece& error_arg) { error_arg_ = error_arg; }
                           ^~~~~~~~~~~
                           re2::StringPiece

I'm puzzled why are we building duckdb again? It seems like it should have been built during cmake

/Applications/CLion.app/Contents/bin/cmake/mac/bin/cmake -DCMAKE_BUILD_TYPE=Debug -DTREAT_WARNINGS_AS_ERRORS=1 -DENABLE_ALL_WARNINGS=1 -DCMAKE_BUILD_TYPE=Debug -G "CodeBlocks - Unix Makefiles" -S /Users/mbasmanova/cpp/velox-1 -B /Users/mbasmanova/cpp/velox-1/cmake-build-debug

but looks like it being built again during Velox build

/Applications/CLion.app/Contents/bin/cmake/mac/bin/cmake --build /Users/mbasmanova/cpp/velox-1/cmake-build-debug --target velox_aggregation_fuzzer_test -- -j 12
...
[  4%] Built target duckdb_func_map_nested
[  4%] Building CXX object _deps/duckdb-build/src/core_functions/scalar/operators/CMakeFiles/duckdb_func_ops.dir/ub_duckdb_func_ops.cpp.o
[  4%] Building CXX object _deps/duckdb-build/src/core_functions/scalar/random/CMakeFiles/duckdb_func_random.dir/ub_duckdb_func_random.cpp.o
[  4%] Building CXX object _deps/duckdb-build/src/core_functions/scalar/string/CMakeFiles/duckdb_func_string.dir/ub_duckdb_func_string.cpp.o

majetideepak · 2023-11-22T15:35:40Z

| I'm puzzled why are we building duckdb again? It seems like it should have been built during cmake

The fetch and prepare of DuckDB happens during CMake. The DuckDB build does happen during the Velox build.

mbasmanova · 2023-11-22T15:36:18Z

The DuckDB build does happen during the Velox build.

Got it.. This is still failing then.

mbasmanova · 2023-11-22T15:36:59Z

On retry, sometimes I get a different error:

In file included from /Users/mbasmanova/cpp/velox-1/cmake-build-debug/_deps/duckdb-build/src/core_functions/scalar/operators/ub_duckdb_func_ops.cpp:2:
In file included from /Users/mbasmanova/cpp/velox-1/cmake-build-debug/_deps/duckdb-src/src/core_functions/scalar/operators/bitwise.cpp:3:
/Users/mbasmanova/cpp/velox-1/cmake-build-debug/_deps/duckdb-src/src/include/duckdb/common/types/cast_helpers.hpp:47:13: error: use of undeclared identifier 'duckdb_fmt'; did you mean 'duckdb_hll'?
                        *--ptr = duckdb_fmt::internal::data::digits[index + 1];
                                 ^~~~~~~~~~
                                 duckdb_hll

majetideepak · 2023-11-22T15:40:37Z

Another unblock option is to build DuckDB externally. CI has steps here.
https://github.com/facebookincubator/velox/blob/main/.circleci/dist_compile.yml#L34
You can then specify DuckDB_SOURCE=SYSTEM.

majetideepak · 2023-11-22T15:47:09Z

ls ./cmake-build-debug/_deps/duckdb-build/third_party/
CMakeFiles		fastpforlib		fsst			libpg_query		miniz			utf8proc
cmake_install.cmake	fmt			hyperloglog		mbedtls			re2

I see that DuckDB builds the above libraries. I am guessing there could be a conflict if they are available in the system as well.

mbasmanova · 2023-11-22T15:53:46Z

Trying to build DuckDB externally.

CMake Error at src/cmake_install.cmake:76 (file):
  file INSTALL cannot copy file
  "/Users/mbasmanova/cpp/duckdb/duckdb-0.8.1/build/src/libduckdb.dylib" to
  "/usr/local/lib/libduckdb.dylib": Permission denied.
Call Stack (most recent call first):
  cmake_install.cmake:96 (include)

make: *** [install] Error 1

mbasmanova · 2023-11-22T15:59:53Z

Figured out permission issue. Trying to build with DuckDB_SOURCE=SYSTEM now.

mbasmanova · 2023-11-22T16:01:46Z

Different error:

In file included from /Users/mbasmanova/cpp/velox-1/velox/type/tz/TimeZoneMap.cpp:19:
In file included from /Users/mbasmanova/cpp/velox-1/cmake-build-debug/_deps/folly-src/folly/container/F14Map.h:35:
/Users/mbasmanova/cpp/velox-1/cmake-build-debug/_deps/folly-src/folly/Range.h:197:12: error: 'char_traits<unsigned char>' is deprecated: char_traits<T> for T not equal to char, wchar_t, char8_t, char16_t or char32_t is non-standard and is provided for a temporary period. It will be removed in LLVM 18, so please migrate off of it. [-Werror,-Wdeprecated-declarations]
      std::char_traits<typename std::remove_const<value_type>::type>;

czentgr · 2023-11-22T16:15:23Z

A quick workaround is to disable deprecated warnings. This can be done in the folly CMakeLists.txt (for the bundled version) or in the Velox CMakeLists.txt (velox/CMakeLists.txt).
For the latter add on line 322 (after the other C++ flags are set)

set(CMAKE_CXX_FLAGS "-Wno-deprecated-declarations")

czentgr · 2023-11-22T16:25:25Z

For the other build issues, does CLion show you the complete command it tried to run (e.g. with all the command line options)?
It appears that the includes for fmt and re are not picked up from the bundled paths for duckdb - i.e. _deps/duckdb-src/third_party/fmt/include and _deps/duckdb-src/third_party/re2 - and possibly other dependencies as well.

I tried to repro the issue but could not (but I'm not using CLion). I do have re2 and fmt installed into my system under /usr/local as well. But there is no conflict.

mbasmanova · 2023-11-22T16:50:31Z

I followed Deepak's advice to build and install newer folly. That solved the problem.

#7235 (comment)

pedroerp · 2023-11-22T17:51:29Z

@majetideepak should we bump our folly version dependency then?

mbasmanova · 2023-12-06T18:02:26Z

Folks, I'm now trying to build Prestissimo and seeing more errors: prestodb/presto#21492

majetideepak · 2023-12-07T14:35:59Z

@mbasmanova With the Folly upgrade, we likely have to upgrade fmt as well. We currently use fmt 8.0.1 an ancient version. There is an issue open to upgrade #7896
I believe you should be able to fix your issue by upgrading to fmt 8.1.0. But this is not tested.
I commented on the issue above.

mbasmanova · 2023-12-07T15:06:43Z

I believe you should be able to fix your issue by upgrading to fmt 8.1.0. But this is not tested.

This is what I ended up doing yesterday to unblock. It worked. Thanks.

mbasmanova added build triage Newly created issue that needs attention. labels Nov 22, 2023

mbasmanova mentioned this issue Nov 22, 2023

Upgrade DuckDB to 0.8.1 #6725

Closed

majetideepak self-assigned this Nov 22, 2023

mbasmanova mentioned this issue Nov 22, 2023

Velox build failing on macOS Sonoma (Xcode 15) with deprecated warnings as errors #7235

Closed

majetideepak mentioned this issue Nov 25, 2023

Move duckdb compile before re2 and fmt #7722

Closed

assignUser linked a pull request Dec 6, 2023 that will close this issue

Upgrade folly to v2023.12.04.00 (from v2022.11.14.00) #7700

Closed

majetideepak mentioned this issue Dec 12, 2023

Upgrade folly to v2023.12.04.00 (from v2022.11.14.00) #7700

Closed

Build errors: error: unknown type name 'StringPiece' #7692

Build errors: error: unknown type name 'StringPiece' #7692

Comments

mbasmanova commented Nov 22, 2023

Problem description

System information

CMake log

mbasmanova commented Nov 22, 2023

mbasmanova commented Nov 22, 2023

mbasmanova commented Nov 22, 2023

mbasmanova commented Nov 22, 2023

mbasmanova commented Nov 22, 2023

majetideepak commented Nov 22, 2023

majetideepak commented Nov 22, 2023

majetideepak commented Nov 22, 2023

mbasmanova commented Nov 22, 2023

majetideepak commented Nov 22, 2023

mbasmanova commented Nov 22, 2023

mbasmanova commented Nov 22, 2023

mbasmanova commented Nov 22, 2023

mbasmanova commented Nov 22, 2023

mbasmanova commented Nov 22, 2023

majetideepak commented Nov 22, 2023

mbasmanova commented Nov 22, 2023

mbasmanova commented Nov 22, 2023

mbasmanova commented Nov 22, 2023

mbasmanova commented Nov 22, 2023

majetideepak commented Nov 22, 2023

majetideepak commented Nov 22, 2023

majetideepak commented Nov 22, 2023

mbasmanova commented Nov 22, 2023

majetideepak commented Nov 22, 2023

mbasmanova commented Nov 22, 2023

mbasmanova commented Nov 22, 2023 • edited Loading

majetideepak commented Nov 22, 2023

mbasmanova commented Nov 22, 2023

majetideepak commented Nov 22, 2023

mbasmanova commented Nov 22, 2023

mbasmanova commented Nov 22, 2023

majetideepak commented Nov 22, 2023 • edited Loading

majetideepak commented Nov 22, 2023

mbasmanova commented Nov 22, 2023

mbasmanova commented Nov 22, 2023

mbasmanova commented Nov 22, 2023

czentgr commented Nov 22, 2023

czentgr commented Nov 22, 2023

mbasmanova commented Nov 22, 2023

pedroerp commented Nov 22, 2023

mbasmanova commented Dec 6, 2023

majetideepak commented Dec 7, 2023 • edited Loading

mbasmanova commented Dec 7, 2023

mbasmanova commented Nov 22, 2023 •

edited

Loading

majetideepak commented Nov 22, 2023 •

edited

Loading

majetideepak commented Dec 7, 2023 •

edited

Loading