Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[C++] Compilation errors with re2 2023.06.02 on windows #38263

Closed
h-vetinari opened this issue Oct 13, 2023 · 4 comments · Fixed by #38265
Closed

[C++] Compilation errors with re2 2023.06.02 on windows #38263

h-vetinari opened this issue Oct 13, 2023 · 4 comments · Fixed by #38265
Assignees
Milestone

Comments

@h-vetinari
Copy link
Contributor

Describe the bug, including details regarding any error messages, version, and platform.

Due to some build changes, it took a while for conda-forge to ship re2 2023.06.02, but now that we're trying to compile arrow against that, we get some failures, though only on windows:

[221/463] Building CXX object src\arrow\CMakeFiles\arrow_shared.dir\compute\kernels\scalar_string_ascii.cc.obj
FAILED: src/arrow/CMakeFiles/arrow_shared.dir/compute/kernels/scalar_string_ascii.cc.obj 
C:\PROGRA~1\MICROS~2\2022\ENTERP~1\VC\Tools\MSVC\1429~1.301\bin\HostX64\x64\cl.exe  /nologo /TP -DABSL_CONSUME_DLL -DARROW_EXPORTING -DARROW_HAVE_RUNTIME_AVX2 -DARROW_HAVE_RUNTIME_AVX512 -DARROW_HAVE_RUNTIME_BMI2 -DARROW_HAVE_RUNTIME_SSE4_2 -DARROW_HDFS -DARROW_MIMALLOC -DARROW_S3_HAS_CRT -DARROW_WITH_BROTLI -DARROW_WITH_BZ2 -DARROW_WITH_LZ4 -DARROW_WITH_RE2 -DARROW_WITH_SNAPPY -DARROW_WITH_TIMING_TESTS -DARROW_WITH_UTF8PROC -DARROW_WITH_ZLIB -DARROW_WITH_ZSTD -DAWS_AUTH_USE_IMPORT_EXPORT -DAWS_CAL_USE_IMPORT_EXPORT -DAWS_CHECKSUMS_USE_IMPORT_EXPORT -DAWS_COMMON_USE_IMPORT_EXPORT -DAWS_COMPRESSION_USE_IMPORT_EXPORT -DAWS_CRT_CPP_USE_IMPORT_EXPORT -DAWS_EVENT_STREAM_USE_IMPORT_EXPORT -DAWS_HTTP_USE_IMPORT_EXPORT -DAWS_IO_USE_IMPORT_EXPORT -DAWS_MQTT_USE_IMPORT_EXPORT -DAWS_S3_USE_IMPORT_EXPORT -DAWS_SDKUTILS_USE_IMPORT_EXPORT -DAWS_SDK_VERSION_MAJOR=1 -DAWS_SDK_VERSION_MINOR=11 -DAWS_SDK_VERSION_PATCH=156 -DAWS_USE_IO_COMPLETION_PORTS -DBOOST_ALL_DYN_LINK -DBOOST_ALL_NO_LIB -DPROTOBUF_USE_DLLS -DURI_STATIC_BUILD -DUSE_IMPORT_EXPORT -DUSE_IMPORT_EXPORT=1 -DUSE_WINDOWS_DLL_SEMANTICS -D_CRT_SECURE_NO_WARNINGS -D_ENABLE_EXTENDED_ALIGNED_STORAGE -Darrow_shared_EXPORTS -I%SRC_DIR%\cpp\build\src -I%SRC_DIR%\cpp\src -I%SRC_DIR%\cpp\src\generated -external:I%SRC_DIR%\cpp\thirdparty\flatbuffers\include -external:I%SRC_DIR%\cpp\thirdparty\hadoop\include -external:ID:\bld\apache-arrow_1697190508068\_h_env\Library\include -external:I%SRC_DIR%\cpp\build\mimalloc_ep\src\mimalloc_ep\include\mimalloc-2.0 -external:W0 /DWIN32 /D_WINDOWS  /GR /EHsc /D_SILENCE_TR1_NAMESPACE_DEPRECATION_WARNING  /EHsc /wd5105 /bigobj /utf-8 /W3 /wd4800 /wd4996 /wd4065  /MD /O2 /Ob2 /DNDEBUG -std:c++17 /showIncludes /Fosrc\arrow\CMakeFiles\arrow_shared.dir\compute\kernels\scalar_string_ascii.cc.obj /Fdsrc\arrow\CMakeFiles\arrow_shared.dir\ /FS -c %SRC_DIR%\cpp\src\arrow\compute\kernels\scalar_string_ascii.cc
%SRC_DIR%\cpp\src\arrow\compute\kernels\scalar_string_ascii.cc(2093): error C2440: 'initializing': cannot convert from 'std::_String_view_iterator<_Traits>' to 'const char *'
        with
        [
            _Traits=std::char_traits<char>
        ]
%SRC_DIR%\cpp\src\arrow\compute\kernels\scalar_string_ascii.cc(2093): note: No user-defined-conversion operator available that can perform this conversion, or the operator cannot be called
%SRC_DIR%\cpp\src\arrow\compute\kernels\scalar_string_ascii.cc(2066): note: while compiling class template member function 'arrow::Status arrow::compute::internal::`anonymous-namespace'::RegexSubstringReplacer<arrow::BinaryType>::ReplaceString(std::string_view,arrow::TypedBufferBuilder<uint8_t,void> *) const'
%SRC_DIR%\cpp\src\arrow\compute\kernels\scalar_string_ascii.cc(1970): note: see reference to function template instantiation 'arrow::Status arrow::compute::internal::`anonymous-namespace'::RegexSubstringReplacer<arrow::BinaryType>::ReplaceString(std::string_view,arrow::TypedBufferBuilder<uint8_t,void> *) const' being compiled
%SRC_DIR%\cpp\src\arrow\compute\kernels\scalar_string_ascii.cc(1957): note: see reference to class template instantiation 'arrow::compute::internal::`anonymous-namespace'::RegexSubstringReplacer<arrow::BinaryType>' being compiled
%SRC_DIR%\cpp\src\arrow\compute\kernels\scalar_string_ascii.cc(1955): note: while compiling class template member function 'arrow::Status arrow::compute::internal::`anonymous-namespace'::ReplaceSubstring<arrow::BinaryType,arrow::compute::internal::`anonymous-namespace'::RegexSubstringReplacer<arrow::BinaryType>>::Exec(arrow::compute::KernelContext *,const arrow::compute::ExecSpan &,arrow::compute::ExecResult *)'
%SRC_DIR%\cpp\src\arrow/compute/kernels/codegen_internal.h(1243): note: see reference to function template instantiation 'arrow::Status arrow::compute::internal::`anonymous-namespace'::ReplaceSubstring<arrow::BinaryType,arrow::compute::internal::`anonymous-namespace'::RegexSubstringReplacer<arrow::BinaryType>>::Exec(arrow::compute::KernelContext *,const arrow::compute::ExecSpan &,arrow::compute::ExecResult *)' being compiled
%SRC_DIR%\cpp\src\arrow/compute/kernels/codegen_internal.h(1240): note: see reference to class template instantiation 'arrow::compute::internal::`anonymous-namespace'::ReplaceSubstring<arrow::BinaryType,arrow::compute::internal::`anonymous-namespace'::RegexSubstringReplacer<arrow::BinaryType>>' being compiled
%SRC_DIR%\cpp\src\arrow\compute\kernels\scalar_string_ascii.cc(2158): note: see reference to function template instantiation 'arrow::compute::ArrayKernelExec (__cdecl *arrow::compute::internal::GenerateVarBinaryToVarBinary<arrow::compute::internal::`anonymous-namespace'::ReplaceSubstringRegex,>(arrow::compute::internal::detail::GetTypeId))(arrow::compute::KernelContext *,const arrow::compute::ExecSpan &,arrow::compute::ExecResult *)' being compiled
%SRC_DIR%\cpp\src\arrow\compute\kernels\scalar_string_ascii.cc(2104): error C2440: '=': cannot convert from 'std::_String_view_iterator<_Traits>' to 'const char *'
        with
        [
            _Traits=std::char_traits<char>
        ]
%SRC_DIR%\cpp\src\arrow\compute\kernels\scalar_string_ascii.cc(2104): note: No user-defined-conversion operator available that can perform this conversion, or the operator cannot be called

Normally there shouldn't be big API changes in re2 (though they did change the SOVERSION per that release). This happens both with/without the unity build.

Component(s)

C++, Packaging

@raulcd
Copy link
Member

raulcd commented Oct 13, 2023

@felipecrv @bkietz @zeroshade this seems related to string view, so not sure who's best to take a look.

@raulcd raulcd changed the title Compilation errors with re2 2023.06.02 on windows [C++] Compilation errors with re2 2023.06.02 on windows Oct 13, 2023
@felipecrv
Copy link
Contributor

@felipecrv @bkietz @zeroshade this seems related to string view, so not sure who's best to take a look.

This is related to the C++ string_view, not the Arrow "string view". I will take a look.

@bkietz
Copy link
Member

bkietz commented Oct 13, 2023

It looks like previously re2::StringPiece::begin() returned a pointer, whereas now it may return an iterator struct instead. The fix is for code in arrow which uses StringPiece::begin() to replace that with StringPiece::data(), which will always return a pointer.

@felipecrv
Copy link
Contributor

It looks like previously re2::StringPiece::begin() returned a pointer, whereas now it may return an iterator struct instead. The fix is for code in arrow which uses StringPiece::begin() to replace that with StringPiece::data(), which will always return a pointer.

I checked the re2 and abseil-cpp code and that's not the case. It seems like the problem is that MSVC doesn't allow converting std::string_view::const_iterator to const char *.

@raulcd raulcd added the Priority: Blocker Marks a blocker for the release label Oct 15, 2023
@raulcd raulcd added this to the 14.0.0 milestone Oct 15, 2023
raulcd pushed a commit that referenced this issue Oct 16, 2023
… where a char pointer is expected (#38265)

### Rationale for this change

The MSVC compiler doesn't seem to allow user code to assume `std::string_view::const_iterator` is `const char*`, so using only `re2::StringPiece` and preferring to call `.data()` instead of `.begin()` should make things more uniform across different compilers and STL implementations.

### What changes are included in this PR?

 - Using `re2::StringPiece` instead of `std::string_view` to interact with `re2`
 - Use `data()` instead of `begin()` where a `char*` is expected

### Are these changes tested?

Yes, by existing tests.
* Closes: #38263

Authored-by: Felipe Oliveira Carvalho <[email protected]>
Signed-off-by: Raúl Cumplido <[email protected]>
raulcd pushed a commit that referenced this issue Oct 16, 2023
… where a char pointer is expected (#38265)

### Rationale for this change

The MSVC compiler doesn't seem to allow user code to assume `std::string_view::const_iterator` is `const char*`, so using only `re2::StringPiece` and preferring to call `.data()` instead of `.begin()` should make things more uniform across different compilers and STL implementations.

### What changes are included in this PR?

 - Using `re2::StringPiece` instead of `std::string_view` to interact with `re2`
 - Use `data()` instead of `begin()` where a `char*` is expected

### Are these changes tested?

Yes, by existing tests.
* Closes: #38263

Authored-by: Felipe Oliveira Carvalho <[email protected]>
Signed-off-by: Raúl Cumplido <[email protected]>
JerAguilon pushed a commit to JerAguilon/arrow that referenced this issue Oct 23, 2023
…egin() where a char pointer is expected (apache#38265)

### Rationale for this change

The MSVC compiler doesn't seem to allow user code to assume `std::string_view::const_iterator` is `const char*`, so using only `re2::StringPiece` and preferring to call `.data()` instead of `.begin()` should make things more uniform across different compilers and STL implementations.

### What changes are included in this PR?

 - Using `re2::StringPiece` instead of `std::string_view` to interact with `re2`
 - Use `data()` instead of `begin()` where a `char*` is expected

### Are these changes tested?

Yes, by existing tests.
* Closes: apache#38263

Authored-by: Felipe Oliveira Carvalho <[email protected]>
Signed-off-by: Raúl Cumplido <[email protected]>
loicalleyne pushed a commit to loicalleyne/arrow that referenced this issue Nov 13, 2023
…egin() where a char pointer is expected (apache#38265)

### Rationale for this change

The MSVC compiler doesn't seem to allow user code to assume `std::string_view::const_iterator` is `const char*`, so using only `re2::StringPiece` and preferring to call `.data()` instead of `.begin()` should make things more uniform across different compilers and STL implementations.

### What changes are included in this PR?

 - Using `re2::StringPiece` instead of `std::string_view` to interact with `re2`
 - Use `data()` instead of `begin()` where a `char*` is expected

### Are these changes tested?

Yes, by existing tests.
* Closes: apache#38263

Authored-by: Felipe Oliveira Carvalho <[email protected]>
Signed-off-by: Raúl Cumplido <[email protected]>
dgreiss pushed a commit to dgreiss/arrow that referenced this issue Feb 19, 2024
…egin() where a char pointer is expected (apache#38265)

### Rationale for this change

The MSVC compiler doesn't seem to allow user code to assume `std::string_view::const_iterator` is `const char*`, so using only `re2::StringPiece` and preferring to call `.data()` instead of `.begin()` should make things more uniform across different compilers and STL implementations.

### What changes are included in this PR?

 - Using `re2::StringPiece` instead of `std::string_view` to interact with `re2`
 - Use `data()` instead of `begin()` where a `char*` is expected

### Are these changes tested?

Yes, by existing tests.
* Closes: apache#38263

Authored-by: Felipe Oliveira Carvalho <[email protected]>
Signed-off-by: Raúl Cumplido <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants