CMake issues when linking against Protobuf v22.3 with Abseil dependency

[alist]

Hello!

I encountered issues when using Protobuf v22.3 with CMake due to its Abseil dependency. I have documented the steps I took and the issues I faced below.

I built and installed Protobuf v22.3 from source with the following commands:

$ cmake -Dprotobuf_BUILD_TESTS=OFF -DABSL_PROPAGATE_CXX_STD=ON -Dprotobuf_BUILD_SHARED_LIBS=OFF -DCMAKE_INSTALL_PREFIX=/usr/local ..
$ cmake --build . -j 10
$ sudo cmake --install .

Then, in my project's CMakeLists.txt, I added:

find_package(Protobuf REQUIRED)

and linked the Protobuf libraries to my target:

target_link_libraries(${TARGET_NAME} PRIVATE protobuf::libprotobuf protobuf::libprotoc)

However, when building my project, I faced multiple unresolved symbols related to utf8-range and Abseil. To resolve these issues, I tried the following:

1. Manually find_package(utf8_range) and added utf8_range::utf8_validity utf8_range::utf8_range to my target (resolving some errors).
2. Manually added find_package(absl) and and linked multiple absl::* targets (the same ones as in the cmake protobuf build script), but still had missing symbols
3. Built Protobuf as a static library and linked against it, but still faced unresolved symbols related to Abseil.

Eventually, I reverted to Protobuf v3.21.12, which has no Abseil dependency while still supporting CMake. With this version, I could build and run my project without any issues using find_package(Protobuf REQUIRED), target_link_libraries as before and nothing else.

I believe there should be a more straightforward way to handle the Abseil dependencies when using Protobuf v22.3 with CMake. Also, I still don't understand why a static build of protobuf would require additional imports. Any guidance or improvements in this area would be greatly appreciated.

Thank you!

[puremourning]

I had the exact same issue. I spent a few hours battling the linker and came up with this:

ABSL_LIBS=                         \
  -labsl_str_format_internal       \
  -labsl_strings                   \
  -labsl_strings_internal          \
  -labsl_log_initialize            \
  -labsl_log_entry                 \
  -labsl_log_flags                 \
  -labsl_log_severity              \
  -labsl_log_internal_check_op     \
  -labsl_log_internal_conditions   \
  -labsl_log_internal_message      \
  -labsl_log_internal_nullguard    \
  -labsl_log_internal_proto        \
  -labsl_log_internal_format       \
  -labsl_log_internal_globals      \
  -labsl_log_internal_log_sink_set \
  -labsl_log_sink                  \
  -labsl_raw_logging_internal      \
  -labsl_log_globals               \
  -lutf8_validity                  \
  -labsl_cord                      \
  -labsl_cordz_info                \
  -labsl_cordz_handle              \
  -labsl_cordz_functions           \
  -labsl_cord_internal             \
  -labsl_crc_cord_state            \
  -labsl_crc32c                    \
  -labsl_crc_internal              \
  -labsl_exponential_biased        \
  -labsl_synchronization           \
  -labsl_time                      \
  -labsl_time_zone                 \
  -labsl_int128                    \
  -labsl_examine_stack             \
  -labsl_stacktrace                \
  -labsl_symbolize                 \
  -labsl_demangle_internal         \
  -labsl_debugging_internal        \
  -labsl_malloc_internal           \
  -labsl_throw_delegate            \
  -labsl_strerror                  \
  -labsl_raw_hash_set              \
  -labsl_hash                      \
  -labsl_city                      \
  -labsl_low_level_hash            \
  -labsl_base                      \
  -labsl_spinlock_wait

not pretty.

[coryan]

Seems like a dup of #12292 see #12292 (comment)

You need to use find_package(Protobuf CONFIG REQUIRED)

[puremourning]

Are there instructions for using protobufs without cmake or Bazel?

[alist]

Makes sense! Can the point of this issue be to add this to the documentation somewhere? I didn't see much docs on using protobuffs with cmake.

From chat:

When you use find_package(protobuf REQUIRED), CMake will first look for a Findprotobuf.cmake module, which is a script provided by the CMake package or the user. If it doesn't find this module, it will fall back to using the package configuration file (protobuf-config.cmake), which is generated by the package itself when it's built and installed. The Findprotobuf.cmake module might not handle the Abseil dependency correctly, causing the issues you encountered.

By using find_package(protobuf REQUIRED CONFIG), you explicitly tell CMake to use the package configuration file (protobuf-config.cmake) instead of the Findprotobuf.cmake module. This file should handle the Abseil dependency correctly, making it easier to link and use Protobuf in your project.

[coryan]

Are there instructions for using protobufs without cmake or Bazel?

Maybe this example helps?

https://github.com/protocolbuffers/protobuf/blob/main/examples/Makefile

It uses pkg-config to find the flags and dependencies for protobuf.

[puremourning]

Are there instructions for using protobufs without cmake or Bazel?

Maybe this example helps?

https://github.com/protocolbuffers/protobuf/blob/main/examples/Makefile

It uses pkg-config to find the flags and dependencies for protobuf.

Thanks. Will try it. But for the record I did try the pkg config shipped with protobuf and it didn't not include any of the abseil libraries. Possible I did it wrong of course.

[coryan]

But for the record I did try the pkg config shipped with protobuf and it didn't not include any of the abseil libraries. Possible I did it wrong of course.

I think it is likely that you did nothing wrong, but ran into #12439, or #12698, or some other problem. If you have a repro that would help the Protobuf team.

[github-actions]

We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please add a comment.

This issue is labeled inactive because the last activity was over 90 days ago.

[fwsGonzo]

This is horrible! How is adding a new dependency like that breaking builds everywhere good software practice?

[fwsGonzo]

Solution that worked for me on MinGW:

$ wget https://repo.msys2.org/mingw/mingw64/mingw-w64-x86_64-protobuf-21.9-1-any.pkg.tar.zst
$ pacman -U --noconfirm mingw-w64-x86_64-protobuf-21.9-1-any.pkg.tar.zst
$ cmake ..
$ ninja

Basically downgrade to a version with the old dependencies. I will migrate to capnproto whenever this is no longer possible.

[jonringer]

For anyone stumbling upon this.. This should work with more recent versions of protobuf:

# populates protobuf:: targets
find_package(Protobuf REQUIRED)

# Communicate to downstream consumers that they also need to link against libprotobuf
target_link_libraries(${TARGET_NAME}-lib
  PUBLIC protobuf::libprotobuf
)

# Should also link against abseil and protobuf
target_link_libraries(${TARGET_NAME}-exe ${TARGET_NAME}-lib)

[partisansb]

Where to add that code? I placed it in CMakeLists.txt near the end after else () find_package(Protobuf NO_MODULE) if (Protobuf_FOUND) set(protobuf_PROTOC_EXE protobuf::protoc) set(protobuf_LIB_PROTOC protobuf::libprotoc) set(protobuf_LIB_PROTOBUF protobuf::libprotobuf) set(protobuf_LIB_PROTOBUF_LITE protobuf::libprotobuf-lite) message(STATUS "CMake installation of Protobuf found.") endif () endif ()

And it just returned errors during $cmake -D BUILD_SHARED_LIBS=ON -D protobuf_BUILD_TESTS=ON -D utf8_range_ENABLE_INSTALL=ON .

This was on version 25.1 on a Debian flavour linux

My main problem is that cmake is not generating protobuf-targets.cmake and then not linking utf-8_range.

Been trying to resolve this issue for 3 days now. Tried so many different versions, and edited files, but not getting anywhere. Anyone know why the script would not create that file?

[jonringer]

My main problem is that cmake is not generating protobuf-targets.cmake and then not linking utf-8_range.

This should be getting generated in both the binary directory (during a build with EXPORT(TARGETS), and in the install directory (through INSTALL(EXPORT). Generally you do something like sudo apt install protobuf-dev and it will install headers, pkg-config files, an the cmake files.

[mvf]

Generally you do something like sudo apt install protobuf-dev and it will install headers, pkg-config files, an the cmake files.

That's how it should be, but Linux distros don't ship protobuf-config.cmake¹, so find_package(Protobuf CONFIG REQUIRED) is a CMake error when building against system libs.

This Works For Me™:

• Keep using the CMake module. Backwards-compatible and avoids having to port to protobuf_generate.
• On Protobuf >=22, ask pkg-config for the version found by CMake and link that to get abseil & friends.

# No changes here, keep using CMake module + protobuf_generate_*
find_package(Protobuf REQUIRED)
protobuf_generate_cpp(PROTO_SRCS PROTO_HDRS foo.proto bar.proto)

...

# Make it link with recent Protobuf
string(REGEX REPLACE "^[0-9]+\.([0-9]+\.[0-9]+)$" "\\1.0" proto_libver "${Protobuf_VERSION}")
if(proto_libver VERSION_GREATER_EQUAL "22")
    find_package(PkgConfig REQUIRED)
    pkg_check_modules(protobuf REQUIRED IMPORTED_TARGET protobuf=${proto_libver})
    target_link_libraries(myapp PRIVATE PkgConfig::protobuf)
else()
    target_link_libraries(myapp PRIVATE protobuf::libprotobuf)
endif()

The regex converts the Protobuf version from the headers (e.g., 4.24.1) to the library version (e.g., 24.1.0, context).² Linking against the imported targets ensures header/defs/libs consistency, even if CMake and pkg-config somehow manage to find different installations. Make sure to not also use ${Protobuf_INCLUDE_DIRS}/${Protobuf_LIBRARIES} to avoid possible issues.

Footnotes

1. Arch is the only exception I found. ↩

2. This doesn't build with pre-releases of Protobuf since their library versions don't end in .0. ↩

[daynauth]

I have the same problem and ended up downgrading, too. Downgrading fixed the issue, but something tells me that some other issue with protobuf will pop up eventually.

[smod]

Since Google led me here, let me share this: #15604 (comment)

Hope it might help.

[yimingqpa]

abseil-cpp 是一坨狗屎，加在工程里面一堆问题。

[github-actions]

We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please add a comment.

This issue is labeled inactive because the last activity was over 90 days ago.

[YanzhaoW]

The current version can't work with CMake's own FindProtobuf.cmake module script, which many old projects are still relying on. I can't find any solution other than using older Protobuf version, such as v3.21.12.

[fwsGonzo]

This is still broken. It's still "impossible" to use static linking with protobuf using official package.

[jonringer]

This is still broken. It's still "impossible" to use static linking with protobuf using official package.

Unfortunately this is kind of awkward with CMake in general. One potential solution would be to honor the FindProtobuf.cmake's usage of Protobuf_USE_STATIC_LIBS. Then in the cmake config file, it could inspect if that is set, and expose the right alias:

cmake .. -DProtobuf_USE_STATIC_LIBS
...
# CMakeLists.txt
find_package(Protobuf CONFIG REQUIRED) # protobuf::libprotobuf is actually static

Another option would be to just expose a _static variant

find_package(Protobuf REQUIRED)
target_link_libraries(myLib PUBLIC protobuf::libprotobuf_static)

[Patrick-ICT]

Makes sense! Can the point of this issue be to add this to the documentation somewhere? I didn't see much docs on using protobuffs with cmake.

From chat:

When you use find_package(protobuf REQUIRED), CMake will first look for a Findprotobuf.cmake module, which is a script provided by the CMake package or the user. If it doesn't find this module, it will fall back to using the package configuration file (protobuf-config.cmake), which is generated by the package itself when it's built and installed. The Findprotobuf.cmake module might not handle the Abseil dependency correctly, causing the issues you encountered.
By using find_package(protobuf REQUIRED CONFIG), you explicitly tell CMake to use the package configuration file (protobuf-config.cmake) instead of the Findprotobuf.cmake module. This file should handle the Abseil dependency correctly, making it easier to link and use Protobuf in your project.

Hi, I changed the find_package to find_package(protobuf REQUIRED CONFIG), however I ran into this problem while cmake:

Which says the protobuf-targets.cmake file is missing. I checked the corresponding directory as follows, and it is not there, but I did not see any errors while building the protobuf.

Is there any solution?

[jonringer]

I think you truncated some of the output, can you copy it into a gist and past the link?

[Patrick-ICT]

Sorry, I have switched to an older version of protobuf, and the error is gone now. Thanks for the reply.

[github-actions]

We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please add a comment.

This issue is labeled inactive because the last activity was over 90 days ago. This issue will be closed and archived after 14 additional days without activity.

[github-actions]

We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please reopen it.

This issue was closed and archived because there has been no new activity in the 14 days since the inactive label was added.