protobuf-cpp: Cannot compile when there are alot of proto files in separate directories

[Chi-EEE]

Xmake Version

2.8.5

Operating System Version and Architecture

Windows 10 22H2

Describe Bug

Xmake only compiles 3/5 directories containing proto files using protobuf-cpp.

Expected Behavior

All the proto files in the directories should get compiled

Project Configuration

The repository which I'm working on is https://github.com/cartographer-project/cartographer

add_rules("mode.debug", "mode.release")

add_requires("boost", {configs = {iostreams = true}})
add_requires("ceres-solver", {configs = {suitesparse = true}})
add_requires("abseil", "cairo", "eigen", "glog", "lua", "protobuf-cpp")

target("cartographer")
    set_kind("$(kind)")
    set_languages("cxx17")

    add_packages(
        "abseil",
        "boost", 
        "ceres-solver", 
        "cairo", 
        "eigen",
        "glog",
        "lua"
    )
    
    add_packages("protobuf-cpp", {public = true})
    add_rules("protobuf.cpp")

    add_files("cartographer/**.proto", {proto_rootdir = "cartographer", proto_autogendir = path.join(os.projectdir(), "build", "proto") , proto_public = true})
    add_headerfiles("$(buildir)/proto/**.h")
    add_includedirs("$(buildir)/proto")

    remove_files("cartographer/**_service.proto")

    add_headerfiles("(cartographer/**.h)")
    add_files("cartographer/**.cc")

    remove_files("cartographer/io/serialization_format_migration.cc")
    remove_headerfiles("cartographer/io/serialization_format_migration.h")

    remove_headerfiles("**/fake_*.h", "**/*test_helpers*.h", "**/mock*.h")
    remove_files("**/fake_*.cc", "**/*test_helpers*.cc", "**/mock*.cc", "**/*_main.cc", "**/*_test.cc")

    -- BUILD_GRPC is not enabled
    remove_headerfiles("cartographer/cloud/**.h")
    remove_files("cartographer/cloud/**.cc")
    remove_files("cartographer/cloud/proto/**.proto")

    add_includedirs(".")

Additional Information and Error Logs

Error Log: https://pastebin.com/6DWxE6n7

[Chi-EEE]

When I debugged Xmake, I saw that 10/44 proto files managed to run the function inside of this call: https://github.com/xmake-io/xmake/blob/master/xmake/rules/protobuf/proto.lua#L195

[waruqi]

xmake tries to resolve the dependencies between proto files and then generates a dependency graph, compiling as much as possible in parallel.

However, xmake currently parses the dependencies from the source code itself, and if it does not generate them correctly, the parallel compilation may fail.

xmake/xmake/rules/protobuf/proto.lua

Line 186 in 8eefa03

local moduledeps, moduledeps_files = module_parser.load(target, sourcebatch, table.join(opt, {proto_rootdir = proto_rootdir}))

So you can try debugging this implementation of dependency resolution.

[Chi-EEE]

Fixed the issue by doing the following:

• Setting the proto_rootdir to the directory where the proto files were: cartographer
• Doing: protobuf.cpp规则导出.h文件 #3678
• Set proto_public to true
• set add_includedirs to the proto_autogendir
• And changed the import inside of the proto files. Example: import "cartographer/..." to import "..."

[Chi-EEE]

The above changes only fixed linux, not windows