bug: protoc failed with Unknown flag

[rdettai]

Search before asking

• I had searched in the issues and found no similar issues.

Version

v0.8.61-nightly

What's Wrong?

It seems the version of protoc I get on buster when I run ./scripts/setup/dev_setup.sh is not the right one because I get the following build error:

#16 144.9   Running: "/usr/bin/protoc" "--include_imports" "--include_source_info" "-o" "/tmp/prost-buildpyg2xO/prost-descriptor-set" "-I" "/databend/src/meta/protos/proto" "--experimental_allow_proto3_optional" "/databend/src/meta/protos/proto/database.proto" "/databend/src/meta/protos/proto/user.proto" "/databend/src/meta/protos/proto/config.proto" "/databend/src/meta/protos/proto/share.proto" "/databend/src/meta/protos/proto/datatype.proto" "/databend/src/meta/protos/proto/metadata.proto" "/databend/src/meta/protos/proto/table.proto"
#16 144.9 
#16 144.9   --- stderr
#16 144.9   Error: Custom { kind: Other, error: "protoc failed: Unknown flag: --experimental_allow_proto3_optional\n" }

How to Reproduce?

Container environment:

ARG DATABEND_VERSION=v0.8.61-nightly

FROM rust:1.64-buster as build
ARG DATABEND_VERSION

RUN git clone --depth 1 --branch ${DATABEND_VERSION} https://github.com/datafuselabs/databend.git 
WORKDIR /databend
RUN ./scripts/setup/dev_setup.sh -y
RUN cargo build --bin=databend-query --bin=databend-meta --bin=databend-metactl --release

Are you willing to submit PR?

• Yes I am willing to submit a PR!

[rdettai]

Is seems that this flag was introduced in protoc 3.12 and exited "experimental" mode with 3.15 (google/protobuf-gradle-plugin#415 (comment)). Don't know when it will stop working with the experimental_ prefix 😖

[rdettai]

In all cases, a good approach would be to pin a version as described in the official docs: https://grpc.io/docs/protoc-installation/#install-pre-compiled-binaries-any-os, but it will annoying to make it architecture and OS independent

[Xuanwo]

We need to do the following things:

• Remove the experimental prefix for protoc.
• Check the protoc version and give a friendly output to users.

Cc @drmingdrmer for comments.

[Xuanwo]

In all cases, a good approach would be to pin a version as described in the official docs: https://grpc.io/docs/protoc-installation/#install-pre-compiled-binaries-any-os, but it will annoying to make it architecture and OS independent

In PR #8047, we will check the protoc version during the build. Thus, the databend only needs to specify the oldest version to build.

For now, all protoc that >= 3.12.0 can be used to build.

[rdettai]

I think that is good enough, as more recent distrib versions (for instance debian bullseye) have protoc>3.12.0 in their package manager. Closing this for now. Thanks for your quick fix!