datastore: protobuf namespace conflict with google.golang.org/appengine/v2

[sg0hsmt]

Client

datastore

Environment

Go executable on Linux

Go Environment

$ go version
go version go1.22.3 linux/amd64

$ go env
GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/home/vscode/.cache/go-build'
GOENV='/home/vscode/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='local'
GOTOOLDIR='/usr/local/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.22.3'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/workspaces/namespace-conflict/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build3064827702=/tmp/go-build -gno-record-gcc-switches'

Code

package main

import (
	"fmt"

	_ "cloud.google.com/go/datastore"
	_ "google.golang.org/appengine/v2"
)

func main() {
	fmt.Println("Hello, World!")
}

Expected behavior

Hello, World! will be displayed.

Actual behavior

A panic will occur.

$ go run main.go 
panic: proto: file "datastore_v3.proto" is already registered
        previously from: "cloud.google.com/go/datastore/internal/gaepb"
        currently from:  "google.golang.org/appengine/v2/internal/datastore"
See https://protobuf.dev/reference/go/faq#namespace-conflict


goroutine 1 [running]:
google.golang.org/protobuf/reflect/protoregistry.init.func1({0x101aa60?, 0xabafe0?}, {0xba6a80, 0xc000036ee0})
        /go/pkg/mod/google.golang.org/[email protected]/reflect/protoregistry/registry.go:56 +0x1ec
google.golang.org/protobuf/reflect/protoregistry.(*Files).RegisterFile(0xc0000122e8, {0xbb48a0, 0xc00019ee00})
        /go/pkg/mod/google.golang.org/[email protected]/reflect/protoregistry/registry.go:130 +0xbc3
google.golang.org/protobuf/internal/filedesc.Builder.Build({{0x97faf0, 0x31}, {0xff81e0, 0x2d00, 0x2d00}, 0xc, 0x37, 0x0, 0x0, {0xbab380, ...}, ...})
        /go/pkg/mod/google.golang.org/[email protected]/internal/filedesc/build.go:112 +0x1d6
google.golang.org/protobuf/internal/filetype.Builder.Build({{{0x97faf0, 0x31}, {0xff81e0, 0x2d00, 0x2d00}, 0xc, 0x37, 0x0, 0x0, {0x0, ...}, ...}, ...})
        /go/pkg/mod/google.golang.org/[email protected]/internal/filetype/build.go:138 +0x17d
google.golang.org/appengine/v2/internal/datastore.file_datastore_v3_proto_init()
        /go/pkg/mod/google.golang.org/appengine/[email protected]/internal/datastore/datastore_v3.pb.go:6420 +0x1b8
google.golang.org/appengine/v2/internal/datastore.init.0()
        /go/pkg/mod/google.golang.org/appengine/[email protected]/internal/datastore/datastore_v3.pb.go:5739 +0xf
exit status 2

Additional context

Importing google.golang.org/appengine/v2 with datastore v1.16.0 or later causes a protobuf namespace conflict.

related: #7760, #9744

[noahdietz]

@sg0hsmt thanks for reporting this. It looks like @codyoss has a potential fix out, but it has some risks. We will continue to consider that, but in the meantime, the Protobuf documentation has some workarounds at the application level, but runtime and build time mitigations: https://protobuf.dev/reference/go/faq/#fix-namespace-conflict

Can you try one of those? Will that work for the time being? Thanks.

[noahdietz]

We are going to move forward with Cody's change to deal with the issue.