Root capnp import not added to generated file unless struct or interface keyword is present

[lthibault]

The following schema file is compiled without the "capnproto.org/go/capnp/v3" import statement

using Go = import "/go.capnp";

@0xdf0f779c8b0574be;

$Go.package("bn");
$Go.import("github.com/blocknative/bn/internal/api/bn");


enum Region {
    unknown               @0;
    usEast1               @1;
    euCentral1            @2;
    apSoutheast1          @3;
}

Adding struct Foo{} anywhere in the file triggers the inclusion of the missing import.

[jared-bounti]

Confirming this bug

@0xf1bfe9b43f32d8ab;

using Go = import "/go.capnp";
$Go.package("citypes");
$Go.import("github.com/myorg/proto/mytypes");

enum MyEnumType {
  foo @0;
  bar @1;
  baz @2;
}

import (
	schemas "capnproto.org/go/capnp/v3/schemas"
)

Compared to my proto files which have structs:

import (
	capnp "capnproto.org/go/capnp/v3"
	text "capnproto.org/go/capnp/v3/encoding/text" // not expected in my simple enum file
	schemas "capnproto.org/go/capnp/v3/schemas"
)

However the compiled Go code will utilize capnp without the import

type MyEnumType_List = capnp.EnumList[MyEnumType]

func NewMyEnumType_List(s *capnp.Segment, sz int32) (MyEnumType_List, error) {
	return capnp.NewEnumList[MyEnumType](s, sz)
}

[zenhack]

I think we should simplify this and just have the imports be inserted unconditionally; rather than try to detect whether they're needed, we should just add some dummy assignments to silence unused import warnings in case they aren't:

var (
    _ = capnp.Struct{}
    _ = text.Encoder{}
    // ...
)

iirc this is what protobuf does.

[jared-bounti]

The v1.0 serializer always adds the capnp import,

I thought about doing the same here in a fork, but wasn't sure if there were cases where the capnp import is unnecessary (EDIT: there are cases where adding the import will cause non-compiling Go code)

Code from the 1.0 Serializer

		fprintf(file, "import (\n")
		fprintf(file, "C \"%s\"\n", go_capnproto_import)

https://github.com/glycerine/go-capnproto/blob/master/capnpc-go/capnpc-go.go#L1038-L1042

The logic of generating the importList seems to suggest capnp is always used, but it wouldn't hurt to force it to be used in the func (g *generator) generate() []byte { method if you're looking for a low-effort fix.

[jared-bounti]

For anyone in need of a workaround, I've added this to my justfile / makefile

  @sd "import \(" "import (\\n\\tcapnp \"capnproto.org/go/capnp/v3\"" $(find . -type f -iname "*.go" -exec grep -L "capnp \"capnproto.org/go/capnp/v3\"" {} \+)

[lthibault]

@jared-bounti Could I convince you to open a PR for this? 🙂

[jared-bounti]

@jared-bounti Could I convince you to open a PR for this? 🙂

I've spent some time looking at it and unfortunately it isn't so simple.

Adding the capnp import , while great for most files, does cause tests to fail on at least four files with their import specs in regen.sh. As an example, a file with only const declarations has no use for the capnp import. Which would then cause non-compiling Go code.

As for where the capnp import is failing to be added for enum types... I've tried tracing it at least half a dozen times and not found the bug.

@0xe41c262b5e08249d;

using Go = import "/go.capnp";
$Go.package("types");
$Go.import("github.com/myorg/proto/types");

const secret :Data = 0x"9f98739c2b53835e 6720a00907abd42f";

Thus my local workaround above would need another pipe to grep Enum to prevent adding the unnecessary import on const-only files. But we don't have any of those over here yet 😄

[zenhack]

Quoting Jared (2022-10-17 14:17:01)

a file with only const declarations has no use for the capnp import. Which would then cause non-compiling Go code.

Yeah, this is why I suggested adding dummy uses of the module to silence these errors. This avoids the need to worry about whether or not the import is otherwise used.