Consider Buf, Submodules and / or Subtrees for proto files

[gotjoshua]

Summary

Clear separation of the protofiles from both regen-ledger and regen-js is needed to streamline updating and syncronizing.

Problem Definition

I think that buf says it best:

Dependency management is usually an afterthought: Protobuf files are vendored manually, with an error-prone copy-and-paste process from GitHub repositories.

Proposal

• Separate the third-party and main proto files into a separate repo "regen-proto"
• Either merge third-party or nest it into the top level of regen-proto
• Import regen-proto into regen-ledger and regen-js
  • Adapt gen scripts as needed so that both ts-proto and the go proto gen work with the new setup

Precise methods need to be considered...

---

For Admin Use

• Not duplicate issue
• Appropriate labels applied
• Appropriate contributors tagged
• Contributor assigned/self-assigned

[blushi]

One idea from @aaronc is to use Buf Schema Registry for proto files https://docs.buf.build/bsr/introduction along with scripts for generating typescript code.

[gotjoshua]

Buf Schema Registry for proto files

buf definately looks nice.
aaand, my main concern is that it smells like vendor lock-in and more tooling.
of course, it may be that it will be similar vendor lock-in to npm, if buf becomes the next big thing...

for sure worth considering, but if a simple git based solution can work, it may be a better choice.

[robert-zaremba]

buf is kind of a standard for Protobuf. Same as the proto tool itself. I wouldn't consider it vendor lock in.

[aaronc]

I'm in the process of setting up BSR for all the SDK types. Once that's done we can do the same for Regen

[ryanchristo]

BSR is now set up for both Cosmos SDK and Regen Ledger:

• https://buf.build/cosmos/cosmos-sdk
• https://buf.build/regen/regen-ledger

The proto files for each repository are updated on every push to master.

BSR makes it easy to import generated code using templates. For example, you can import generated golang code for the regen-ledger proto files with the following:

go get go.buf.build/protocolbuffers/go/regen/regen-ledger

We are not currently using this method within the regen-ledger repository, which would slow down our development process, but this might be a suitable path for importing generated typescript code in regen-js (regen-network/regen-js#4).

Whether this method is used or a separate script is created within the regen-js repository, there is now a single source of truth for both the cosmos-sdk and regen-ledger proto files and the need to copy-and-paste the proto files is no longer an issue.

Thanks for opening this issue @gotjoshua. I think it's safe to close now but feel free to reopen if not.