Repo structure and binaries' improvements

[moul]

Please note that the comments should be read. The body of this issue is not yet updated with the latest decisions.

This issue seeks to clarify concepts and suggest future updates, related to #1105, where I proposed a gnosdk space.

The gno.land/cmd/gno and gnosdk should focus on user experience, while the rest of gno.land, gnosdk, and tm2 should showcase standalone binaries and minimalist libraries.

The main binary, gno.land/cmd/gno, should be the developers' go-to binary in the ecosystem, catering to both developers and non-developers. Considering gno.land's significant role with github-style realms, IBC, and ICS features, it's fitting to house the ecosystem binary, gno.land/cmd/gno, here, even if it extends beyond gno.land.

gnoland, designed for production runtime management and daemon operation, should remain minimalist, focusing on node initiation and network joining, leaving extensive configuration and management tools to gnosdk or gno under a node/genesis subcommand.

The fooland binary, aligning with other cosmos chains, serves as a central hub, combining standalone libraries/binaries into one. Once established, the secondary binary, "gno," becomes necessary, allowing "fooland" to focus on unique features and enabling gnolang developers to use standard ecosystem tools.

I support including basic account management and client functionality in the gno binary, reducing the need for gnokey, except for demonstrating a minimal client for a new chain.

Proposal for structural target:

In bold, the things that are not existing yet.

• gno.land
  • pkg
    • gnoland: a node app for gno.land
    • ...
  • cmd
    • gnokey: a standalone minimal client for gno.land
      • query, maketx, etc. (client functionality)
      • list, create, etc. (account management)
    • gnoland: a standalone minimal production node for gno.land
      • start
      • reset (WIP)
    • gnoweb: a standalone minimal production gnoweb for gno.land
    • gno: the main binary for the gno ecosystem (moved from gnovm/cmd/)
      • test, build, doc, etc.: Go-like commands for gno; I suggest we switch from gno.Machine to gnovm.Keeper
      • genesis: tools for manipulating the genesis of gno.land
      • node {backup,restore}: new place for gnotxport
      • node stats,...: other helpers related to node management
      • dev: an opinionated shortcut to run a single gnoland node + gnoweb in memory/disk
• gnovm
  • pkg
    • gnolang: current gnolang/gno.Machine
    • commands: composable CLI commands, equivalent of tm2/keys/client
  • cmd
    • gnolang: new binary with blockchain-less gno.Machine commands (most of today's gno binary)
• gnosdk
  • pkg
    • genesis: a library for manipulating the genesis
    • codegen: helpers for code generation in the gno ecosystem
  • cmd
    • fooland: a unified library with gnokey, gnoland, gnotxport, etc. to run a foo.land node or interact with it
      • query, maketx: gnokey's client functionality
      • list, create: gnokey's account management
      • node
        • start: start gnoland
        • txport: backup and restore functionality from gnotxport to gno node
        • genesis-flows-such-as-POS-coordination
      • web: gnoweb
      • faucet: gnofaucet
• tm2
  • pkg
    • ...

[moul]

Based on today's discussion, I have officially relocated the main binary, gno, to gno.land/. This move positions it as the ecosystem's primary binary.

Additionally, I have introduced a new binary called gnolang. Similar to gnokey, it is a minimal composable binary along with its library. This allows for the composition of binaries for various purposes. Importantly, it enables us to maintain the independence and testability of gnovm without any external dependencies.

To clarify, while I personally prefer using only the gno binary, we can propose a way for forkers and builders to quickly get started with the gnolang binary. Meanwhile, we can continue developing integration tests and other components in gnovm/ that are separate from the rest of the project.

Edit: bonus, what do you think about keeping the structure mentioned above, but also.... create a top-level main.go file to install the gno binary with go install github.com/gnolang/gno?

[thehowl]

Edit: bonus, what do you think about keeping the structure mentioned above, but also.... create a top-level main.go file to install the gno binary with go install github.com/gnolang/gno?

I like this!

---

The OP also looks good to me. A few further proposals/q's:

• Remove /pkg/ from tm2 (unless we plan to add commands to tm2)
• Q: gnosdk/pkg/codegen should also contain, for instance, [META] improved native bindings #814's genstd?
• Q: when should I use GnoVM and when should I use GnoSDK? Would feat: add gno.land/pkg/gnoclient (Gno.land Go client) #1047 (gnoclient) go in GnoSDK?
• Q: who is the user of fooland, why do they use it, how do they use it?

[jaekwon]

GNO:

• Invariant: gno must be the language, not the whole project.
• In our directory structure, github.com/gnolang/gno/gnovm/pkg/gnolang should instead be
  github.com/gnolang/gno/gnolang/pkg/gnovm. the vm is inside the implementation of the language.
  gno should only live in github.com/gnolang/gno/gnolang/cmd/gno.
• gnokey is available as gno key for convenience, but gno is not blockchain aware.

GNOLAND:

• THE SINGULAR STANDARD production node for gno.land.
• Is minimal according to our philosophy

GNOKEY:

• Minimal console key/wallet for gnolang.

And I would suggest renaming GNOSDK to GNOLANDTOOLS, for any chain based on the GNO VM.

[thehowl]

@jaekwon

• Why gnolandtools if the chains are not necessarily relating to "land"?
• If we're switching names up, I propose using github.com/gnolang/gno/lang/pkg/gno (so current gnovm -> lang, pkg/gnolang -> gno). This is both to reduce the number of times we write "gno" in the import path from 4 to 3, as well as to match the dir name of the package to the actual package name
• Does gno key then just run gnokey with os/exec?

[jaekwon]

• yes gnolandtools gno*tools or gnochaintools, anything that solves your problem would be better, but why not start this way i wonder.
• ...

[moul]

I'm not sure if I remember everything we discussed during the call, but here are the key points:

• We should adhere to the Unix philosophy of using multiple small and dedicated binaries.
• We should strive to avoid having multiple methods for accomplishing the same task, particularly in documentation writing.
• Due to reasons like trademark, fork-friendliness, and others, we should keep gno within gnovm/ and avoid incorporating blockchain features.
• We can consider implementing a feature where "gno " checks if "gno-subcommand" exists in $PATH.
• gnoland, gno, gnokey, etc., should remain minimal. However, this doesn't mean we can't have helpers and flows. Let's create a top-level folder called "gnolandtools" (I dislike the long name) for things like creating flows using multiple Unix-style commands or including functionalities that we don't want to include in the minimal binaries. Essentially, it's similar to what I previously referred to as "gnosdk," but with a new name. For example:
  • "gno dev" (META 'gno' CLI (DevX discussion) #1017, feat: Use docker-compose for local transaction replication #1238, feat: add local dev setup script #1237)
  • gno mod-publish (gno mod publish, and automatic loading of examples/* on gnoland's startup #743)
    • question: should we move gno mod into its own git-style subbinary? I think so.
  • or the PoS coordination tools that @albttx wants to add.

---

Edit: If it's not gnosdk/ and since I don't like gnolandtools, what are your thoughts on using github.com/gnolang/tools/gno{,land}-xxx instead? In this case, xxx represents a folder that contains a specific tool, such as tools/gno-mod, tools/gno-dev, tools/gno-publish, and tools/gnoland-monitoring.

[moul]

See #1256

[thehowl]

So:

• Work in a direction of having small binaries that "do one thing and do it well"
• When cmd/gno does not recognize a sub command, try to find gno$command in path. (allows for gno key)
• Commands can depend on others, by exec'ing them.
• Features like gno dev can be implemented as a gnodev binary that merges things together.
• The contrib directory contains "community contributions", and as such many useful commands and extensions to gno which are useful for development (but not part of the "core" gno binaries).

Is this roughly our current position? @moul

I think we should have only one of misc or contrib.

[moul]

Is this roughly our current position? @moul

Yeah, that’s the idea.

For the Git method, let's hold off and have users specifically use "gnodev" rather than "gno dev." We might consider enabling "gno$command" execution later on.

I think we should have only one of misc or contrib.

I favor flagging 'contribs/gno*/' directories for binaries, similar to '/usr/local/bin', and using 'misc' for everything else, possibly with subdirectories, akin to '/opt' — assuming I've got the Unix standard folders right.

[thehowl]

Added to the Rouen Retreat board. I think a greater discussion on the strategy and writing out a document on what we want for binary/repo structure might be good :)