Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: grpc-only mode #11430

Merged
merged 16 commits into from
Mar 25, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,8 @@ Ref: https://keepachangelog.com/en/1.0.0/

### Features

* [\#11430](https://github.com/cosmos/cosmos-sdk/pull/11430) Introduce a new `grpc-only` flag, such that when enabled, will start the node in a query-only mode. Note, gRPC MUST be enabled with this flag.
alexanderbez marked this conversation as resolved.
Show resolved Hide resolved
* (x/upgrade) [\#11116](https://github.com/cosmos/cosmos-sdk/pull/11116) `MsgSoftwareUpgrade` and has been added to support v1beta2 msgs-based gov proposals.
* (x/bank) [\#11417](https://github.com/cosmos/cosmos-sdk/pull/11417) Introduce a new `SpendableBalances` gRPC query that retrieves an account's total (paginated) spendable balances.
* [\#11441](https://github.com/cosmos/cosmos-sdk/pull/11441) Added a new method, `IsLTE`, for `types.Coin`. This method is used to check if a `types.Coin` is less than or equal to another `types.Coin`.
* (x/upgrade) [\#11116](https://github.com/cosmos/cosmos-sdk/pull/11116) `MsgSoftwareUpgrade` and has been added to support v1beta2 msgs-based gov proposals.
Expand Down
106 changes: 65 additions & 41 deletions server/start.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ import (
"github.com/tendermint/tendermint/abci/server"
tcmd "github.com/tendermint/tendermint/cmd/tendermint/commands"
tmos "github.com/tendermint/tendermint/libs/os"
tmservice "github.com/tendermint/tendermint/libs/service"
"github.com/tendermint/tendermint/node"
"github.com/tendermint/tendermint/rpc/client/local"
tmtypes "github.com/tendermint/tendermint/types"
Expand All @@ -33,8 +34,8 @@ import (
storetypes "github.com/cosmos/cosmos-sdk/store/types"
)

// Tendermint full-node start flags
const (
// Tendermint full-node start flags
flagWithTendermint = "with-tendermint"
flagAddress = "address"
flagTransport = "transport"
Expand All @@ -53,22 +54,19 @@ const (
FlagPruningInterval = "pruning-interval"
FlagIndexEvents = "index-events"
FlagMinRetainBlocks = "min-retain-blocks"
)

// GRPC-related flags.
const (
// state sync-related flags
FlagStateSyncSnapshotInterval = "state-sync.snapshot-interval"
FlagStateSyncSnapshotKeepRecent = "state-sync.snapshot-keep-recent"

// gRPC-related flags
flagGRPCOnly = "grpc-only"
flagGRPCEnable = "grpc.enable"
flagGRPCAddress = "grpc.address"
flagGRPCWebEnable = "grpc-web.enable"
flagGRPCWebAddress = "grpc-web.address"
)

// State sync-related flags.
const (
FlagStateSyncSnapshotInterval = "state-sync.snapshot-interval"
FlagStateSyncSnapshotKeepRecent = "state-sync.snapshot-keep-recent"
)

// StartCmd runs the service passed in, either stand-alone or in-process with
// Tendermint.
func StartCmd(appCreator types.AppCreator, defaultNodeHome string) *cobra.Command {
Expand Down Expand Up @@ -96,6 +94,11 @@ will not be able to commit subsequent blocks.

For profiling and benchmarking purposes, CPU profiling can be enabled via the '--cpu-profile' flag
which accepts a path for the resulting pprof file.

The node may be started in a 'query only' mode where only the gRPC and JSON HTTP
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are the queries from Tendermint supported in this mode? Stuff like block?

Copy link
Contributor Author

@alexanderbez alexanderbez Mar 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, this is gRPC only mode. The doc and implementation states Tendermint is disabled (i.e. not started). Tendermint cannot be supported.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it sync new state though?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, because again, Tendermint is not running. This just serves existing data via gRPC.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahhh yeah, thank you & sorry for the dumb question. 😁

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No such thing :p

API services are enabled via the 'grpc-only' flag. In this mode, Tendermint is
bypassed and can be used when legacy queries are needed after an on-chain upgrade
is performed. Note, when enabled, gRPC will also be automatically enabled.
`,
PreRunE: func(cmd *cobra.Command, _ []string) error {
serverCtx := GetServerContextFromCmd(cmd)
Expand All @@ -120,8 +123,6 @@ which accepts a path for the resulting pprof file.
return startStandAlone(serverCtx, appCreator)
}

serverCtx.Logger.Info("starting ABCI with Tendermint")

// amino is needed here for backwards compatibility of REST routes
err = startInProcess(serverCtx, clientCtx, appCreator)
errCode, ok := err.(ErrorCode)
Expand Down Expand Up @@ -152,6 +153,7 @@ which accepts a path for the resulting pprof file.
cmd.Flags().Uint(FlagInvCheckPeriod, 0, "Assert registered invariants every N blocks")
cmd.Flags().Uint64(FlagMinRetainBlocks, 0, "Minimum block height offset during ABCI commit to prune Tendermint blocks")

cmd.Flags().Bool(flagGRPCOnly, false, "Start the node in gRPC query only mode (no Tendermint process is started)")
cmd.Flags().Bool(flagGRPCEnable, true, "Define if the gRPC server should be enabled")
cmd.Flags().String(flagGRPCAddress, config.DefaultGRPCAddress, "the gRPC server address to listen on")

Expand Down Expand Up @@ -206,7 +208,6 @@ func startStandAlone(ctx *Context, appCreator types.AppCreator) error {
return WaitForQuitSignals()
}

// legacyAminoCdc is used for the legacy REST API
func startInProcess(ctx *Context, clientCtx client.Context, appCreator types.AppCreator) error {
cfg := ctx.Config
home := cfg.RootDir
Expand Down Expand Up @@ -253,34 +254,40 @@ func startInProcess(ctx *Context, clientCtx client.Context, appCreator types.App
return err
}

tmNode, err := node.New(
cfg,
ctx.Logger,
abciclient.NewLocalCreator(app),
genDoc,
var (
tmNode tmservice.Service
gRPCOnly = ctx.Viper.GetBool(flagGRPCOnly)
)
if err != nil {
return err
}
if gRPCOnly {
ctx.Logger.Info("starting node in gRPC only mode; Tendermint is disabled")
config.GRPC.Enable = true
} else {
ctx.Logger.Info("starting node with ABCI Tendermint in-process")

ctx.Logger.Debug("initialization: tmNode created")
if err := tmNode.Start(); err != nil {
return err
tmNode, err = node.New(cfg, ctx.Logger, abciclient.NewLocalCreator(app), genDoc)
if err != nil {
return err
}

if err := tmNode.Start(); err != nil {
return err
}
}
ctx.Logger.Debug("initialization: tmNode started")

// Add the tx service to the gRPC router. We only need to register this
// service if API or gRPC is enabled, and avoid doing so in the general
// case, because it spawns a new local tendermint RPC client.
if config.API.Enable || config.GRPC.Enable {
if (config.API.Enable || config.GRPC.Enable) && tmNode != nil {
node, ok := tmNode.(local.NodeService)
if !ok {
panic("unable to set node type. Please try reinstalling the binary.")
return fmt.Errorf("unable to set node type; please try re-installing the binary")
}

localNode, err := local.New(node)
if err != nil {
panic(err)
return err
}

clientCtx = clientCtx.WithClient(localNode)

app.RegisterTxService(clientCtx)
Expand All @@ -294,16 +301,16 @@ func startInProcess(ctx *Context, clientCtx client.Context, appCreator types.App
return err
}

clientCtx := clientCtx.
WithHomeDir(home).
WithChainID(genDoc.ChainID)
clientCtx := clientCtx.WithHomeDir(home).WithChainID(genDoc.ChainID)

if config.GRPC.Enable {
_, port, err := net.SplitHostPort(config.GRPC.Address)
if err != nil {
return err
}

grpcAddress := fmt.Sprintf("127.0.0.1:%s", port)

// If grpc is enabled, configure grpc client for grpc gateway.
grpcClient, err := grpc.Dial(
grpcAddress,
Expand All @@ -313,6 +320,7 @@ func startInProcess(ctx *Context, clientCtx client.Context, appCreator types.App
if err != nil {
return err
}

clientCtx = clientCtx.WithGRPCClient(grpcClient)
ctx.Logger.Debug("grpc client assigned to client context", "target", grpcAddress)
}
Expand All @@ -330,6 +338,7 @@ func startInProcess(ctx *Context, clientCtx client.Context, appCreator types.App
select {
case err := <-errCh:
return err

case <-time.After(types.ServerStartTime): // assume server started successfully
}
}
Expand All @@ -338,11 +347,13 @@ func startInProcess(ctx *Context, clientCtx client.Context, appCreator types.App
grpcSrv *grpc.Server
grpcWebSrv *http.Server
)

if config.GRPC.Enable {
grpcSrv, err = servergrpc.StartGRPCServer(clientCtx, app, config.GRPC.Address)
if err != nil {
return err
}

if config.GRPCWeb.Enable {
grpcWebSrv, err = servergrpc.StartGRPCWeb(grpcSrv, config)
if err != nil {
Expand All @@ -352,28 +363,40 @@ func startInProcess(ctx *Context, clientCtx client.Context, appCreator types.App
}
}

// At this point it is safe to block the process if we're in gRPC only mode as
// we do not need to start Rosetta or handle any Tendermint related processes.
if gRPCOnly {
// wait for signal capture and gracefully return
return WaitForQuitSignals()
}

var rosettaSrv crgserver.Server
if config.Rosetta.Enable {
offlineMode := config.Rosetta.Offline
if !config.GRPC.Enable { // If GRPC is not enabled rosetta cannot work in online mode, so it works in offline mode.

// If GRPC is not enabled rosetta cannot work in online mode, so it works in
// offline mode.
if !config.GRPC.Enable {
offlineMode = true
}

conf := &rosetta.Config{
Blockchain: config.Rosetta.Blockchain,
Network: config.Rosetta.Network,
TendermintRPC: ctx.Config.RPC.ListenAddress,
GRPCEndpoint: config.GRPC.Address,
Addr: config.Rosetta.Address,
Retries: config.Rosetta.Retries,
Offline: offlineMode,
Blockchain: config.Rosetta.Blockchain,
Network: config.Rosetta.Network,
TendermintRPC: ctx.Config.RPC.ListenAddress,
GRPCEndpoint: config.GRPC.Address,
Addr: config.Rosetta.Address,
Retries: config.Rosetta.Retries,
Offline: offlineMode,
Codec: clientCtx.Codec.(*codec.ProtoCodec),
InterfaceRegistry: clientCtx.InterfaceRegistry,
}
conf.WithCodec(clientCtx.InterfaceRegistry, clientCtx.Codec.(*codec.ProtoCodec))

rosettaSrv, err = rosetta.ServerFromConfig(conf)
if err != nil {
return err
}

errCh := make(chan error)
go func() {
if err := rosettaSrv.Start(); err != nil {
Expand All @@ -384,6 +407,7 @@ func startInProcess(ctx *Context, clientCtx client.Context, appCreator types.App
select {
case err := <-errCh:
return err

case <-time.After(types.ServerStartTime): // assume server started successfully
}
}
Expand Down Expand Up @@ -411,6 +435,6 @@ func startInProcess(ctx *Context, clientCtx client.Context, appCreator types.App
ctx.Logger.Info("exiting...")
}()

// Wait for SIGINT or SIGTERM signal
// wait for signal capture and gracefully return
robert-zaremba marked this conversation as resolved.
Show resolved Hide resolved
return WaitForQuitSignals()
}