Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: configurable fastnode (backport #13321) #13337

Merged
merged 6 commits into from
Sep 20, 2022
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,10 +50,16 @@ Ref: https://keepachangelog.com/en/1.0.0/
### Improvements

* [#13233](https://github.com/cosmos/cosmos-sdk/pull/13233) Add `--append` to `add-genesis-account` sub-command to append new tokens after an account is already created.
<<<<<<< HEAD
* (x/group) [#13214](https://github.com/cosmos/cosmos-sdk/pull/13214) Add `withdraw-proposal` command to group module's CLI transaction commands.
* (x/auth) [#13048](https://github.com/cosmos/cosmos-sdk/pull/13048) Add handling of AccountNumberStoreKeyPrefix to the simulation decoder.
* (simapp) [#13108](https://github.com/cosmos/cosmos-sdk/pull/13108) Call `SetIAVLCacheSize` with the configured value in simapp.
* [#13318](https://github.com/cosmos/cosmos-sdk/pull/13318) Keep the balance query endpoint compatible with legacy blocks
=======
* [#13236](https://github.com/cosmos/cosmos-sdk/pull/13236) Integrate Filter Logging
* [#13301](https://github.com/cosmos/cosmos-sdk/pull/13301) Keep the balance query endpoint compatible with legacy blocks
* [#13321](https://github.com/cosmos/cosmos-sdk/pull/13321) Add flag to disable fast node migration and usage.
>>>>>>> 412e2fc86 (feat: configurable fastnode (#13321))

### Bug Fixes

Expand Down
5 changes: 5 additions & 0 deletions baseapp/options.go
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,11 @@ func SetIAVLCacheSize(size int) func(*BaseApp) {
return func(bapp *BaseApp) { bapp.cms.SetIAVLCacheSize(size) }
}

// SetIAVLDisableFastNode enables(false)/disables(true) fast node usage from the IAVL store.
func SetIAVLDisableFastNode(disable bool) func(*BaseApp) {
return func(bapp *BaseApp) { bapp.cms.SetIAVLDisableFastNode(disable) }
}

// SetInterBlockCache provides a BaseApp option function that sets the
// inter-block cache.
func SetInterBlockCache(cache sdk.MultiStorePersistentCache) func(*BaseApp) {
Expand Down
43 changes: 43 additions & 0 deletions fuzz/tests/store_internal_proofs_createnonmembershipproof_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
//go:build gofuzz || go1.18

package tests

import (
"encoding/json"
"testing"

iavlproofs "github.com/cosmos/cosmos-sdk/store/tools/ics23/iavl"
"github.com/cosmos/iavl"
db "github.com/tendermint/tm-db"
)

type serialize struct {
Data map[string][]byte
Key string
}

func FuzzStoreInternalProofsCreateNonmembershipProof(f *testing.F) {
f.Fuzz(func(t *testing.T, data []byte) {
sz := new(serialize)
if err := json.Unmarshal(data, sz); err != nil {
return
}
if len(sz.Data) == 0 || len(sz.Key) == 0 {
return
}
tree, err := iavl.NewMutableTree(db.NewMemDB(), 0, false)
if err != nil {
t.Fatal(err)
}
for k, v := range sz.Data {
tree.Set([]byte(k), v)
}
icp, err := iavlproofs.CreateNonMembershipProof(tree, []byte(sz.Key))
if err != nil {
return
}
if icp == nil {
panic("nil CommitmentProof with nil error")
}
})
}
5 changes: 5 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,12 @@ require (
github.com/cosmos/cosmos-proto v1.0.0-alpha7
github.com/cosmos/cosmos-sdk/db v1.0.0-beta.1
github.com/cosmos/go-bip39 v1.0.0
<<<<<<< HEAD
github.com/cosmos/iavl v0.19.1
=======
github.com/cosmos/gogoproto v1.4.2
github.com/cosmos/iavl v0.19.2-0.20220916140702-9b6be3095313
>>>>>>> 412e2fc86 (feat: configurable fastnode (#13321))
github.com/cosmos/ledger-cosmos-go v0.11.1
github.com/gogo/gateway v1.1.0
github.com/gogo/protobuf v1.3.2
Expand Down
7 changes: 7 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -235,8 +235,15 @@ github.com/cosmos/go-bip39 v1.0.0 h1:pcomnQdrdH22njcAatO0yWojsUnCO3y2tNoV1cb6hHY
github.com/cosmos/go-bip39 v1.0.0/go.mod h1:RNJv0H/pOIVgxw6KS7QeX2a0Uo0aKUlfhZ4xuwvCdJw=
github.com/cosmos/gorocksdb v1.2.0 h1:d0l3jJG8M4hBouIZq0mDUHZ+zjOx044J3nGRskwTb4Y=
github.com/cosmos/gorocksdb v1.2.0/go.mod h1:aaKvKItm514hKfNJpUJXnnOWeBnk2GL4+Qw9NHizILw=
<<<<<<< HEAD
github.com/cosmos/iavl v0.19.1 h1:3gaq9b6SjiB0KBTygRnAvEGml2pQlu1TH8uma5g63Ys=
github.com/cosmos/iavl v0.19.1/go.mod h1:X9PKD3J0iFxdmgNLa7b2LYWdsGd90ToV5cAONApkEPw=
=======
github.com/cosmos/iavl v0.19.2-0.20220916140702-9b6be3095313 h1:R7CnaI/0OLwOusy7n9750n8fqQ3yCQ8OJQI2L3ws9RA=
github.com/cosmos/iavl v0.19.2-0.20220916140702-9b6be3095313/go.mod h1:X9PKD3J0iFxdmgNLa7b2LYWdsGd90ToV5cAONApkEPw=
github.com/cosmos/keyring v1.2.0 h1:8C1lBP9xhImmIabyXW4c3vFjjLiBdGCmfLUfeZlV1Yo=
github.com/cosmos/keyring v1.2.0/go.mod h1:fc+wB5KTk9wQ9sDx0kFXB3A0MaeGHM9AwRStKOQ5vOA=
>>>>>>> 412e2fc86 (feat: configurable fastnode (#13321))
github.com/cosmos/ledger-cosmos-go v0.11.1 h1:9JIYsGnXP613pb2vPjFeMMjBI5lEDsEaF6oYorTy6J4=
github.com/cosmos/ledger-cosmos-go v0.11.1/go.mod h1:J8//BsAGTo3OC/vDLjMRFLW6q0WAaXvHnVc7ZmE8iUY=
github.com/cosmos/ledger-go v0.9.2 h1:Nnao/dLwaVTk1Q5U9THldpUMMXU94BOTWPddSmVB6pI=
Expand Down
46 changes: 26 additions & 20 deletions server/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -81,9 +81,13 @@ type BaseConfig struct {
// IndexEvents defines the set of events in the form {eventType}.{attributeKey},
// which informs Tendermint what to index. If empty, all events will be indexed.
IndexEvents []string `mapstructure:"index-events"`

// IavlCacheSize set the size of the iavl tree cache.
IAVLCacheSize uint64 `mapstructure:"iavl-cache-size"`

// IAVLDisableFastnNode enables or disables the fast sync node.
IAVLDisableFastnNode bool `mapstructure:"iavl-disable-fastnode"`

// AppDBBackend defines the type of Database to use for the application and snapshots databases.
// An empty string indicates that the Tendermint config's DBBackend value should be used.
AppDBBackend string `mapstructure:"app-db-backend"`
Expand Down Expand Up @@ -236,15 +240,16 @@ func (c *Config) GetMinGasPrices() sdk.DecCoins {
func DefaultConfig() *Config {
return &Config{
BaseConfig: BaseConfig{
MinGasPrices: defaultMinGasPrices,
InterBlockCache: true,
Pruning: pruningtypes.PruningOptionDefault,
PruningKeepRecent: "0",
PruningInterval: "0",
MinRetainBlocks: 0,
IndexEvents: make([]string, 0),
IAVLCacheSize: 781250, // 50 MB
AppDBBackend: "",
MinGasPrices: defaultMinGasPrices,
InterBlockCache: true,
Pruning: pruningtypes.PruningOptionDefault,
PruningKeepRecent: "0",
PruningInterval: "0",
MinRetainBlocks: 0,
IndexEvents: make([]string, 0),
IAVLCacheSize: 781250, // 50 MB
IAVLDisableFastnNode: false,
AppDBBackend: "",
},
Telemetry: telemetry.Config{
Enabled: false,
Expand Down Expand Up @@ -306,17 +311,18 @@ func GetConfig(v *viper.Viper) (Config, error) {

return Config{
BaseConfig: BaseConfig{
MinGasPrices: v.GetString("minimum-gas-prices"),
InterBlockCache: v.GetBool("inter-block-cache"),
Pruning: v.GetString("pruning"),
PruningKeepRecent: v.GetString("pruning-keep-recent"),
PruningInterval: v.GetString("pruning-interval"),
HaltHeight: v.GetUint64("halt-height"),
HaltTime: v.GetUint64("halt-time"),
IndexEvents: v.GetStringSlice("index-events"),
MinRetainBlocks: v.GetUint64("min-retain-blocks"),
IAVLCacheSize: v.GetUint64("iavl-cache-size"),
AppDBBackend: v.GetString("app-db-backend"),
MinGasPrices: v.GetString("minimum-gas-prices"),
InterBlockCache: v.GetBool("inter-block-cache"),
Pruning: v.GetString("pruning"),
PruningKeepRecent: v.GetString("pruning-keep-recent"),
PruningInterval: v.GetString("pruning-interval"),
HaltHeight: v.GetUint64("halt-height"),
HaltTime: v.GetUint64("halt-time"),
IndexEvents: v.GetStringSlice("index-events"),
MinRetainBlocks: v.GetUint64("min-retain-blocks"),
IAVLCacheSize: v.GetUint64("iavl-cache-size"),
IAVLDisableFastnNode: v.GetBool("iavl-disable-fastnode"),
AppDBBackend: v.GetString("app-db-backend"),
},
Telemetry: telemetry.Config{
ServiceName: v.GetString("telemetry.service-name"),
Expand Down
4 changes: 4 additions & 0 deletions server/config/toml.go
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,10 @@ index-events = [{{ range .BaseConfig.IndexEvents }}{{ printf "%q, " . }}{{end}}]
# Default cache size is 50mb.
iavl-cache-size = {{ .BaseConfig.IAVLCacheSize }}

# IavlDisableFastnNode enables or disables the fast node feature of IAVL.
# Default is false.
iavl-disable-fastnode = {{ .BaseConfig.IAVLDisableFastnNode }}

# AppDBBackend defines the database backend type to use for the application and snapshots DBs.
# An empty string indicates that a fallback will be used.
# First fallback is the deprecated compile-time types.DBBackend value.
Expand Down
4 changes: 4 additions & 0 deletions server/mock/store.go
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,10 @@ func (ms multiStore) SetIAVLCacheSize(size int) {
panic("not implemented")
}

func (ms multiStore) SetIAVLDisableFastNode(disable bool) {
panic("not implemented")
}

func (ms multiStore) SetInitialVersion(version int64) error {
panic("not implemented")
}
Expand Down
1 change: 1 addition & 0 deletions server/start.go
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ const (
FlagIndexEvents = "index-events"
FlagMinRetainBlocks = "min-retain-blocks"
FlagIAVLCacheSize = "iavl-cache-size"
FlagIAVLFastNode = "iavl-disable-fastnode"

// state sync-related flags
FlagStateSyncSnapshotInterval = "state-sync.snapshot-interval"
Expand Down
162 changes: 162 additions & 0 deletions simapp/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
module cosmossdk.io/simapp

go 1.18

require (
cosmossdk.io/api v0.2.1
cosmossdk.io/core v0.2.0
cosmossdk.io/depinject v1.0.0-alpha.2
cosmossdk.io/math v1.0.0-beta.3
github.com/cosmos/cosmos-sdk v0.0.0-00010101000000-000000000000
github.com/golang/mock v1.6.0
github.com/spf13/cast v1.5.0
github.com/spf13/cobra v1.5.0
github.com/spf13/viper v1.13.0
github.com/stretchr/testify v1.8.0
github.com/tendermint/tendermint v0.37.0-alpha.2
github.com/tendermint/tm-db v0.6.7
google.golang.org/protobuf v1.28.1
)

require (
cloud.google.com/go v0.100.2 // indirect
cloud.google.com/go/compute v1.6.1 // indirect
cloud.google.com/go/iam v0.3.0 // indirect
cloud.google.com/go/storage v1.14.0 // indirect
cosmossdk.io/errors v1.0.0-beta.7 // indirect
filippo.io/edwards25519 v1.0.0-rc.1 // indirect
github.com/99designs/go-keychain v0.0.0-20191008050251-8e49817e8af4 // indirect
github.com/99designs/keyring v1.2.1 // indirect
github.com/ChainSafe/go-schnorrkel v0.0.0-20200405005733-88cbf1b4c40d // indirect
github.com/armon/go-metrics v0.4.1 // indirect
github.com/aws/aws-sdk-go v1.40.45 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/bgentry/go-netrc v0.0.0-20140422174119-9fd32a8b3d3d // indirect
github.com/bgentry/speakeasy v0.1.0 // indirect
github.com/btcsuite/btcd v0.22.1 // indirect
github.com/cenkalti/backoff/v4 v4.1.3 // indirect
github.com/cespare/xxhash v1.1.0 // indirect
github.com/cespare/xxhash/v2 v2.1.2 // indirect
github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e // indirect
github.com/cockroachdb/apd/v2 v2.0.2 // indirect
github.com/coinbase/rosetta-sdk-go v0.8.0 // indirect
github.com/confio/ics23/go v0.7.0 // indirect
github.com/cosmos/btcutil v1.0.4 // indirect
github.com/cosmos/cosmos-proto v1.0.0-alpha7 // indirect
github.com/cosmos/go-bip39 v1.0.0 // indirect
github.com/cosmos/gogoproto v1.4.2 // indirect
github.com/cosmos/gorocksdb v1.2.0 // indirect
github.com/cosmos/iavl v0.19.2-0.20220916140702-9b6be3095313 // indirect
github.com/cosmos/ledger-cosmos-go v0.11.1 // indirect
github.com/cosmos/ledger-go v0.9.2 // indirect
github.com/creachadair/taskgroup v0.3.2 // indirect
github.com/danieljoos/wincred v1.1.2 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/desertbit/timer v0.0.0-20180107155436-c41aec40b27f // indirect
github.com/dgraph-io/badger/v2 v2.2007.4 // indirect
github.com/dgraph-io/ristretto v0.1.0 // indirect
github.com/dgryski/go-farm v0.0.0-20200201041132-a6ae2369ad13 // indirect
github.com/dustin/go-humanize v1.0.0 // indirect
github.com/dvsekhvalnov/jose2go v1.5.0 // indirect
github.com/felixge/httpsnoop v1.0.2 // indirect
github.com/fsnotify/fsnotify v1.5.4 // indirect
github.com/go-kit/kit v0.12.0 // indirect
github.com/go-kit/log v0.2.1 // indirect
github.com/go-logfmt/logfmt v0.5.1 // indirect
github.com/godbus/dbus v0.0.0-20190726142602-4481cbc300e2 // indirect
github.com/gogo/gateway v1.1.0 // indirect
github.com/gogo/googleapis v1.4.1-0.20201022092350-68b0159b7869 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/glog v1.0.0 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.2 // indirect
github.com/golang/snappy v0.0.4 // indirect
github.com/google/btree v1.0.1 // indirect
github.com/google/orderedcode v0.0.1 // indirect
github.com/googleapis/gax-go/v2 v2.4.0 // indirect
github.com/gorilla/handlers v1.5.1 // indirect
github.com/gorilla/mux v1.8.0 // indirect
github.com/gorilla/websocket v1.5.0 // indirect
github.com/grpc-ecosystem/go-grpc-middleware v1.3.0 // indirect
github.com/grpc-ecosystem/grpc-gateway v1.16.0 // indirect
github.com/gsterjov/go-libsecret v0.0.0-20161001094733-a6f4afe4910c // indirect
github.com/gtank/merlin v0.1.1 // indirect
github.com/gtank/ristretto255 v0.1.2 // indirect
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
github.com/hashicorp/go-getter v1.6.2 // indirect
github.com/hashicorp/go-immutable-radix v1.3.1 // indirect
github.com/hashicorp/go-safetemp v1.0.0 // indirect
github.com/hashicorp/go-version v1.6.0 // indirect
github.com/hashicorp/golang-lru v0.5.5-0.20210104140557-80c98217689d // indirect
github.com/hashicorp/hcl v1.0.0 // indirect
github.com/hdevalence/ed25519consensus v0.0.0-20220222234857-c00d1f31bab3 // indirect
github.com/improbable-eng/grpc-web v0.15.0 // indirect
github.com/inconshreveable/mousetrap v1.0.0 // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
github.com/jmhodges/levigo v1.0.0 // indirect
github.com/klauspost/compress v1.15.9 // indirect
github.com/lib/pq v1.10.6 // indirect
github.com/libp2p/go-buffer-pool v0.1.0 // indirect
github.com/magiconair/properties v1.8.6 // indirect
github.com/manifoldco/promptui v0.9.0 // indirect
github.com/mattn/go-isatty v0.0.16 // indirect
github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 // indirect
github.com/mimoo/StrobeGo v0.0.0-20210601165009-122bf33a46e0 // indirect
github.com/minio/highwayhash v1.0.2 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/mitchellh/go-testing-interface v1.0.0 // indirect
github.com/mitchellh/mapstructure v1.5.0 // indirect
github.com/mtibben/percent v0.2.1 // indirect
github.com/pelletier/go-toml v1.9.5 // indirect
github.com/pelletier/go-toml/v2 v2.0.5 // indirect
github.com/petermattis/goid v0.0.0-20180202154549-b0b1615b78e5 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/prometheus/client_golang v1.13.0 // indirect
github.com/prometheus/client_model v0.2.0 // indirect
github.com/prometheus/common v0.37.0 // indirect
github.com/prometheus/procfs v0.8.0 // indirect
github.com/rakyll/statik v0.1.7 // indirect
github.com/rcrowley/go-metrics v0.0.0-20201227073835-cf1acfcdf475 // indirect
github.com/rs/cors v1.8.2 // indirect
github.com/sasha-s/go-deadlock v0.3.1 // indirect
github.com/spf13/afero v1.8.2 // indirect
github.com/spf13/jwalterweatherman v1.1.0 // indirect
github.com/spf13/pflag v1.0.5 // indirect
github.com/subosito/gotenv v1.4.1 // indirect
github.com/syndtr/goleveldb v1.0.1-0.20210819022825-2ae1ddf74ef7 // indirect
github.com/tendermint/btcd v0.1.1 // indirect
github.com/tendermint/crypto v0.0.0-20191022145703-50d29ede1e15 // indirect
github.com/tendermint/go-amino v0.16.0 // indirect
github.com/ulikunitz/xz v0.5.8 // indirect
github.com/zondax/hid v0.9.1-0.20220302062450-5552068d2266 // indirect
go.etcd.io/bbolt v1.3.6 // indirect
go.opencensus.io v0.23.0 // indirect
golang.org/x/crypto v0.0.0-20220722155217-630584e8d5aa // indirect
golang.org/x/exp v0.0.0-20220722155223-a9213eeb770e // indirect
golang.org/x/net v0.0.0-20220726230323-06994584191e // indirect
golang.org/x/oauth2 v0.0.0-20220411215720-9780585627b5 // indirect
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab // indirect
golang.org/x/term v0.0.0-20220722155259-a9ba230a4035 // indirect
golang.org/x/text v0.3.7 // indirect
golang.org/x/xerrors v0.0.0-20220517211312-f3a8303e98df // indirect
google.golang.org/api v0.81.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20220725144611-272f38e5d71b // indirect
google.golang.org/grpc v1.49.0 // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
nhooyr.io/websocket v1.8.6 // indirect
pgregory.net/rapid v0.5.3 // indirect
sigs.k8s.io/yaml v1.3.0 // indirect
)

replace (
github.com/99designs/keyring => github.com/cosmos/keyring v1.2.0
github.com/cosmos/cosmos-sdk => ../.
// Fix upstream GHSA-h395-qcrw-5vmq vulnerability.
// TODO Remove it: https://github.com/cosmos/cosmos-sdk/issues/10409
github.com/gin-gonic/gin => github.com/gin-gonic/gin v1.8.1
github.com/gogo/gateway => github.com/notional-labs/gateway v1.1.1-0.20220417180718-8e60e17a098d
)
Loading