Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

panic: runtime error: invalid memory address or nil pointer dereference #3032

Closed
wjdfx opened this issue Jun 18, 2020 · 2 comments · Fixed by #3033
Closed

panic: runtime error: invalid memory address or nil pointer dereference #3032

wjdfx opened this issue Jun 18, 2020 · 2 comments · Fixed by #3033
Labels
c:bug Category: bug p:0 Priority: High! bugs, address immediately

Comments

@wjdfx
Copy link

wjdfx commented Jun 18, 2020

SUMMARY

When start oasis-node got an error: panic: runtime error: invalid memory address or nil pointer dereference

ISSUE TYPE
  • Bug Report
COMPONENT NAME

oasis-node

OASIS NODE VERSION
oasis-node --version
Software version: 20.8.1
Runtime protocol version: 0.14.0
Consensus protocol version: 0.25.0
Committee protocol version: 0.9.0
Tendermint core version: 0.33.4
ABCI library version: 0.16.2
Go toolchain version: 1.14.4
OS / ENVIRONMENT

Ubuntu 18.04

STEPS TO REPRODUCE
Log
`{"caller":"common.go:211","cur":1024,"level":"warn","max":1048576,"module":"oasis-node","msg":"the node user has a very low RLIMIT_NOFILE, consider increasing","ts":"2020-06-18T16:35:54.934234044Z"}
{"Version":"20.8.1","caller":"node.go:572","level":"info","module":"oasis-node","msg":"Starting oasis-node","ts":"2020-06-18T16:35:54.935913744Z"}
{"caller":"helpers.go:40","level":"info","module":"common/persistent","msg":"All 0 tables opened in 0s","ts":"2020-06-18T16:35:54.959449486Z"}
{"caller":"node.go:625","consensus_pk":"YMht021ec8eMkSk9vkKs+LV8+WL/8XIgttcQIKYiHlo=","level":"info","module":"oasis-node","msg":"loaded/generated node identity","node_pk":"CeK2kK5crFPeQaYmRG/y3H7xt+j+yKDFoIluRKRGXgM=","p2p_pk":"/vKW4yscxI0F8ifZTXOzfmnERS14YJc50fb7RNnlUj4=","tls_pk":"fVYqMZkZudGLtJj6j4cUrtryM6Q4jaj+HzvG3GAXAYg=","ts":"2020-06-18T16:35:54.96502765Z"}
{"caller":"helpers.go:40","level":"info","module":"mkvs/db/badger","msg":"All 0 tables opened in 0s","ts":"2020-06-18T16:35:55.080871795Z"}
{"caller":"client.go:99","level":"warn","module":"ias/proxyclient","msg":"IAS proxy is not configured, all reports will be mocked","ts":"2020-06-18T16:35:55.284963546Z"}
{"caller":"registry.go:421","err":"consensus: no committed blocks","level":"error","module":"registry/tendermint","msg":"runtime notifier: unable to get a list of runtimes","ts":"2020-06-18T16:35:55.291671601Z"}
{"caller":"node.go:770","level":"info","module":"oasis-node","msg":"starting Oasis node","ts":"2020-06-18T16:35:55.299110012Z"}
{"caller":"registry.go:360","level":"info","module":"runtime/registry","msg":"no supported runtimes configured","ts":"2020-06-18T16:35:55.302557726Z"}
{"caller":"worker.go:171","level":"info","module":"worker/storage","msg":"not starting storage worker as it is disabled","ts":"2020-06-18T16:35:55.33340542Z"}
{"caller":"worker.go:43","level":"info","module":"worker/executor","msg":"not starting executor worker as it is disabled","ts":"2020-06-18T16:35:55.333486546Z"}
{"caller":"worker.go:44","level":"info","module":"worker/txnscheduler","msg":"not starting transaction scheduler as it is disabled","ts":"2020-06-18T16:35:55.333519135Z"}
{"caller":"worker.go:41","level":"info","module":"worker/merge","msg":"not starting merge worker as it is disabled","ts":"2020-06-18T16:35:55.333548655Z"}
{"caller":"worker.go:52","level":"info","module":"worker/common","msg":"not starting common worker as it is disabled","ts":"2020-06-18T16:35:55.333572118Z"}
{"caller":"worker.go:88","level":"info","module":"worker/keymanager","msg":"not starting key manager worker as it is disabled","ts":"2020-06-18T16:35:55.333588186Z"}
{"caller":"worker.go:953","level":"info","module":"worker/registration","msg":"starting node registration service","ts":"2020-06-18T16:35:55.333600015Z"}
{"caller":"worker.go:60","level":"info","module":"worker/sentry","msg":"not starting sentry worker as it is disabled","ts":"2020-06-18T16:35:55.333625946Z"}
{"caller":"service.go:139","impl":"multiAppConn","level":"info","module":"tendermint:proxy","msg":"Starting multiAppConn service","ts":"2020-06-18T16:35:55.358251707Z"}
{"caller":"service.go:139","connection":"query","impl":"localClient","level":"info","module":"tendermint:proxy/abci-client","msg":"Starting localClient service","ts":"2020-06-18T16:35:55.358317226Z"}
{"caller":"service.go:139","connection":"mempool","impl":"localClient","level":"info","module":"tendermint:proxy/abci-client","msg":"Starting localClient service","ts":"2020-06-18T16:35:55.358351043Z"}
{"caller":"service.go:139","connection":"consensus","impl":"localClient","level":"info","module":"tendermint:proxy/abci-client","msg":"Starting localClient service","ts":"2020-06-18T16:35:55.358376246Z"}
{"caller":"service.go:139","impl":"EventBus","level":"info","module":"tendermint:events","msg":"Starting EventBus service","ts":"2020-06-18T16:35:55.358395681Z"}
{"caller":"service.go:139","impl":"PubSub","level":"info","module":"tendermint:events/pubsub","msg":"Starting PubSub service","ts":"2020-06-18T16:35:55.35841643Z"}
{"caller":"service.go:139","impl":"IndexerService","level":"info","module":"tendermint:txindex","msg":"Starting IndexerService service","ts":"2020-06-18T16:35:55.358447053Z"}
{"caller":"replay.go:260","hash":"","height":0,"level":"info","module":"tendermint:consensus","msg":"ABCI Handshake App Info","protocol-version":1638400,"software-version":"","ts":"2020-06-18T16:35:55.358555116Z"}
{"appHeight":0,"caller":"replay.go:299","level":"info","module":"tendermint:consensus","msg":"ABCI Replay Blocks","stateHeight":0,"storeHeight":0,"ts":"2020-06-18T16:35:55.360609528Z"}
{"caller":"genesis.go:55","common_pool":"8488535000000000000","last_block_fees":"0","level":"warn","mode":"init chain","module":"consensus/tendermint/abci","msg":"InitChain: moving last block fees into common pool","ts":"2020-06-18T16:35:55.366229328Z"}
{"appHash":"","appHeight":0,"caller":"replay.go:279","level":"info","module":"tendermint:consensus","msg":"Completed ABCI Handshake - Tendermint and App are synced","ts":"2020-06-18T16:35:55.426520606Z"}
{"addr":"08C0CB207D8F5F0E4830633B850C80C3F9CA5D3C","caller":"node.go:310","level":"info","module":"tendermint:consensus","msg":"This node is not a validator","pubKey":"PubKeyEd25519{60C86DD36D5E73C78C91293DBE42ACF8B57CF962FFF17220B6D71020A6221E5A}","ts":"2020-06-18T16:35:55.427804264Z"}
{"ID":"3b67db4bba2509cd6abc0d35f6f8bf4f6366eeca","caller":"node.go:509","file":"/mnt/serverdir/node/tendermint/config/node_key.json","level":"info","module":"tendermint:p2p","msg":"P2P Node ID","ts":"2020-06-18T16:35:55.452987178Z"}
{"addrs":[],"caller":"switch.go:558","level":"info","module":"tendermint:p2p","msg":"Adding persistent peers","ts":"2020-06-18T16:35:55.453109248Z"}
{"caller":"switch.go:576","ids":[],"level":"info","module":"tendermint:p2p","msg":"Adding unconditional peer ids","ts":"2020-06-18T16:35:55.4531487Z"}
{"addr":"[email protected]:26656","book":"/mnt/serverdir/node/tendermint/config/addrbook.json","caller":"addrbook.go:175","level":"info","module":"tendermint:p2p","msg":"Add our address to book","ts":"2020-06-18T16:35:55.453314563Z"}
{"addr":"[email protected]:26656","book":"/mnt/serverdir/node/tendermint/config/addrbook.json","caller":"addrbook.go:175","level":"info","module":"tendermint:p2p","msg":"Add our address to book","ts":"2020-06-18T16:35:55.453368381Z"}
{"caller":"service.go:139","impl":"P2P Switch","level":"info","module":"tendermint:p2p","msg":"Starting P2P Switch service","ts":"2020-06-18T16:35:55.453525371Z"}
{"caller":"service.go:139","impl":"Evidence","level":"info","module":"tendermint:evidence","msg":"Starting Evidence service","ts":"2020-06-18T16:35:55.453563847Z"}
{"caller":"service.go:139","impl":"PEX","level":"info","module":"tendermint:pex","msg":"Starting PEX service","ts":"2020-06-18T16:35:55.453620538Z"}
{"book":"/mnt/serverdir/node/tendermint/config/addrbook.json","caller":"service.go:139","impl":"AddrBook","level":"info","module":"tendermint:p2p","msg":"Starting AddrBook service","ts":"2020-06-18T16:35:55.453661304Z"}
{"caller":"service.go:139","impl":"Mempool","level":"info","module":"tendermint:mempool","msg":"Starting Mempool service","ts":"2020-06-18T16:35:55.453740393Z"}
{"caller":"service.go:139","impl":"BlockchainReactor","level":"info","module":"tendermint:blockchain","msg":"Starting BlockchainReactor service","ts":"2020-06-18T16:35:55.453769415Z"}
{"caller":"service.go:139","impl":"BlockPool","level":"info","module":"tendermint:blockchain","msg":"Starting BlockPool service","ts":"2020-06-18T16:35:55.45380464Z"}
{"caller":"service.go:139","impl":"ConsensusReactor","level":"info","module":"tendermint:consensus","msg":"Starting Consensus service","ts":"2020-06-18T16:35:55.453864177Z"}
{"caller":"reactor.go:73","fastSync":true,"level":"info","module":"tendermint:consensus","msg":"Reactor ","ts":"2020-06-18T16:35:55.45390395Z"}
{"book":"/mnt/serverdir/node/tendermint/config/addrbook.json","caller":"file.go:22","level":"info","module":"tendermint:p2p","msg":"Saving AddrBook to file","size":0,"ts":"2020-06-18T16:35:55.453947427Z"}
{"caller":"pex_reactor.go:450","level":"info","module":"tendermint:pex","msg":"Ensure peers","numDialing":0,"numInPeers":0,"numOutPeers":0,"numToDial":20,"ts":"2020-06-18T16:35:55.454032578Z"}
{"caller":"pex_reactor.go:524","level":"info","module":"tendermint:pex","msg":"No addresses to dial. Falling back to seeds","ts":"2020-06-18T16:35:55.455990609Z"}
{"address":"[email protected]:26656","caller":"switch.go:693","level":"info","module":"tendermint:p2p","msg":"Dialing peer","ts":"2020-06-18T16:35:55.456063839Z"}
{"caller":"grpc.go:334","level":"info","module":"grpc/internal","msg":"starting gRPC server","ts":"2020-06-18T16:35:55.457257786Z"}
{"address":"/mnt/serverdir/node/internal.sock","caller":"grpc.go:349","level":"info","module":"grpc/internal","msg":"gRPC server started","network":"unix","ts":"2020-06-18T16:35:55.457403761Z"}
{"caller":"node.go:846","level":"info","module":"oasis-node","msg":"initialization complete: ready to serve","ts":"2020-06-18T16:35:55.457543865Z"}
{"caller":"service.go:139","impl":"Peer{MConn{34.86.145.181:26656} 9fbca3364885ebc82bf5be8156bb4df0a563bbf6 out}","level":"info","module":"tendermint:p2p","msg":"Starting Peer service","peer":"[email protected]:26656","ts":"2020-06-18T16:35:55.801698095Z"}
{"caller":"service.go:139","impl":"MConn{34.86.145.181:26656}","level":"info","module":"tendermint:p2p","msg":"Starting MConnection service","peer":"[email protected]:26656","ts":"2020-06-18T16:35:55.801790405Z"}
{"caller":"switch.go:813","level":"info","module":"tendermint:p2p","msg":"Added peer","peer":"Peer{MConn{34.86.145.181:26656} 9fbca3364885ebc82bf5be8156bb4df0a563bbf6 out}","ts":"2020-06-18T16:35:55.801904Z"}
{"caller":"connection.go:593","conn":"MConn{34.86.145.181:26656}","level":"info","module":"tendermint:p2p","msg":"Connection is closed @ recvRoutine (likely by the other side)","peer":"[email protected]:26656","ts":"2020-06-18T16:35:56.018229529Z"}
{"caller":"service.go:168","impl":"MConn{34.86.145.181:26656}","level":"info","module":"tendermint:p2p","msg":"Stopping MConnection service","peer":"[email protected]:26656","ts":"2020-06-18T16:35:56.018381208Z"}
{"caller":"switch.go:324","err":"EOF","level":"error","module":"tendermint:p2p","msg":"Stopping peer for error","peer":"Peer{MConn{34.86.145.181:26656} 9fbca3364885ebc82bf5be8156bb4df0a563bbf6 out}","ts":"2020-06-18T16:35:56.018585376Z"}
{"caller":"service.go:168","impl":"Peer{MConn{34.86.145.181:26656} 9fbca3364885ebc82bf5be8156bb4df0a563bbf6 out}","level":"info","module":"tendermint:p2p","msg":"Stopping Peer service","peer":"[email protected]:26656","ts":"2020-06-18T16:35:56.01872587Z"}
{"caller":"reactor.go:616","level":"info","module":"tendermint:consensus","msg":"Stopping gossipVotesRoutine for peer","peer":"Peer{MConn{34.86.145.181:26656} 9fbca3364885ebc82bf5be8156bb4df0a563bbf6 out}","ts":"2020-06-18T16:35:56.103695292Z"}
{"caller":"reactor.go:474","level":"info","module":"tendermint:consensus","msg":"Stopping gossipDataRoutine for peer","peer":"Peer{MConn{34.86.145.181:26656} 9fbca3364885ebc82bf5be8156bb4df0a563bbf6 out}","ts":"2020-06-18T16:35:56.103802702Z"}
{"caller":"reactor.go:745","level":"info","module":"tendermint:consensus","msg":"Stopping queryMaj23Routine for peer","peer":"Peer{MConn{34.86.145.181:26656} 9fbca3364885ebc82bf5be8156bb4df0a563bbf6 out}","ts":"2020-06-18T16:35:57.811179353Z"}
{"caller":"log.go:30","level":"info","module":"grpc","msg":"transport: loopyWriter.run returning. connection error: desc = \"transport is closing\"","ts":"2020-06-18T16:35:58.245094894Z"}
{"caller":"log.go:30","level":"info","module":"grpc","msg":"transport: loopyWriter.run returning. connection error: desc = \"transport is closing\"","ts":"2020-06-18T16:35:58.25257213Z"}
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x1b8 pc=0xdc82e9]

goroutine 27 [running]:
github.com/oasisprotocol/oasis-core/go/consensus/tendermint/api.NewBlock(0x0, 0x1d0d940)
	github.com/oasisprotocol/oasis-core/go/consensus/tendermint/api/api.go:173 +0x29
github.com/oasisprotocol/oasis-core/go/consensus/tendermint.(*tendermintService).GetBlock(0xc006746600, 0x1d0d940, 0xc00d736750, 0x0, 0xc006746600, 0x4196cb, 0xc000032000)
	github.com/oasisprotocol/oasis-core/go/consensus/tendermint/tendermint.go:692 +0x87
github.com/oasisprotocol/oasis-core/go/consensus/api.handlerGetBlock.func1(0x1d0d940, 0xc00d736750, 0x163a500, 0x2c9b0e0, 0x10, 0xc00db356e0, 0x41aab3, 0x16f8a00)
	github.com/oasisprotocol/oasis-core/go/consensus/api/grpc.go:284 +0x89
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1(0x1d0d940, 0xc00d736750, 0x163a500, 0x2c9b0e0, 0x163a500, 0x2c9b0e0, 0x0, 0x0)
	github.com/grpc-ecosystem/[email protected]/chain.go:31 +0x114
github.com/oasisprotocol/oasis-core/go/common/grpc/auth.UnaryServerInterceptor.func1(0x1d0d940, 0xc00d736750, 0x163a500, 0x2c9b0e0, 0xc00dbdeb20, 0xc00dd56d20, 0xc00d736750, 0xc00db357b8, 0x5237d8, 0x180f900)
	github.com/oasisprotocol/oasis-core/go/common/grpc/auth/auth.go:44 +0x20d
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1(0x1d0d940, 0xc00d736750, 0x163a500, 0x2c9b0e0, 0x16da320, 0x2c99760, 0x1873840, 0x2c99760)
	github.com/grpc-ecosystem/[email protected]/chain.go:34 +0x9c
github.com/oasisprotocol/oasis-core/go/common/grpc.serverUnaryErrorMapper(0x1d0d940, 0xc00d736750, 0x163a500, 0x2c9b0e0, 0xc00dbdeb20, 0xc00dd56d20, 0x0, 0xc00dbdeb80, 0xf6eca99ddda03400, 0x0)
	github.com/oasisprotocol/oasis-core/go/common/grpc/errors.go:80 +0x55
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1(0x1d0d940, 0xc00d736750, 0x163a500, 0x2c9b0e0, 0xc00011dce0, 0x1e, 0x1d0d940, 0xc00d736750)
	github.com/grpc-ecosystem/[email protected]/chain.go:34 +0x9c
github.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing.UnaryServerInterceptor.func1(0x1d0d940, 0xc00d736690, 0x163a500, 0x2c9b0e0, 0xc00dbdeb20, 0xc00dd56d20, 0xc00db35970, 0x4d99d6, 0x5eeb97f4, 0x16bf7b6a)
	github.com/grpc-ecosystem/[email protected]/tracing/opentracing/server_interceptors.go:30 +0x165
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1(0x1d0d940, 0xc00d736690, 0x163a500, 0x2c9b0e0, 0xc00677e088, 0xc00db359d0, 0x15f2d80, 0xc00dbdcf60)
	github.com/grpc-ecosystem/[email protected]/chain.go:34 +0x9c
github.com/oasisprotocol/oasis-core/go/common/grpc.(*grpcLogAdapter).unaryLogger(0xc0011ff920, 0x1d0d940, 0xc00d736690, 0x163a500, 0x2c9b0e0, 0xc00dbdeb20, 0xc00dd56d20, 0x50, 0x183b880, 0x1, ...)
	github.com/oasisprotocol/oasis-core/go/common/grpc/grpc.go:166 +0x16b
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1(0x1d0d940, 0xc00d736690, 0x163a500, 0x2c9b0e0, 0xc00dbdeb20, 0xc00dbdeb40, 0x180f900, 0xc00dbdcf01, 0xc00d736690, 0xc00db35ba0)
	github.com/grpc-ecosystem/[email protected]/chain.go:39 +0x155
github.com/oasisprotocol/oasis-core/go/consensus/api.handlerGetBlock(0x19017e0, 0xc006746600, 0x1d0d940, 0xc00d736690, 0xc00dc74ea0, 0xc0011ff950, 0x1d0d940, 0xc00d736690, 0xc00dbdcf58, 0x1)
	github.com/oasisprotocol/oasis-core/go/consensus/api/grpc.go:286 +0x197
google.golang.org/grpc.(*Server).processUnaryRPC(0xc0002401a0, 0x1d23aa0, 0xc00db5b800, 0xc00dfe0200, 0xc00dc4d7d0, 0x2b05250, 0x0, 0x0, 0x0)
	google.golang.org/[email protected]/server.go:1082 +0x50a
google.golang.org/grpc.(*Server).handleStream(0xc0002401a0, 0x1d23aa0, 0xc00db5b800, 0xc00dfe0200, 0x0)
	google.golang.org/[email protected]/server.go:1405 +0xccb
google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc00dbdcf10, 0xc0002401a0, 0x1d23aa0, 0xc00db5b800, 0xc00dfe0200)
	google.golang.org/[email protected]/server.go:746 +0xa1
created by google.golang.org/grpc.(*Server).serveStreams.func1
	google.golang.org/[email protected]/server.go:744 +0xa1`
EXPECTED RESULTS
@kostko
Copy link
Member

kostko commented Jun 18, 2020 •
edited
Loading

Thanks for your report, this is a corner case that happens during certain RPC queries while there are no blocks committed yet. Will fix this ASAP!

@kostko kostko added c:bug Category: bug p:0 Priority: High! bugs, address immediately labels Jun 18, 2020
@kostko kostko mentioned this issue Jun 18, 2020
Merged
1 task
@kostko
Copy link
Member

kostko commented Jun 18, 2020 •
edited
Loading

Proposed fix in #3033, will backport it to 20.8.x.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
c:bug Category: bug p:0 Priority: High! bugs, address immediately
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

go/consensus/tendermint: Properly handle no committed blocks oasisprotocol/oasis-core
2 participants
@kostko @wjdfx