Error "Attempt to decode value on failed cursor: DownField(metadata)" indexing against tezos ithacanet

[jun0tpyrc]

2022-march-release-40

Can talk to v12.1 tezos && db , keep getting some blocks & print in log, but won't write finally & exit

2022.04.01 10:36:40 [lorre-tezos-indexer-akka.actor.default-dispatcher-7] ERROR TezosNodeFetchers (tech.cryptonomic.conseil.indexer.tezos.TezosNodeFetchers.applyOrElse:40) - I fetched a block definition from tezos node that I'm unable to decode: {"protocol":"PtHangz2aRngywmSRGGvrcTyMbbdpWdpFKuS4uMWxg2RaH9i1qx","chain_id":"NetXnHfVqm9iesp","hash":"BLJqv5yJ9BvXkVS6RVmSH4FGreKJMgsYevHrzG3ABg9W3XEdfDM","header":{"level":477,"proto":1,"predecessor":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","timestamp":"2022-01-26T06:07:05Z","validation_pass":4,"operations_hash":"LLoZUuhFPN7kE8F8qsSoG9Nsd7RmBip3mvFpZ3S56NSgPfaKcNjzp","fitness":["01","00000000000001dc"],"context":"CoVhNAzdpPfdwm6gGa6QUJzLmR14oWnfmqpsJ3dFCzhe8PACCAMj","priority":0,"proof_of_work_nonce":"10723d9c2c250800","liquidity_baking_escape_vote":false,"signature":"sigsQUUmFm6Hq68eYN1b613A1w4Zf3ahnWBVnopbprEqdAsjnNFgDQUvoWJ6nwfXVGwpcAHWsvxCbJAXmdDFwq1N587PzaJ9"},"operations":[[{"protocol":"PtHangz2aRngywmSRGGvrcTyMbbdpWdpFKuS4uMWxg2RaH9i1qx","chain_id":"NetXnHfVqm9iesp","hash":"oor1HhAJbuxwUtME2Cx32jUfmnviFuX2cbTGXx2caBDHCNwN1kf","branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","contents":[{"kind":"endorsement_with_slot","endorsement":{"branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","operations":{"kind":"endorsement","level":476},"signature":"sigVvVnFWjznwdA2mtKFFdjRKhf5b1SmZoVMoniiMzGsShzgk2jXNGyMWkunUyWz7uANocYfvbTLz81X88Ls9h5DgVgvzT4i"},"slot":62}],"signature":"edsigtXomBKi5CTRf5cjATJWSyaRvhfYNHqSUGrn4SdbYRcGwQrUGjzEfQDTuqHhuA8b2d8NarZjz8TRf65WkpQmo423BtomS8Q"},{"protocol":"PtHangz2aRngywmSRGGvrcTyMbbdpWdpFKuS4uMWxg2RaH9i1qx","chain_id":"NetXnHfVqm9iesp","hash":"ooAbf3JPpz4tq2S3dk6PpMz3VnPH9QiccMaLwMQuju8TDFLRCyC","branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","contents":[{"kind":"endorsement_with_slot","endorsement":{"branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","operations":{"kind":"endorsement","level":476},"signature":"sigw8E3jAuEXBK42SyryREx9oKMd4qSY136EeTNbAaprbqZX6ZYepUoWeGYC7zSpzo3yGYoMR9Lf8yHwqFWwTNuL26X7R6u5"},"slot":16}],"signature":"edsigtXomBKi5CTRf5cjATJWSyaRvhfYNHqSUGrn4SdbYRcGwQrUGjzEfQDTuqHhuA8b2d8NarZjz8TRf65WkpQmo423BtomS8Q"},{"protocol":"PtHangz2aRngywmSRGGvrcTyMbbdpWdpFKuS4uMWxg2RaH9i1qx","chain_id":"NetXnHfVqm9iesp","hash":"opPewSqMsQu3nWVP9FPQ3QD1Dp7MJX5fU8gu1fLusA1Nukoavtp","branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","contents":[{"kind":"endorsement_with_slot","endorsement":{"branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","operations":{"kind":"endorsement","level":476},"signature":"sigfibs3n2W3fkvvBa9FLP6k2JkGSQmJCxg81CwToLsZXuKkFJaC4dmb7N2UPgG6GkPcqfSASoFtam8Yho3QJGQQPnBeanww"},"slot":60}],"signature":"edsigtXomBKi5CTRf5cjATJWSyaRvhfYNHqSUGrn4SdbYRcGwQrUGjzEfQDTuqHhuA8b2d8NarZjz8TRf65WkpQmo423BtomS8Q"},{"protocol":"PtHangz2aRngywmSRGGvrcTyMbbdpWdpFKuS4uMWxg2RaH9i1qx","chain_id":"NetXnHfVqm9iesp","hash":"onokoMJ8utRDJr3bFoBnwzcAqv7eBnxrCoPVJZrjFohc3j5FiVy","branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","contents":[{"kind":"endorsement_with_slot","endorsement":{"branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","operations":{"kind":"endorsement","level":476},"signature":"sigsNUg1aoWstWdscGpsWGjS3tihSEZoabHQjsAavRDdN8HzoPkWhxCQXmqs5tGPngAFJ91Caehwa4R4cUd4JkQ8sFpq631Q"},"slot":1}],"signature":"edsigtXomBKi5CTRf5cjATJWSyaRvhfYNHqSUGrn4SdbYRcGwQrUGjzEfQDTuqHhuA8b2d8NarZjz8TRf65WkpQmo423BtomS8Q"},{"protocol":"PtHangz2aRngywmSRGGvrcTyMbbdpWdpFKuS4uMWxg2RaH9i1qx","chain_id":"NetXnHfVqm9iesp","hash":"ooDxyqT1no7rqj3LDTTTJ2f4pdLxyqoCdo5fyDJtZfjdpPStaj5","branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","contents":[{"kind":"endorsement_with_slot","endorsement":{"branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","operations":{"kind":"endorsement","level":476},"signature":"sigNr2vquXgpAZzuqWrfd7YDctizp31vKjXjPcHx8QmTqo2mbigvX3LBeGbR4Hwfg7Qwtvsfznq8mPWnLNfYrZrJtmpCX7j1"},"slot":6}],"signature":"edsigtXomBKi5CTRf5cjATJWSyaRvhfYNHqSUGrn4SdbYRcGwQrUGjzEfQDTuqHhuA8b2d8NarZjz8TRf65WkpQmo423BtomS8Q"},{"protocol":"PtHangz2aRngywmSRGGvrcTyMbbdpWdpFKuS4uMWxg2RaH9i1qx","chain_id":"NetXnHfVqm9iesp","hash":"ong486NEgzgm8J1atAQRpkr5X941erx7RVbiu8gNc5z9PkWQdkc","branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","contents":[{"kind":"endorsement_with_slot","endorsement":{"branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","operations":{"kind":"endorsement","level":476},"signature":"siga26YWNpj1VhXUQyaWdc7XtARGF4cfaGD888CHJd9b8mwXrwTEZeCbVxzK2gP5NU3enMQo9V5q8tXzSJGjPEYFiHwJBzBp"},"slot":2}],"signature":"edsigtXomBKi5CTRf5cjATJWSyaRvhfYNHqSUGrn4SdbYRcGwQrUGjzEfQDTuqHhuA8b2d8NarZjz8TRf65WkpQmo423BtomS8Q"},{"protocol":"PtHangz2aRngywmSRGGvrcTyMbbdpWdpFKuS4uMWxg2RaH9i1qx","chain_id":"NetXnHfVqm9iesp","hash":"ooZAEnGsdKupjYGuVJz6U2LQHKqh1jrCvYQX1qMWQHaZu8HsYub","branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","contents":[{"kind":"endorsement_with_slot","endorsement":{"branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","operations":{"kind":"endorsement","level":476},"signature":"sigsVCPH83PYaTTa1o7U3XeuFi2gJUobWxy1wcU2KNYZ3zgWzDn2oyreJrE73oGrCWGZ7re4oX5sFABvXZorR5Cs35zeWfTh"},"slot":21}],"signature":"edsigtXomBKi5CTRf5cjATJWSyaRvhfYNHqSUGrn4SdbYRcGwQrUGjzEfQDTuqHhuA8b2d8NarZjz8TRf65WkpQmo423BtomS8Q"},{"protocol":"PtHangz2aRngywmSRGGvrcTyMbbdpWdpFKuS4uMWxg2RaH9i1qx","chain_id":"NetXnHfVqm9iesp","hash":"ooUSEYf5ceMPzRmp8zgcondBiLRfJtaAPvunJ64ZS6apvMLDV33","branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","contents":[{"kind":"endorsement_with_slot","endorsement":{"branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","operations":{"kind":"endorsement","level":476},"signature":"sigem2zRaPG6iVKssyRknv8LQwAjnS65JvAqKJQRjAvrQNmDpVJ2MHaY35YJ717ovh6TqtzR3MVBJjZ65NgADwVypXHxme8T"},"slot":9}],"signature":"edsigtXomBKi5CTRf5cjATJWSyaRvhfYNHqSUGrn4SdbYRcGwQrUGjzEfQDTuqHhuA8b2d8NarZjz8TRf65WkpQmo423BtomS8Q"},{"protocol":"PtHangz2aRngywmSRGGvrcTyMbbdpWdpFKuS4uMWxg2RaH9i1qx","chain_id":"NetXnHfVqm9iesp","hash":"oo2DBosPz4x2dUEe8pDP3pszxhFez9JtbUxXrq9XTTVHSiFbYRD","branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","contents":[{"kind":"endorsement_with_slot","endorsement":{"branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","operations":{"kind":"endorsement","level":476},"signature":"sigUToT37tHvs1bh6indQRuc55HhojPfemfprQPZjBvLF6w8YYBvUkMU6yGKbtwaRL4PQzUAftxmEc3kAbnzCJHG25FgwHJC"},"slot":5}],"signature":"edsigtXomBKi5CTRf5cjATJWSyaRvhfYNHqSUGrn4SdbYRcGwQrUGjzEfQDTuqHhuA8b2d8NarZjz8TRf65WkpQmo423BtomS8Q"},{"protocol":"PtHangz2aRngywmSRGGvrcTyMbbdpWdpFKuS4uMWxg2RaH9i1qx","chain_id":"NetXnHfVqm9iesp","hash":"ooYja4FVK1EZ2Lti36bYvqoPpmuuFSntq2ThZA7T9mDQ5TA54JM","branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","contents":[{"kind":"endorsement_with_slot","endorsement":{"branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","operations":{"kind":"endorsement","level":476},"signature":"siganbH5Lsah3bbCDeiyWNS8LJdUvuAJbmderz79KnuSCxGeTfXaFRC8t81MqGaYPTjAAz41KxDmgYnDfSL82eQZX8tLvGYG"},"slot":18}],"signature":"edsigtXomBKi5CTRf5cjATJWSyaRvhfYNHqSUGrn4SdbYRcGwQrUGjzEfQDTuqHhuA8b2d8NarZjz8TRf65WkpQmo423BtomS8Q"},{"protocol":"PtHangz2aRngywmSRGGvrcTyMbbdpWdpFKuS4uMWxg2RaH9i1qx","chain_id":"NetXnHfVqm9iesp","hash":"opJX2mmLUmb8xBqMQ9qNX7x9R8XoG8HSgvpmB2HuAwSViLB4mHG","branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","contents":[{"kind":"endorsement_with_slot","endorsement":{"branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","operations":{"kind":"endorsement","level":476},"signature":"sigccF3eVb33bpmpGy4Y8H3zvPKVhHAFi6GWHHkJ9K1UJtxLTeBhgD57bkam5D54vxccHtn2xonX7wo3guaKw1hBm1p39XnF"},"slot":22}],"signature":"edsigtXomBKi5CTRf5cjATJWSyaRvhfYNHqSUGrn4SdbYRcGwQrUGjzEfQDTuqHhuA8b2d8NarZjz8TRf65WkpQmo423BtomS8Q"},{"protocol":"PtHangz2aRngywmSRGGvrcTyMbbdpWdpFKuS4uMWxg2RaH9i1qx","chain_id":"NetXnHfVqm9iesp","hash":"oorDJkeLX4yRosh947ARumHVoRQTBovaNxXZczzNRS3vG9eCYHC","branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","contents":[{"kind":"endorsement_with_slot","endorsement":{"branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","operations":{"kind":"endorsement","level":476},"signature":"sigTexDpRsKyXwgmM2PPv8kY1YB9not8AK8redBFEMbi2K9rY7XHHqAA6CbwFhQ6SRKx4PQBqw7bKg62ZpP5dLePKGB8BNvb"},"slot":24}],"signature":"edsigtXomBKi5CTRf5cjATJWSyaRvhfYNHqSUGrn4SdbYRcGwQrUGjzEfQDTuqHhuA8b2d8NarZjz8TRf65WkpQmo423BtomS8Q"},{"protocol":"PtHangz2aRngywmSRGGvrcTyMbbdpWdpFKuS4uMWxg2RaH9i1qx","chain_id":"NetXnHfVqm9iesp","hash":"ooLYXiZdbRULX22duPzvaKZ5dAihZ3D9aHh9y5UigXfjLdmRGaj","branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","contents":[{"kind":"endorsement_with_slot","endorsement":{"branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","operations":{"kind":"endorsement","level":476},"signature":"sigVwpp2k6NPL53VfegiTeHKHAkGYuakTzFC12f2Zf3RqN3NL5NbaeyqZaL4TJjkrxbvWW1R1rUbNy1Fta7uPPtJsofkzhSo"},"slot":34}],"signature":"edsigtXomBKi5CTRf5cjATJWSyaRvhfYNHqSUGrn4SdbYRcGwQrUGjzEfQDTuqHhuA8b2d8NarZjz8TRf65WkpQmo423BtomS8Q"},{"protocol":"PtHangz2aRngywmSRGGvrcTyMbbdpWdpFKuS4uMWxg2RaH9i1qx","chain_id":"NetXnHfVqm9iesp","hash":"oonmiApJE793nQw2ucvUGc5vsMSQMHezvd461Dvgkz7ebxEXzG2","branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","contents":[{"kind":"endorsement_with_slot","endorsement":{"branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","operations":{"kind":"endorsement","level":476},"signature":"sigQQZvCbY9haML16DgcPsxQwkWi3hdV8AWMxhAjA1V77hmC3cz5AJ4y2T58YGTHRwGQH7nNdqbcCGGjSiT3jLmNAXWkLNAb"},"slot":29}],"signature":"edsigtXomBKi5CTRf5cjATJWSyaRvhfYNHqSUGrn4SdbYRcGwQrUGjzEfQDTuqHhuA8b2d8NarZjz8TRf65WkpQmo423BtomS8Q"},{"protocol":"PtHangz2aRngywmSRGGvrcTyMbbdpWdpFKuS4uMWxg2RaH9i1qx","chain_id":"NetXnHfVqm9iesp","hash":"ooMuBKYnRQDinxA723DDiahcRgGvEixWQibiWSCz5i6vjqh4yp7","branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","contents":[{"kind":"endorsement_with_slot","endorsement":{"branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","operations":{"kind":"endorsement","level":476},"signature":"sigTrLEKTZTuXFMACnuuDB6WTXCkTDwiN7ScoXmieqtkE2i8rYZ3EatVuZ2bjmwXuTVChaupMYdmcnbZPK6cqN6sAMoirZy5"},"slot":10}],"signature":"edsigtXomBKi5CTRf5cjATJWSyaRvhfYNHqSUGrn4SdbYRcGwQrUGjzEfQDTuqHhuA8b2d8NarZjz8TRf65WkpQmo423BtomS8Q"},{"protocol":"PtHangz2aRngywmSRGGvrcTyMbbdpWdpFKuS4uMWxg2RaH9i1qx","chain_id":"NetXnHfVqm9iesp","hash":"onjX4HtCVsFf2NBt9taM48iYgz69KK4ikzx17devsCWFnin3HK2","branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","contents":[{"kind":"endorsement_with_slot","endorsement":{"branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","operations":{"kind":"endorsement","level":476},"signature":"sigWVkEX3EkH8jRnEodeeLz8DaUJhJMtazJxFNnqDjayBhMtNMddsjJx8th4fuPL4KBKTooJXsWCg4XodYW9RxV8XpuFxL5D"},"slot":3}],"signature":"edsigtXomBKi5CTRf5cjATJWSyaRvhfYNHqSUGrn4SdbYRcGwQrUGjzEfQDTuqHhuA8b2d8NarZjz8TRf65WkpQmo423BtomS8Q"},{"protocol":"PtHangz2aRngywmSRGGvrcTyMbbdpWdpFKuS4uMWxg2RaH9i1qx","chain_id":"NetXnHfVqm9iesp","hash":"opLuTVueeKBNapFBMSxDWPNuShVZcKfBpw3vjGrhE58ziNdVRFj","branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","contents":[{"kind":"endorsement_with_slot","endorsement":{"branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","operations":{"kind":"endorsement","level":476},"signature":"sigpSVXyP1FC43SR98z74usQZFkZprfgrBFrxvauxAbL3KRMSXX3YffBgpdgYzfmy9piBWNVkv8Y38jkJy7n7nr5Wx48BYkC"},"slot":27}],"signature":"edsigtXomBKi5CTRf5cjATJWSyaRvhfYNHqSUGrn4SdbYRcGwQrUGjzEfQDTuqHhuA8b2d8NarZjz8TRf65WkpQmo423BtomS8Q"},{"protocol":"PtHangz2aRngywmSRGGvrcTyMbbdpWdpFKuS4uMWxg2RaH9i1qx","chain_id":"NetXnHfVqm9iesp","hash":"ooZpwMzyoooGvTQjsUsVv54sck9c62jfMMe75abAV7iMA6jN43h","branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","contents":[{"kind":"endorsement_with_slot","endorsement":{"branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","operations":{"kind":"endorsement","level":476},"signature":"sigWQioijVuYs2wpR5QwtTRpSJH7iQ1E9ZnSM6B4JtPoQYA45gCkyoak8QePtfpw1m5DK79qza4A4RxmLiw7jXwFggEbuFam"},"slot":8}],"signature":"edsigtXomBKi5CTRf5cjATJWSyaRvhfYNHqSUGrn4SdbYRcGwQrUGjzEfQDTuqHhuA8b2d8NarZjz8TRf65WkpQmo423BtomS8Q"},{"protocol":"PtHangz2aRngywmSRGGvrcTyMbbdpWdpFKuS4uMWxg2RaH9i1qx","chain_id":"NetXnHfVqm9iesp","hash":"oo35Rmqbf1F91CJSzPMQz4gfgEn1RmRF14SxnFP2UZc49f6vyvR","branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","contents":[{"kind":"endorsement_with_slot","endorsement":{"branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","operations":{"kind":"endorsement","level":476},"signature":"sigeAb4XQZbu5Ciz1z3F4XKLGfqJV21RvTVFchJCu63ZFHhLBULnFRri3t8KRA8cnXq96BP5CvHWWu8naArDBKKKzWvMhwUC"},"slot":7}],"signature":"edsigtXomBKi5CTRf5cjATJWSyaRvhfYNHqSUGrn4SdbYRcGwQrUGjzEfQDTuqHhuA8b2d8NarZjz8TRf65WkpQmo423BtomS8Q"},{"protocol":"PtHangz2aRngywmSRGGvrcTyMbbdpWdpFKuS4uMWxg2RaH9i1qx","chain_id":"NetXnHfVqm9iesp","hash":"ooR9DxUFJAw7j2dvnAK1B4iy4g1K9B91vP8aDHWDaMvo4YxvGrW","branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","contents":[{"kind":"endorsement_with_slot","endorsement":{"branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","operations":{"kind":"endorsement","level":476},"signature":"sigbzH3y5bHtgqRKFJKGHEzqTYGmK3Py64Zsy1mWoyRaS8YoXANashS9mydAGSXEju9VnAN76VFm8n8WCS3GnDm2WeRJtztM"},"slot":50}],"signature":"edsigtXomBKi5CTRf5cjATJWSyaRvhfYNHqSUGrn4SdbYRcGwQrUGjzEfQDTuqHhuA8b2d8NarZjz8TRf65WkpQmo423BtomS8Q"},{"protocol":"PtHangz2aRngywmSRGGvrcTyMbbdpWdpFKuS4uMWxg2RaH9i1qx","chain_id":"NetXnHfVqm9iesp","hash":"opJm1cMaLnVeiybhUBZgRQSUfwtJhGGe7FdkFR7PTWmnAzWtbFi","branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","contents":[{"kind":"endorsement_with_slot","endorsement":{"branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","operations":{"kind":"endorsement","level":476},"signature":"sigUMUygZ12XFeMEPtJLKQ4ZZ1DjwdXwovBUojkLieArVn3VSsH7kQCfqSs4unoZfZqcPzWJEm6E8742ZLzMUnUHn7hkTpey"},"slot":35}],"signature":"edsigtXomBKi5CTRf5cjATJWSyaRvhfYNHqSUGrn4SdbYRcGwQrUGjzEfQDTuqHhuA8b2d8NarZjz8TRf65WkpQmo423BtomS8Q"},{"protocol":"PtHangz2aRngywmSRGGvrcTyMbbdpWdpFKuS4uMWxg2RaH9i1qx","chain_id":"NetXnHfVqm9iesp","hash":"oog5eTuFsjh6h3P7YEvHuQQzDb3hr6SmRBYUUXXbKpEmdoHwj6m","branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","contents":[{"kind":"endorsement_with_slot","endorsement":{"branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","operations":{"kind":"endorsement","level":476},"signature":"sighKTBdjJLakiDpMsL4Gz28dpndeWfQCypBSVitPPNSXvMfmopJtqQKCWkgMy8SyF5L6NAsoLqPUgkWoNTLZ79UrB3gQgTQ"},"slot":23}],"signature":"edsigtXomBKi5CTRf5cjATJWSyaRvhfYNHqSUGrn4SdbYRcGwQrUGjzEfQDTuqHhuA8b2d8NarZjz8TRf65WkpQmo423BtomS8Q"},{"protocol":"PtHangz2aRngywmSRGGvrcTyMbbdpWdpFKuS4uMWxg2RaH9i1qx","chain_id":"NetXnHfVqm9iesp","hash":"ooN77KzSAV86qqC5FuVmrRTyhy1Bk3BN3ShDU2nsSxYirtHrjVS","branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","contents":[{"kind":"endorsement_with_slot","endorsement":{"branch":"BLFFF5JZnnD3ygaCWfGFChzv3rnFzJAndLb2kKPdvksxD31dRT8","operations":{"kind":"endorsement","level":476},"signature":"signjkooTuzDLYPAQ6bnegm4LTANuGvLsrNKzmg9ABfFQoJHSrTWWFP14Jheg2k7cT4i6irsk4N253JybytnE8P2tyTcMYyj"},"slot":73}],"signature":"edsigtXomBKi5CTRf5cjATJWSyaRvhfYNHqSUGrn4SdbYRcGwQrUGjzEfQDTuqHhuA8b2d8NarZjz8TRf65WkpQmo423BtomS8Q"}],[],[],[]]}
io.circe.DecodingFailure$$anon$2: Attempt to decode value on failed cursor: DownField(metadata)


2022.04.01 10:36:40 [lorre-tezos-indexer-akka.actor.default-dispatcher-8] ERROR TezosIndexer    (tech.cryptonomic.conseil.indexer.tezos.TezosIndexer.processTezosBlocks:288) - Could not fetch blocks from client
io.circe.DecodingFailure$$anon$2: Attempt to decode value on failed cursor: DownField(metadata)


2022.04.01 10:36:40 [main] INFO  t.c.c.i.Lorre   (tech.cryptonomic.conseil.indexer.Lorre.shutdown:83) - Doing clean-up.

2022.04.01 10:36:40 [main] INFO  HikariDataSource (com.zaxxer.hikari.HikariDataSource) -  - Shutdown initiated...

2022.04.01 10:36:40 [main] INFO  HikariDataSource (com.zaxxer.hikari.HikariDataSource) -  - Shutdown completed.

Exception in thread "main" java.util.concurrent.TimeoutException: Futures timed out after [10 seconds]
	at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:259)
	at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:263)
	at scala.concurrent.Await$.$anonfun$result$1(package.scala:220)
	at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:57)
	at scala.concurrent.Await$.result(package.scala:146)
	at tech.cryptonomic.conseil.indexer.Lorre$.shutdown(Lorre.scala:84)
	at tech.cryptonomic.conseil.indexer.Lorre$.delayedEndpoint$tech$cryptonomic$conseil$indexer$Lorre$1(Lorre.scala:80)
	at tech.cryptonomic.conseil.indexer.Lorre$delayedInit$body.apply(Lorre.scala:21)
	at scala.Function0.apply$mcV$sp(Function0.scala:39)
	at scala.Function0.apply$mcV$sp$(Function0.scala:39)
	at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
	at scala.App.$anonfun$main$1$adapted(App.scala:80)
	at scala.collection.immutable.List.foreach(List.scala:392)
	at scala.App.main(App.scala:80)
	at scala.App.main$(App.scala:78)
	at tech.cryptonomic.conseil.indexer.Lorre$.main(Lorre.scala:21)
	at tech.cryptonomic.conseil.indexer.Lorre.main(Lorre.scala)
2022.04.01 10:36:50 [shutdownHook1] INFO  t.c.c.i.Lorre   (tech.cryptonomic.conseil.indexer.Lorre.shutdown:83) - Doing clean-up.

[jun0tpyrc]

Seeing similar things on mainnet

  About to start processing data on the mainnet network for tezos



2022.04.02 04:18:13 [main] INFO  TezosIndexer    (tech.cryptonomic.conseil.indexer.tezos.TezosIndexer.checkTezosConnection:111) - Successfully made initial connection to Tezos

2022.04.02 04:18:13 [lorre-tezos-indexer-akka.actor.default-dispatcher-8] INFO  TezosIndexer    (tech.cryptonomic.conseil.indexer.tezos.TezosIndexer.processTezosBlocks:240) - Processing Tezos Blocks..

2022.04.02 04:18:14 [lorre-tezos-indexer-akka.actor.default-dispatcher-8] ERROR TezosIndexer    (tech.cryptonomic.conseil.indexer.tezos.TezosIndexer.processTezosBlocks:288) - Could not fetch blocks from client
io.circe.DecodingFailure$$anon$2: Attempt to decode value on failed cursor: DownField(metadata),DownArray,DownField(contents),MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRi
ght,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRigh
t,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,MoveRight,DownArray,MoveRight,MoveRight,MoveRight,DownArray


2022.04.02 04:18:14 [main] INFO  t.c.c.i.Lorre   (tech.cryptonomic.conseil.indexer.Lorre.shutdown:83) - Doing clean-up.

2022.04.02 04:18:14 [main] INFO  HikariDataSource (com.zaxxer.hikari.HikariDataSource) -  - Shutdown initiated...

[piotrkosecki]

Hi, have you tried node 12.2 and 2022-march-release-41?

[piotrkosecki]

BTW first node does not look like it is in archive mode.

[piotrkosecki]

After our investigation it looks like tezos nodes 12.1 and 12.2 are not always returning metadata. Because of that Lorre breaks. The fix is to use node with --metadata-size-limit=unlimited parameter, but unfortunately you need to resync your node as metadata which is missing cannot be recovered.