bitcoin-cli gettxoutsetinfo returns empty set

[photis]

19:31 ~/bin/statoshi $ bitcoin-cli gettxoutsetinfo
{
}
19:40 ~/bin/statoshi $ 

Notice how it takes 9 minutes to process that request.
I got to this because the utxoset gauges were missing in my Grafana installation. Running the latest Statoshi version.

[jlopp]

Hm, I know there was a bug in Bitcoin Core that caused this endpoint to underreport the UTXO set statistics but it was fixed and merged into Statoshi a few months ago.

9 minutes is a really long time for the call - it should be more like 10 - 20 seconds. Are you running on low end hardware?

[photis]

Thanks for getting back to me.
As I tried to say (English is not my mother tongue), I didn't see the utmost gauges in Influxdb (or Grafana for that matter) so I went looking in the source for the location where they were generated. I found them in the routine that also displays gettxoutsetinfo in bitcoin-cli. So I tried that. With the empty set as a return result (and a suspiciously long time to process that request). All other info requests through bitcoin-cli return practically instantly.

For your information, I downloaded Statoshi like:

sudo git clone https://github.com/jlopp/statoshi.git

2 days ago. I'm running it on a Raspberry 2, after I had run the regular Bitcoin Core full node for 2 weeks without notable problems (side-loaded the blockchain though from my Macbook Pro, where it had been kept up-to-date for months).

I don't think there is a problem with Influxdb (I realise that's the difference with your setup with Graphite). All the other measurements from Statoshi are passed fine. The fact that bitcoin-cli chokes on on gettxoutsetinfo (no result) must mean something, no?

[jlopp]

My first concern is that it's a Raspberry Pi - I've never tried running Statoshi on that hardware, though I do have some RPi nodes at home. I can try running that call on them to see if Bitcoin Core has the same problem. If you have a Bitcoin Core binary for RPi, I'd also suggest running it and seeing if the same issue occurs when you make the bitcoin-cli call to get the UTXO set statistics.

[photis]

Okay, I will make a new Bitcoin Core binary and run that for comparison. I'll get back to you when I'm done. Don't bother with the RPi at home (yet) ;-).
ttyl

[photis]

Sorry for the delay, I was called away.
In the mean time, I rebuild the Bitcoin Core binary (v0.11.2 cloned from github) and it gives me the same result. Again, as I said before, every single other RPC call works fine, just not gettxoutsetinfo. That goes for the Bitcoin Core version and for the Statoshi version.
But, if I'm the only one with this problem, I'd suggest we let it rest.

[jlopp]

You've confirmed my suspicion that the problem is not Statoshi-specific, so the question now is why does the call not return anything? My first guess is that there's a timeout being triggered - I see that the bitcoin-cli RPC client sets a default timeout of 900 seconds - are you sure that you're getting the empty response after exactly 9 minutes and not actually 15 minutes? If it's 15 minutes then this is likely the answer. If it is 9 minutes there there may be a bug or a timeout elsewhere and it could be worth opening an issue in Bitcoin Core with all of this information.

You're the first person who has told me that they are running Statoshi on a Raspberry Pi, so you're probably the first to find this particular performance issue. It's unfortunate, because even if it is possible to fix the timeout by overriding it (you can pass "-rpcclienttimeout=$SECONDS" to bitcoin-cli during the utxoset request) then it sounds like it's going to take more time to calculate the stats than it will take, on average, for the next block to come in, which means the machine will likely be perpetually under a high load if you want to update the stats for each block.

[photis]

After I started the standard Bitcoin Core daemon, it took 7 minutes (see below) to return.

23:00 pi@raspinode:~ $ bitcoin-cli getinfo
{
    "version" : 110200,
    "protocolversion" : 70002,
    "blocks" : 393197,
    "timeoffset" : 0,
    "connections" : 13,
    "proxy" : "",
    "difficulty" : 113354299801.47113037,
    "testnet" : false,
    "relayfee" : 0.00005000,
    "errors" : ""
}
23:10 pi@raspinode:~ $ bitcoin-cli gettxoutsetinfo
{
}
23:17 pi@raspinode:~ $

There are quite a few people running a Bitcoin full node on Raspberries. And I'm probably at a disadvantage as I only have a narrow pipe to the internet (ADSL (7MB/0.8MB) in the woods of the Hunsrück Nature Reserve in Rhineland-Palatinate, Germany). I haven't examined the source yet but I suspect that it throttles the number of connections to accommodate the memory (1GB) constraints and low bandwidth. In any case it runs very smoothly and averages on 29-32 connections, without burning the cpu of the RPi 2. Memory is kept at a steady 80-90%. I plan on shipping off the Bitcoin Node to a co-location site with a bigger pipe, and probably add a few more if it works out well.

[photis]

I did a few more, each code block is copied verbatim.
This one took 7 minutes as well.

20:05 pi@raspinode:~/bin/bitcoin.orig $ bitcoin-cli gettxoutsetinfo
{
}
20:12 pi@raspinode:~/bin/bitcoin.orig $ bitcoin-cli getinfo
{
    "version" : 110200,
    "protocolversion" : 70002,
    "blocks" : 393193,
    "timeoffset" : 0,
    "connections" : 12,
    "proxy" : "",
    "difficulty" : 113354299801.47113037,
    "testnet" : false,
    "relayfee" : 0.00005000,
    "errors" : ""
}
20:17 pi@raspinode:~/bin/bitcoin.orig $

Another one at 8 minutes

20:38 pi@raspinode:~ $ bitcoin-cli gettxoutsetinfo
{
}
20:46 pi@raspinode:~ $

[photis]

Btw, since I have your attention, may I suggest that you put the IP address of the Statsd server in a config file instead of hard-coded in the source? It would make it a whole lot more flexible imho.

[jlopp]

7 minutes, eh? In that case I'm not sure why this is occurring. I'd actually recommend you open an issue in Bitcoin Core and link to this issue. The one thing I will note about the UTXO set call is that it's incredibly disk and CPU intensive.

Bitcoin nodes can (currently) run fine on Raspberry Pi hardware, though as you noted it can't handle more than several dozen peer connections due to the limited memory. However, during normal operations a bitcoin node never makes the UTXO set call.

I'll open a separate issue regarding the statsd configuration; thanks.

[jlopp]

Sounds like there's no further action for me to take here, so I'll close the issue and hope they fix it upstream.