Measure size of current ledger with respect to claimable balances

[ire-and-curses]

We need an estimate of the current ledger state:

• Size (in GB)
• Number of ledger entries
• Breakdown of ledger entries by type

We want to know the current memory requirements for captive core, and how much headroom we have if ledger size continues to increase.

[bartekn]

Would be great to have metrics on this in /metrics (ex. we can get latest number of ledger entries via state verifier so that no new code is needed).

[bartekn]

OK, number of ledger entries is now tracked by metric in #4015. Moving this back to Current Sprint to continue working on size of DB.

[bartekn]

I was confused: this is not related to Postbgres DB size but RAM size. This can be hard to do. If we want to get memory usage of Stellar-Core we'd need to execute system command and I don't think a breakdown per ledger entry is possible without Stellar-Core help (like exposing metrics).

[jcx120]

Let's start with DB disk size per ledger so we can reliably extrapolate growth of storage needs.

[ire-and-curses]

I'm confused by this @bartekn.

It seems to me that
i) we know the current total RAM usage of captive core
ii) we know almost all of that RAM is holding current state
iii) we are ingesting this data so we must know what the breakdown of ledger entries is on the Horizon side

So why can't we answer this question? If (for example) RAM is 10GB and 3/4 of the ledger entries are claimable balances, then claimable balances are responsible for 7.5GB. What am I missing?

[bartekn]

i) we know the current total RAM usage of captive core

We can't access this data from Go code directly (maybe using exec on a system command is a solution but I'm not sure - this is system dependent).

ii) we know almost all of that RAM is holding current state

Probably but there's also Overlay and some other components. I can agree that it's probably 90%+ but I don't know if it's 90% or 99%.

iii) we are ingesting this data so we must know what the breakdown of ledger entries is on the Horizon side
If (for example) RAM is 10GB and 3/4 of the ledger entries are claimable balances, then claimable balances are responsible for 7.5GB. What am I missing?

The problem here is the ledger entries are not equal in size. A data entry can be much smaller that claimable balance with a bunch of claimants. If the estimated values are OK then we can probably calculate it this way but we still don't have access to RAM usage of CC from Horizon code (see: i)).

[bartekn]

Closing because better memory analysis requires exposing more metrics in Stellar-Core. However, given we probably need to move CC to disk such metric is probably no longer needed.