server: background data api loading

[chappjc]

A dex host that has been up for just several months with a handful of markets can take more than a minute to start as it waits to load.

Here is a very tolerable example with younger markets:

2023-02-16 01:15:44.407 [INF] DEX: Preparing historical market data API for market dcr_btc...
2023-02-16 01:15:49.488 [DBG] ASSET[eth][ETH]: Tip change from 8499031 to 8499032.
2023-02-16 01:15:51.900 [DBG] DB: select epoch candles in: 7.492611697s
...
2023-02-16 01:15:51.914 [INF] DEX: Preparing historical market data API for market ltc_btc...
2023-02-16 01:15:56.565 [DBG] DB: select epoch candles in: 4.650463052s
...
2023-02-16 01:15:56.573 [INF] DEX: Preparing historical market data API for market dcr_ltc...
2023-02-16 01:15:59.459 [DBG] DB: select epoch candles in: 2.885920879s
...
2023-02-16 01:15:59.482 [INF] DEX: Preparing historical market data API for market dcr_bch...
2023-02-16 01:16:02.216 [DBG] DB: select epoch candles in: 2.733269363s
...
2023-02-16 01:16:02.227 [INF] DEX: Preparing historical market data API for market dcr_eth...
2023-02-16 01:16:05.126 [DBG] DB: select epoch candles in: 2.898609128s

It adds up with more markets and it grows with time (more epochs stored).

Two thoughts this PR is investigating:

• does this historical data really need to be loaded into data API for the markets to start operating (accepting orders and negotiating swaps, esp. active ones)?
• can doing these concurrently add any performance improvement?

I started this branch an in August 2022, but never got to finishing it, partly because I forgot. This is here as a placeholder and for comments.

The other obvious question is how can this loading be optimized, and what is the hotspot. I did profile at one point and it's mostly in the Go DB code that's binning the data. PostgreSQL and/or disk to not seem to be much issue.

[buck54321]

The hard-coded 0 at

dcrdex/server/db/driver/pg/epochs.go

Line 119 in 3cb388a

rows, err := a.db.QueryContext(ctx, stmt, 0)

is curious. I don't know that we need to scan through every epoch. We should be able to set a reasonable limit on the start time based on the bin size and candles.CacheSize.

[chappjc]

The 24hr bin size is hard to get around. Just getting a hundred or so bins is still bad.
We need to cap that though, you're right.

[buck54321]

We could also cache day candles in the db and only calculate the current day from epochs, like we're doing with a lot of stuff in dcrdata.

[chappjc]

converted to issue #2315