Allow node operators to choose db of choice for eth events logs

[kasteph]

Checklist

• This is not a new feature or an enhancement to the Filecoin protocol. If it is, please open an FIP issue.
• This is not a new feature request. If it is, please file a feature request instead.
• This is not brainstorming ideas. If you have an idea you'd like to discuss, please open a new discussion on the lotus forum and select the category as Ideas.
• I have a specific, actionable, and well motivated improvement to propose.

Lotus component

• lotus daemon - chain sync
• lotus miner - mining and block production
• lotus miner/worker - sealing
• lotus miner - proving(WindowPoSt)
• lotus miner/market - storage deal
• lotus miner/market - retrieval deal
• lotus miner/market - data transfer
• lotus client
• lotus JSON-RPC API
• lotus message management (mpool)
• Other

Improvement Suggestion

Lotus currently stores event logs in an embedded SQLite database. As a consequence for Lily, we would either have to adapt the work of @iand in Lily or continuously call the Lotus API.

I would like to request that node operators be able to specify in the config.toml or via an environment variable a connection string to the relational database of their choice.

[f8-ptrk]

support for that. postgres would be nice

[f8-ptrk]

another complication that "only sqlite" could bring in the future is size. there is no proper way to delegate sqlite to remote systems afaik

[i-norden]

Another issue with sqlite that we have been running into is when trying to backfill the msgindex, using the shed command, on an archive that is currently running (and also writing to the msgindex.db). Since sqlite only allows a single writer, if (when) these two processes try to obtain the write lock at the same time whichever process tries to obtain the lock second throws an error

ERROR   lotus-shed      lotus-shed/main.go:137  failed to insert message cid bafy2bzacedfjnjyf7jjqiy5m7xf6ttmymciarxbzhfblmezswkbkskodkzjxm in tipset bafy2bzacebtngwpymbuc6w6wttidvksw74guszpn5674bj2evdv5pdm6ccvmw at epoch 2545675: database is locked

This issue would also likely be addressed in an alternative way by #11007

One might suggest that the backfill command should only be ran with the node off, but turning an archive off for as long as it takes to complete the backfill (a few weeks by our estimates based on how long it ran and how far it got before running into that error) is not practical for multiple reasons.

I am also suspecting that sqlite is causing some other issues with the eth API when trying to make a lot of concurrent queries to endpoints such as eth_getLogs. I haven't finished exploring this yet but will provide more context when possible.